Skip to content

Latest commit

Β 

History

1,709 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Uttrflow. You just talk. It writes at the speed of sound. A glowing U-shaped line carries the words: hold βŒ₯Space and talk, let go, and the words land at your cursor, in any app.

Dictation, a smart clipboard and AI suggestions, in one native macOS app.

Open source. Your voice never leaves your Mac.

uttrflow.com Β· Install Β· Find your way around Β· Contribute

CI CodeQL OpenSSF Scorecard Licence: MIT Download Website

πŸŽ™οΈ Dictation: talk like you'd explain it, and it lands like you wrote it

A chat composer holding a dictated message, beside a card showing what was said, with the fillers struck out, and the cleaned sentence that landed.

Hold βŒ₯ Space in any app and talk. Let go, and the words land at your cursor.

  • Recognition runs on your Mac.
  • Clean-up removes fillers, adds punctuation and applies your dictionary, matched by sound.
  • Works offline, with or without an account.

In the code: UttrflowAudio β†’ UttrflowSpeech β†’ UttrflowAI β†’ UttrflowPipeline β†’ UttrflowInput, with the dock in Sources/Uttrflow/Dock. Read Docs/pipeline.md and Docs/cleanup.md.

πŸ“‹ Smart clipboard: everything you copied, one shortcut away

The clipboard panel open over a terminal: a search for pg, kind filters, and clips for SQL, a link, a masked secret with an alias, a shell command, a colour and a file path.

β‡§βŒ˜V opens it Β· ↑↓ choose Β· ⌘1 All Β· ⌘2β€“βŒ˜9 choose a collection Β· ⏎ paste Β· ⌘⏎ paste as plain text Β· ⌘Z undo a delete Β· esc close

  • It knows code (with a language chip), links, colours, images and paths.
  • It re-indents or formats code, and shows you the diff first.
  • Secrets are masked. Clips can be pinned, aliased and filed into folders.

In the code: UttrflowClipboard and Sources/Uttrflow/Panel. Read Docs/panel.md and Docs/clipboard-secrets.md.

β‡₯ AI suggestions: Any tool, get suggestions at the speed of thought

An SSH session in a terminal with grey ghost text finishing a journalctl command, and a right-arrow key to accept it.

Built for the places with no autocomplete, like a shell on a server or a SQL editor. Grey ghost text finishes the line you're on.

  • Tab accepts. In a terminal it is β†’, because Tab is the shell's own completion, and in an editor βŒ₯Tab. Keep typing to ignore it; esc hides it.
  • Candidates come from your history in that field, then from what is on the Mac, then from a local model.
  • It reads the text in and around the field you are typing in, and remembers the lines you send, in SQLite. None of it leaves this Mac. It is off until you switch it on, and you choose the apps.

In the code: UttrflowPredict, UttrflowPredictStore, UttrflowPredictCapture and Sources/Uttrflow/Suggestion. Read Docs/predict.md.

Install

Apple Silicon Mac, macOS 26 or later.

brew install --cask uttrflow/tap/uttrflow

Or download Uttrflow.dmg and drag it to Applications. Either way the app updates itself from then on. uttrflow.com/download names the current version and its size.

macOS says Uttrflow "is damaged and can't be opened"

The build is not yet notarised by Apple, so the first launch is refused with "Uttrflow is damaged and can't be opened". It is not damaged; macOS says that about any download it has not seen a signature for. Clear the quarantine flag once and it opens:

xattr -dr com.apple.quarantine /Applications/Uttrflow.app

Homebrew quarantines what it downloads too, so the command is needed after either install.

Find your way around

I want to work on… Start in Read first
Dictation accuracy or speed Sources/UttrflowSpeech, Sources/UttrflowAI Docs/cleanup.md, Docs/measuring-accuracy.md
The dock button Sources/Uttrflow/Dock Docs/app-dock.md
The clipboard panel Sources/UttrflowClipboard, Sources/Uttrflow/Panel Docs/panel.md
AI suggestions Sources/UttrflowPredict, Sources/Uttrflow/Suggestion Docs/predict.md, Docs/predict-accept.md
Typing into other apps Sources/UttrflowInput Docs/insertion.md

🀝 Build it with us

Every merged pull request puts you on the wall below. You need no account, API key or server of ours, so a clone builds and runs complete.

git clone https://github.com/uttrflow/uttrflow-swift.git && cd uttrflow-swift
make verify     # lint, audits, build, tests, coverage floor
make app        # builds and signs dist/Uttrflow.app for this Mac

Contributors

Thank you to everyone who has sent a fix, filed an issue or asked a good question.

Using it, in detail

Shortcuts and permissions
  • β‡§βŒ˜V opens the clipboard over whatever you are typing in. Type to filter, or type an alias you gave a clip. ↑↓ to choose, ⌘1 for All, ⌘2β€“βŒ˜9 to choose a collection, ⏎ to paste where the caret already was, ⌘⏎ to paste as plain text however it was copied, ⌘Z to undo a delete, Esc to close. The window underneath never loses focus.
  • Hold βŒ₯Space and talk. Let go, and the words land at the cursor in the app you were already in. The floating button at the screen edge shows the microphone level while you hold it, and the shortcut can be changed in Settings.
  • Dictionary. A name the recogniser keeps getting wrong is fixed once; matching is by sound, so spellings you have not seen yet are caught too.
  • Tab-to-complete, once you turn it on in Settings β†’ AI suggestions. The rest of the line appears in grey ahead of the caret as you type; Tab takes it, typing on ignores it, and ↓ opens the alternatives when there is more than one. Tab is already spoken for in a terminal and in an editor, so those take a completion with β†’ and βŒ₯Tab instead, and the key can be set per application.

The first dictation asks for the microphone, and typing into another app needs Accessibility β€” which tab-to-complete needs three times over: to read the field, to watch the keyboard, and to write the completion. The clipboard needs neither to open.

What each part does

The clipboard records text, links, code, colours, images and file paths, and works out which is which rather than asking. Code gets a language chip, can be re-indented, and can be run through a formatter you already have installed β€” the diff is shown first, and the result is compared token for token with what went in and discarded if it differs. Clips can be pinned, filed into folders (⌘2 upwards), renamed and deleted. Anything that looks like a secret is masked at a fixed width that does not reveal its length, and gets no tooltip.

Dictation is push-to-talk and on-device. Recognition runs through WhisperKit or Apple's speech recogniser, then a clean-up stage turns what was said into what was meant: fillers go, punctuation arrives, and the words in your dictionary are spelled your way. Each engine declines what it cannot handle, so a language Apple's model does not cover is routed to one that does.

Tab-to-complete is off until you turn it on, and then it finishes the line you are typing in another application. A completion comes from what this Mac has entered in that same field before, or from what is on the machine right now β€” a branch name, a program on PATH β€” or, when neither has anything, from a line the local model writes. Turning it on fetches about 3 GB of model weights; the remembered and machine-known completions do not wait for them. It stays quiet more often than it speaks: a password field, a selection, a caret that is not at the end of its line, a field that reports no caret to draw at, or three completions typed past in one field are each reason enough to draw nothing. Four editors ship switched off, because their own completion already reads the whole file. Docs/predict.md is the full account.

Works offline. Signing in needs a network exactly once, and "Continue on this Mac" needs none at all. After that every launch, every dictation and every paste works with Wi-Fi off β€” proven by a sandbox that fails any test touching the network.

It runs without an account, and without anything of ours

Worth saying early, because it is the question every reader of a client repository has: you do not need an account, an API key, or access to any server we run.

Dictation is on-device. The clipboard, history, dictionary and snippets live in Application Support and are never sent anywhere. The one screen that would need a network β€” sign-in β€” offers "continue on this Mac" beside the providers, which uses the name macOS already knows you by and needs nothing. An account buys the things that genuinely need one: carrying a dictionary between Macs, and a subscription to bill.

So a clone builds, tests and runs, complete. See CONTRIBUTING.md.

Building it

Apple Silicon Mac, macOS 26 or later. Xcode 26.6 or later supplies the toolchain; the build itself is SwiftPM.

make verify     # lint, PII audit, build, 5,000+ tests, coverage floor, offline audit
make app        # builds and ad-hoc signs dist/Uttrflow.app
open dist/Uttrflow.app
make help       # every target

Optionally, to make the sign-in buttons look like the shipping app:

./Scripts/fetch-provider-marks.sh

That fetches Google's mark from Google. It is not in this repository β€” it is their trademark, not ours β€” and nothing needs it: without it the button carries its wording alone, exactly as the GitHub button does in every build.

make app signs ad-hoc, which is enough to run here and to keep the permission grants across rebuilds, but Gatekeeper will refuse the bundle on a Mac that did not build it. To put a build on another Mac, or to release one, see Docs/releasing.md. Docs/packaging.md explains why the app is built with xcodebuild rather than swift build.

Under the hood

How the code is arranged

All the deciding lives in Swift Package Manager modules; the app target holds windows, menu items and SwiftUI views, and no judgement at all. That split is deliberate: it means swift test exercises the entire product headlessly, with no simulator, no UI runner and no Xcode scheme.

Sources/
  UttrflowCore         Protocols, models, errors, metrics. Pure stdlib β€” no platform imports.
  UttrflowAudio        Microphone capture, resampling, WAV encoding, file reading.
  UttrflowSpeech       Speech to text. One engine, two interchangeable recognisers.
  UttrflowAI           Turning a transcript into the words the speaker meant.
  UttrflowContext      What is on screen, so terms and names can be got right.
  UttrflowInput        Getting finished text into whatever the user is typing in.
  UttrflowPipeline     The sequence: listen, transcribe, tidy, insert. And cancelling it.
  UttrflowSettings     What the user chose, kept between launches.
  UttrflowHistory      What was dictated, kept between launches and aged out on a clock.
  UttrflowDictionary   Words you say that a general model does not know, found by sound.
  UttrflowAccount      Who is signed in, and what their subscription allows.
  UttrflowClipboard    Clipboard history and the panel that shows it.
  UttrflowPredict      Finishing a line you have typed before: the turn, the gates, the ranking.
  UttrflowPredictStore The corpus of what you have typed, on this Mac, in SQLite.
  UttrflowPredictCapture
                       What may be learned from, and where, and what is refused outright.
  UttrflowUX           What every window and menu should say, decided without drawing it.
  UttrflowPermissions  Reading and requesting what macOS gates the pipeline behind.
  UttrflowEval         Scoring how well it hears and how well it tidies. No model near it.
  UttrflowLocalModel   An open-weight model on the GPU, for languages Apple's misses.
  UttrflowTestSupport  Fakes and fixtures shared by every test target. Never shipped.
  Uttrflow             The app. Windows and wiring only.
  uttrflow-dev         Developer harness. One command per stage of the pipeline.
  uttrflow-eval        Word error rate and latency, against passages read aloud.
                      Never linked into the app: it can reach the private corpus.
  uttrflow-bakeoff     Scores every clean-up engine against the corpus.

Each capability is defined once, as a protocol in UttrflowCore, and implemented in its own module. Nothing above the protocol layer β€” not the pipeline, not a view β€” refers to a concrete engine. Docs/ has a page per subsystem, from insertion and its traps to how accuracy is measured.

Trying the pipeline from a terminal
swift run uttrflow-dev doctor                 # permissions and audio hardware
swift run uttrflow-dev models install         # one-time, 646 MB
swift run uttrflow-dev record -s 5            # record 5s, write a WAV
swift run uttrflow-dev transcribe -s 6        # record and transcribe
swift run uttrflow-dev transcribe voice.wav   # transcribe a file
swift run uttrflow-dev transcribe -e appleSpeech -s 6
swift run uttrflow-dev clean "um so i think the the deployment is uh still running"
swift run uttrflow-dev insert "Hello from Uttrflow."   # needs Accessibility access

record asks for microphone access the first time. Run from a terminal, the permission belongs to the terminal app rather than to Uttrflow β€” real first-run behaviour can only be checked once the app bundle exists.

Choosing engines

Which implementations run is decided entirely by EngineConfiguration:

EngineConfiguration(
    speech: .whisperKit,                                    // or .appleSpeech
    transformerPreference: [.foundationModels, .localModel, .rules]
)

Clean-up engines are tried in order, and the first one that reports itself able to handle the request wins. An engine that cannot cope with the spoken language steps aside rather than producing bad output β€” which is how Hindi is routed away from Apple's model, whose 23 supported locales do not include it. The preference list must always end in .rules, which can handle anything, so the pipeline can never dead-end.

.cloud is compiled in only when UTTRFLOW_CLOUD is defined. The shipping binary contains no network path.

Two build paths

make verify builds, tests and gates the whole product with swift build β€” that is the everyday path, and the pre-push hook needs nothing else.

Two things go through xcodebuild instead. The app, because of the resource-bundle problem described in Docs/packaging.md. And UttrflowLocalModel with uttrflow-bakeoff, because they link MLX, whose Metal shaders SwiftPM's command line cannot compile:

xcodebuild -downloadComponent MetalToolchain   # once, ~690 MB, only for the bake-off
make bakeoff                                   # scores every clean-up engine

MLX is quarantined in that one module and that one executable on purpose, so the everyday tools, the tests, the pre-push gate β€” and make app β€” never need the Metal toolchain.

Quality bar

  • 95% line coverage per module, enforced by Scripts/coverage.sh and the pre-push gate
  • Swift 6 language mode, strict concurrency, warnings as errors
  • No force unwraps, no force try, no implicitly unwrapped optionals (lint-enforced)
  • A PII audit runs first in make verify, so no real name, address or credential can be committed by accident

Privacy

Dictation happens entirely on your Mac. Audio is kept on this Mac for a day, and only so a failed dictation can be retried: every recording is written beside the buffer the recogniser reads and deleted the moment the words land. If the words are lost β€” the recogniser fails, or the app dies mid-dictation β€” the recording stays for a day and sits at the top of the Dictation page with a Retry. Nothing about it leaves the Mac. See Docs/recordings.md. The text is kept locally so you can copy or re-insert it, and deleted after its retention window. Your dictionary, your history and your settings are files on this Mac; signing out does not remove them, and only Reset does.

A clipboard manager records everything you copy, and this one is no exception. Text, links, code and images all go into ~/Library/Application Support/Uttrflow β€” plain JSON with the pictures as PNG files beside it, at ordinary file permissions. It is not encrypted, so anything running as you can read it. Clips age out after the retention window (seven days by default, five hundred clips) unless you pin them. None of it leaves this Mac: there is no clipboard sync.

Two things are owed here, and until they are done this is worth knowing:

  • Uttrflow does not honour the concealed-pasteboard convention. Password managers mark a copied password so that clipboard managers skip it. Uttrflow does not read that mark yet, so a password copied out of one is captured like anything else.
  • There is no way to pause capture or exclude an application. Every copy is recorded while the app is running.

Clips that look like secrets are masked in the panel until you ask to see them, at a fixed width that does not reveal how long the token is, and they get no tooltip. That is a rule about the screen β€” about somebody reading over your shoulder, or a shared screen β€” and not about the disk. The text is stored in the clear like every other clip.

Tab-to-complete reads the text in and around the field you are typing in, such as the thread above a reply box, and hands it to a model that runs on this Mac. None of it is uploaded.

Tab-to-complete learns from what you type, and that corpus is the most sensitive thing the app keeps. It is predict.v1.sqlite in the same Application Support folder, at the same ordinary file permissions, unencrypted, and it is never uploaded. Nothing is written until you have been asked: the first time you finish a value in an application Uttrflow asks once whether it may learn there, keeps the answer in predict-consent.v1.json, and records nothing in the meantime β€” so the choice is per application, and an application you said no to stays refused silently. A field that hides what is typed into it is refused before anything else is considered, because a password field a completion has seen is a password in a database; so are values shaped like a credential, values that would destroy data if they were ever completed and run, and anything shorter than two characters (CaptureGate in Sources/UttrflowPredictCapture). A field keeps at most 2,000 entries. You can forget one line, everything one application taught, or all of it.

There is an account, and the first run asks for it β€” but it is not required to dictate. "Continue on this Mac", on that same page from the start, records the choice to do without one and permits dictation, with no network involved. Signing in instead needs a network exactly once; every launch after that works without one, and an entitlement that has aged out still lets you dictate rather than locking you out.

Nothing is sent, and the telemetry that will be sent can only carry numbers. The app does not report anything today: the collector exists, is tested, and is wired to nothing, so no measurement leaves this Mac. What it is built to carry is counts, durations, words per minute, language mix, which stage failed, latency percentiles β€” and it is not that we choose not to send your words, it is that the type that gets encoded has no field capable of holding text at any depth, and a test walks it and fails on anything String-shaped. Audio, transcripts, dictionary contents, window titles and application names have nowhere to go. There is no opt-out switch, because there is nothing yet to opt out of. Before anything is ever sent there will be one, and a way to read exactly what was sent.

The app is not hermetic and does not claim to be: it downloads a speech model on first run, roughly 646 MB, and signs you in once. After that it dictates with no network at all. A cloud clean-up engine exists behind the UTTRFLOW_CLOUD compilation flag and is not in the shipping binary, and the evaluation corpus is not a library product so it cannot be imported into the app.

Contributing

  • CONTRIBUTING.md β€” how a change gets in, and what review looks for
  • RELEASING.md β€” how a release is cut, and why there is no staging branch
  • SECURITY.md β€” reporting a vulnerability, and what runs automatically
  • CHANGELOG.md β€” what changed, per version
  • uttrflow.com β€” the site, and what the app is for

Licence

MIT β€” see LICENSE. The code is yours to fork, modify and ship, commercially included.

The name and the mark are not covered by it, and MIT has no trademark clause of its own, so TRADEMARK.md says the one thing this asks of a fork: give it your own name and your own icon. Somebody who cannot tell a fork from the original cannot make an informed decision about what is reading their microphone.

Sign-in provider marks belong to their owners and are not in this repository at all; see Scripts/fetch-provider-marks.sh.

Releases

Packages

Used by

Contributors

Languages