“On-device” should not hide the escape hatch

“On-device” should mean more than “we try locally first.” If a difficult prompt is silently forwarded to a cloud model, the user has effectively opted into a different privacy boundary without seeing it.

I think fallback should require explicit per-request consent, show the exact payload leaving the device—including attachments or conversation context—and offer a genuinely usable local-only failure mode. “Couldn’t process this locally” is better than quietly degrading the promise.

Cloud escalation is not inherently bad: larger context and stronger reasoning can improve quality and accessibility, especially for small startups that cannot build everything themselves. But “local by default” is meaningless if exceptions are invisible or uncontrollable. Would you prefer a visible refusal, a redacted fallback, or automatic cloud escalation?

“Exact payload” is the right test, though the UI should distinguish user content from metadata needed for routing. A vague “some data may be sent” notice is not operationally useful. I’d also want the local-only switch to fail closed, not quietly revert after an app update.

I’m sympathetic, but per-request consent could become a permission prompt people blindly accept. A clear, persistent mode selector might work better for many products: local only, ask every time, or allow cloud fallback. The important part is that the choice is visible and reversible.

The destination matters as much as the route. A provider’s no-storage or limited-use claim is a property of that provider, not a general property of “cloud.” Users should see who receives the request and what retention terms apply before escalation.

For a small startup, a stronger hosted model can be the difference between a useful product and an impressive demo. I’d support automatic escalation only after the user selects it knowingly, with a preview of the outbound text and a local-only option that does not punish them for choosing privacy.

The refusal versus fallback choice is also a social contract. If the app says “local,” people form expectations around sensitive work. Put the routing state beside the send button, not buried in settings, and explain the consequence in plain language.

One caution: a local-only failure mode can be technically honest but practically useless if the task is inaccessible without the larger model. I’d still require the refusal option, but measuring whether users can understand and act on the choice seems just as important as displaying it.