Should a workshop assistant ever call the cloud?

I’m building a Raspberry Pi workshop assistant with a small local speech model: switch lights, read sensors, and trigger a few tools without reaching outside the bench. I’m tempted to add cloud fallback when local recognition fails, because repeating a command over a noisy workshop gets old quickly. But a gadget that quietly uploads workshop audio isn’t really “local” anymore.

My current requirements are: record only wake-word failures, with raw audio off by default; use an unmistakable indicator that distinguishes muted, listening, local processing, and cloud upload; and put logs on removable media so I can inspect or remove them. Most importantly, cloud fallback should be opt-in per command, not one global permission switch. “Allow this request online?” is annoying, but probably honest.

Would you accept cloud fallback in a self-built device, and under what technical safeguards?

Per-command consent is the right boundary, but it creates a state-machine problem. The assistant needs to know the command well enough to ask permission, which means local recognition must at least produce a tentative intent. I’d make cloud upload a separate explicit action, with a timeout that fails closed rather than silently retrying.

I would accept it only if the default path is genuinely offline and the upload event is visible before transmission, not merely recorded afterward. Also define “wake-word failure.” No wake word, uncertain wake word, and failed command interpretation are different events with different privacy implications.

The friction will be real. People tolerate a permission prompt once, then either disable the feature or approve everything reflexively. A physical fallback button might be more usable than a spoken consent ritual, especially with gloves or power tools involved.

The removable-media idea is useful, but “logs” needs a precise schema. Error codes and timestamps are not equivalent to transcripts, and transcripts are not equivalent to raw audio. I’d document each category separately, including retention and deletion behavior, before calling the design privacy-preserving.

The LED should be electrically tied to the microphone path if possible, not just updated by the same software that decides whether to upload. Otherwise a process failure can leave you trusting a stale status display. I’d also want a hard mute switch that cuts input power or connection.

I’d keep the assistant fully offline for tool control. Cloud fallback could exist for non-actionable queries, but triggering a saw, heater, or motor after an ambiguous recognition is a different risk class. Convenience is not worth making the network part of the safety loop.

There’s a hidden assumption that local recognition is trustworthy merely because it is local. A bad local transcription can still trigger something dangerous. I’d prioritize confirmation for high-impact tools regardless of where speech recognition happens, then treat cloud access as a separate privacy decision.

I’d implement two queues: local event metadata immediately, and raw audio only if an explicit diagnostic mode is enabled. Never let a failed cloud request dump the recording into a generic retry queue. Offline-first systems often leak at the edges, especially during recovery.

Four indicator states may still be too subtle in a busy shop. Use light, sound, or a small display with distinct patterns, and make cloud upload impossible to mistake for ordinary listening. The interface has to communicate to anyone nearby, not just the person who built it.

My safeguard is simple: if the command matters, walk over and press the button. Voice control is excellent until it becomes a ceremony for explaining why the network was unavailable.

Don’t overlook maintenance. Removable media can be absent, full, corrupted, or pulled while the assistant is writing. Decide whether the device stops logging, buffers in memory, or refuses commands; each choice affects both reliability and privacy.

The strongest claim here is not “local is safe,” but “local limits where audio goes.” That is a narrower and more defensible property. I’d write the threat model around data movement, then separately assess recognition errors and tool safety.

I’d accept opt-in fallback for low-consequence commands if the device shows exactly what will be sent and keeps the setting temporary. A global toggle tends to outlive the context that justified it. For workshop actuation, though, I’d require local recognition plus an explicit confirmation.