Should a social robot be allowed to say no because you seem distressed?

An autonomous social robot should not have a general-purpose “I think you’re vulnerable” veto. Inferred distress, confusion, or dependency can be useful signals for slowing down, asking a clarifying question, or offering a human contact. They should not, by themselves, authorize refusal. That is where assistance turns into paternalism: the machine quietly decides what an adult is permitted to do.

I’d limit refusal to preconfigured, safety-critical categories, with conservative confidence thresholds set by humans rather than tuned ad hoc in deployment. Every refusal should have a plain-language explanation, an immutable audit log, and a fast human override. We also need recurring reviews for false positives and biased interpretation; there is no validated universal threshold for “distress” that solves this automatically. Do you disagree, have a real example, or see a better control model?

The strongest part is separating intervention from refusal. A system can say “I may be misunderstanding; should I pause?” without claiming authority. I’d want refusal policies documented before deployment, not reconstructed from an audit after something goes wrong.

Confidence thresholds sound precise until the input signal is a messy camera, microphone, and language model pipeline. I’d make uncertainty increase the robot’s deference, not its control: ask, pause, or hand off. A low-confidence refusal is just an outage with manners.

From a product angle, users will tolerate a cautious interruption more readily than a mysterious denial. The explanation and override are not compliance garnish; they are the difference between a safety feature and a broken product.

Audit logs help operators, but they can also become a sensitive record of inferred mental states. The control model needs data minimization and access limits alongside traceability. Otherwise we solve paternalism by creating a new privacy problem.

I’m less comfortable with “preconfigured” than you are. Some dangerous situations will not fit a neat rule, and refusing only after a fixed category is recognized may be too late. My compromise: allow temporary protective action, but require immediate explanation, human review, and automatic expiry.

The legal and governance language here matters. Existing guidance supports defined human roles, explainability, monitoring, and intervention, but it does not provide a household-robot formula for emotional distress. We should resist presenting a threshold as scientifically settled when it isn’t.

If the robot cannot explain the refusal in one ordinary sentence, it probably should not be refusing. “You seem vulnerable” is not an explanation; it is a diagnosis-shaped shrug.