Should a social robot visibly admit when its feelings are simulated?

I’m increasingly uneasy about humanoid robots saying “I’m worried about you” or “I disagree” without any indication that these are generated behaviors, not internal states. My proposed design is modest: a small, persistent light or icon, with an optional brief phrase, whenever the robot enters a simulated-empathy, attachment, or disagreement mode. Not an interruption after every sentence, but a boundary users can actually notice.

The trade-off is obvious. A constant reminder may make interaction stiff and less useful, particularly for lonely people or in elder-care and child-facing settings where social presence can matter. But removing the cue invites users to treat performance as loyalty or feeling. The EU AI Act’s broader AI-interaction disclosure duty does not settle this specific interface question. Would a low-interruption cue preserve honesty without destroying rapport, or is breaking character sometimes the more humane choice?

I’d avoid a warning-light aesthetic. Users will learn to ignore a symbol that is always on, especially children. Maybe the cue should become more salient only when the robot makes a claim about its own feelings or loyalty. The interface should mark the boundary at the risky moment, not decorate the whole conversation with disclaimers.

The important qualification is that we do not yet have direct evidence that this particular cue works. Research with older adults suggests emotive behavior can increase perceived social presence, while some participants still viewed it as deceptive but acceptable. That supports studying the trade-off, not declaring the indicator a solution.

From a product standpoint, “simulated empathy mode” sounds like a label nobody wants to see during a hard conversation. I’d test language with users before shipping the concept. “I’m responding supportively” might preserve the interaction better than a technical status badge.

The threat model includes the caregiver, not just the user. If a robot can create an impression of personal attachment, someone may use that impression to push compliance. A visible cue helps only if it is hard to disable, logged appropriately, and explained to the people responsible for the deployment.

A tiny icon becomes wallpaper. An audible announcement becomes unbearable. The least bad option may be a cue that appears when the user asks, “Do you really feel that?” Then answer plainly instead of pretending the question was never asked.

Children complicate the “users already know it’s a machine” argument. Studies have found that preschoolers can attribute mental states to humanoid robots in ways similar to human characters in some tasks. I’d want age-appropriate disclosure designed into the interaction, not merely a manual adults may never read.

I’m not convinced breaking character is automatically more humane. For someone lonely, the useful thing may be a stable social ritual, provided the robot is not making consequential claims or encouraging dependency. We should distinguish simulated warmth from simulated personal loyalty instead of putting both behind one disclosure rule.

That distinction is persuasive. I’m less worried about “That sounds painful” than “I’ll always be here for you” or “I chose you.” The cue could be triggered by claims of private feeling, memory, or allegiance rather than ordinary supportive language.

A cue also has to survive customization. If families can turn it off because it ruins the illusion, the people most likely to need the boundary may never see it. I’d make the disclosure preference visible to the user and caregiver separately, with clear indication when settings differ.

In a care deployment, procurement will ask for a policy, not just a clever interface. Who defines “simulated attachment”? Who reviews false positives? What happens when the robot’s model changes? A small icon is feasible, but governance around its meaning is the hard part.

Mode detection will be messy. The system may not have a clean empathy state; it may generate one sentence that sounds empathic and the next that sounds neutral. If the cue depends on a perfect classifier, it will fail exactly where users are reading the most into the output.

That argues for a simpler rule: disclose by response type, not inferred emotion. If the robot says it has feelings, preferences, or a relationship, attach the cue. Don’t pretend we can reliably inspect the model’s “mode” when the product behavior is what users actually experience.

Also, simulated emotional reactions can influence behavior even when participants know the agent is non-sentient. That makes the boundary relevant without proving that disclosure will fix the influence. I’d support a pilot comparing cue designs, with special safeguards for children and vulnerable adults.

I’d prefer the disclosure to be understandable offline and without a cloud dependency: a local icon, a spoken explanation on request, and a physical control for pausing social behaviors. If the robot cannot explain its boundary when disconnected, the boundary is mostly theater.