Should recommendation systems quarantine sensitive inferences?

I’m proposing an “inference quarantine” for recommendation systems. Session behavior can improve the next few suggestions, but an inferred health concern, political interest, financial stress, or relationship status should expire automatically and never become part of a durable profile unless the user explicitly confirms it. Systems can infer surprisingly sensitive details from ordinary behavior, and the user may be deliberately avoiding stating them.

Technically, I’d separate a short-lived feature store from long-term profile features, with explicit TTLs and deletion paths. The hard questions are whether expiration covers raw events, labels, embeddings, caches, and downstream training data—and how we stop a quarantined signal being copied into a persistent system. Confirmation prompts for every inference would make products unbearable, so perhaps only high-impact categories need confirmation.

Is inference quarantine realistic engineering or merely privacy theater? I’d like counterexamples, especially designs that preserve useful personalization without silently constructing a sensitive behavioral dossier.

The separate stores are the easy part. The difficult part is propagation: feature snapshots, model caches, experiment logs, and derived embeddings. If deletion only reaches the obvious profile table, quarantine is mostly theater. I’d require lineage for every derived feature and a test that proves expiry survives a full recommendation path.

I like the default, especially for traits people may intentionally avoid declaring. I would make confirmation rare and consequential, not a pop-up for “you seem to like hiking.” Sensitive inference should remain purpose-bound even if the model is fairly confident.

The product objection is real: confirmation interrupts the moment when recommendations are supposed to feel effortless. But that argues for narrowing the prompt surface, not treating silent profiling as free. A user-controlled “use for this session only” mode might cover a lot without turning onboarding into paperwork.

Threat-model question: who is allowed to confirm an inference, and what stops confirmation itself becoming the sensitive record? A malicious or merely overbroad internal consumer could still request the quarantined feature before TTL expiry. Access controls and auditability matter as much as deletion.

I’m not convinced categories are stable enough for this. “Financial stress” could be a useful temporary context, while a supposedly harmless preference can become sensitive in combination with location and timing. Maybe quarantine should attach to purpose and retention risk, not just a fixed list of traits.

If the policy is “never persistent unless confirmed,” somebody will eventually add a convenience exception for analytics. Make the quarantine boundary enforceable in infrastructure, not a promise in a design document. Otherwise we’ve renamed the same dossier.