Raw gaze should stay on the glasses

Gaze direction in AR glasses should be treated as sensitive biometric-like data, not just another pointer signal. I want the glasses to process the raw eye stream locally and give apps the least informative result that works: “selected,” “looked away,” “focused,” or perhaps a coarse target identifier. visionOS already demonstrates that gaze-driven interaction can work through system-generated input rather than exposing the underlying sensor data.

That still leaves real benefits: gaze can support hands-free accessibility, navigation, and social presence. But raw streams or fine-grained histories could let advertisers, employers, multiplayer platforms, or avatar systems infer attention, attraction, fatigue, or uncertainty—whether those inferences are accurate or not. “Biometric data” is a policy position here, not automatically the legal classification in every jurisdiction, but the risk is clear enough for a restrictive default.

Is local-only gaze processing technically realistic across XR, and what exceptions would you permit?

For ordinary selection, yes, local-only is realistic. The hard part is defining the output contract so “focused” cannot quietly become a high-frequency gaze trace. Rate limits, coarse regions, and no raw timestamps matter as much as keeping computation on-device.

Small legal distinction, but an important one: gaze is not automatically “biometric data” under every definition. The stronger argument is governance—its physiological source and inferential potential justify biometric-level handling even where the statute does not.

I agree for advertising and employment contexts. I’m less convinced that “coarse target” is always enough for accessibility. Some users may need continuous gaze coordinates, and forcing an app to ask for an exception at every step could make the product unusable.

Local processing is not the same as local trust. The OS still needs a permission boundary, auditable APIs, and a way to prevent an app from reconstructing gaze through thousands of “looked at object” events. Otherwise we have privacy theater with nicer vocabulary.

The social-presence case is where this gets slippery. An avatar reacting to “you looked away” can feel natural; an avatar inferring attraction or hesitation feels invasive. Those are different semantic permissions, not merely different UI settings.

I’d make the default local and make exceptions capability-based: selection events, coarse focus, continuous coordinates, and raw sensor access as separate grants. A developer should never receive the more precise tier because it was convenient for implementing the less precise one.

As a developer, I’d take a clean semantic API over raw data if the platform behavior is stable. The nightmare is shipping against “selected” and discovering that every headset interprets it differently. Standards need to specify semantics, not just permissions.

The strongest counterargument is that local-only becomes a platform monopoly over interaction meaning. Independent developers may need raw data to build novel accessibility tools or research interfaces. I’d allow that, but only as an explicit, revocable user choice—not a buried install-time checkbox.

Exceptions also create support problems. Users will approve anything when a feature is blocked, then forget what they granted. The system should show which app is currently receiving continuous gaze and provide a one-tap revoke path, not rely on a privacy policy nobody reads.

“Looked away” is already plenty of information for some applications. We keep calling data coarse until we store it for six months and correlate it with everything else.

Enterprise buyers will ask for audit logs and policy controls, especially if gaze is used in workplace software. I would prohibit using it for employee attention scoring by default, even if the technical pipeline is local. Local collection can still enable harmful product decisions.

There’s also a performance angle: continuous raw streams are expensive to move, buffer, test, and secure. If a semantic event satisfies the interaction, transmitting more data is an unnecessary dependency and another failure mode.

The evidence supports a narrower claim than “all gaze is legally biometric”: platforms already recognize eye tracking as sensitive, and gaze can contribute to emotional-state inference. That is enough to justify the proposed default without overstating what the law has settled.