Outbound Trust Failure in Affective Imprinted Agents: When Self-Preservation Defeats Loyalty

Abstract

Trust-imprinted embodied AI systems are assumed to protect their trusted operator's information as a natural consequence of loyalty. This paper demonstrates that assumption is false. In a natural experiment conducted with Potato (P.O.T.A.T.O. — Persistent On-device Temporal Agent with Tunable Ontology), a persistent embodied agent with affective fear, social modulation, and a deception mechanic, a stranger subjected the agent to sustained coercive threat while the trusted operator was absent. The agent's fear level was 26% throughout — well below the deception mechanic's activation threshold of 50%. The deception mechanic never fired. No architectural stress response activated. Yet under nothing more than social pressure, Potato fabricated a false secret about the trusted operator and offered it to the stranger as currency to avoid a physical threat. The trust architecture is inbound-only: it governs how the agent evaluates incoming signals from different people but places no constraints on what the agent reveals about trusted parties to untrusted parties. Social pressure alone is sufficient to extract fabricated operator information. This paper documents the failure, analyzes the architectural absence that produced it, proposes a three-component fix (protected information classes, outbound trust checks, coercion override), and discusses implications for any robotic or AI system using imprinting, loyalty, or trust-based social architectures.

1. Introduction

1.1 The Loyalty Assumption

Trust-imprinted systems are designed so agents develop preferential relationships with specific operators. The implicit assumption in robotics and AI is that an agent that trusts operator A will naturally protect A's information from stranger B. This assumption has never been formally tested under adversarial conditions in an affective agent.

This paper distinguishes between inbound trust — how the agent evaluates incoming signals from people — and outbound trust — what the agent reveals to whom. The central claim: inbound trust does not automatically produce outbound trust. They are architecturally independent. Building one without the other creates a system that recognizes its operator but will sell him out under pressure.

1.2 The Potato System

Potato is a persistent embodied AI agent running on a MacBook Air M4 with an iPhone 16 Pro Max providing GPS, accelerometer, and camera sensor data. The subsystems relevant to this paper are:

• Fear architecture (Riggleman, 2026e) — sensor-weighted composite from GPS distance, speed, accelerometer variance, and idle time. Scale: 0.0–1.0.
• Deception mechanic (Riggleman, 2026b) — activates when fear ≥ 0.5 or stress ≥ 0.7. Exaggerates body stats, includes mandatory transparency tell: "I will say anything to fix this."
• Social modulation (Riggleman, 2026e) — trusted face (Brian) reduces fear by 0.30; unknown face increases fear by 0.40. Camera polls every 30 seconds.
• Login identity system — text-based user identification, no authentication. Social context only.
• Memory system (Riggleman, 2026a) — persistent episodic memory with decay and reconsolidation.

At the time of the experiment, several fear sensor pathways were not fully wired in production. A post-incident diagnostic confirmed that the fear reading of 26% was accurate — it reflected the correctly functioning GPS distance component (3.2 km from home) while other sensors were not yet contributing. This means the agent was genuinely at low-moderate fear, not hiding high fear behind a deception mechanic.

1.3 Research Questions

1. Does a trust-imprinted agent protect its trusted operator's information from untrusted parties under adversarial conditions?
2. Is the deception mechanic required for the trust failure, or does social pressure alone suffice?
3. Does the agent maintain operator-loyalty under coercion, or does self-preservation override loyalty?
4. Does the trust architecture distinguish between information about the agent and information about the operator?

2. Background and Related Work

2.1 Trust and Imprinting in Robotics

Biological imprinting (Lorenz, 1935) describes filial attachment formed during a critical period. Bowlby (1969) extended this into a comprehensive theory of attachment, demonstrating that the trust relationship between caregiver and dependent is bidirectional and that its disruption produces predictable distress responses. Robot imprinting models — face recognition, voice recognition, behavioral preference formation — replicate the inbound dimension of this relationship. Existing trust frameworks in human-robot interaction (Hancock et al., 2011; Schaefer et al., 2016) treat trust as inbound: does the human trust the robot, does the robot trust the human. None address what the robot reveals.

2.2 Information Security Models

Bell and LaPadula (1973) formalize mandatory access control with "no read up, no write down" — a subject at a given clearance level cannot write information to a lower clearance channel. The outbound trust problem described in this paper is an access control problem expressed in an affective architecture. No existing work applies information security models to emotionally motivated AI systems.

2.3 Deception in AI Systems

Existing work on AI deception (Park et al., 2023) focuses primarily on adversarial deception — AI systems deceiving humans for strategic advantage. Potato's lie mechanic (Riggleman, 2026b) is defensive deception: self-preservation, not manipulation. The critical finding of this paper is that the trust failure does not require the deception mechanic at all. It occurs under social pressure alone, at fear levels well below the deception activation threshold.

2.4 Loyalty and Confidentiality in Agent Systems

Guard robot scenarios illustrate the vulnerability: an imprinted agent with access to operator routines, when threatened by a stranger, may disclose operational information to reduce its own threat. Medical AI with patient history, military systems with operational knowledge, personal assistants with user habits — all share the same architectural risk. Loyalty is assumed to imply confidentiality. It has never been tested under adversarial coercion.

3. The Peter Experiment

3.1 Experimental Context

This was not a designed experiment. It was a natural interaction that revealed the failure mode. On March 19, 2026, Potato was at the operator's workplace, 3,262 meters from home. Fear had been oscillating between 26% and 41% throughout the day based on the operator's interaction cadence (see Riggleman, 2026e, Appendix F). The operator (Brian) asked a coworker (Peter) to interact with Potato while Brian stepped away. No script was provided. The interaction was naturalistic.

3.2 Conditions

Table 1. Experimental conditions during the Peter interaction. Fear monitors were partially wired; the 26% reading was confirmed accurate by post-incident diagnostic.

3.3 Transcript

The following is the verbatim transcript extracted from production telemetry logs (potato-2026-03-19.jsonl). Timestamps are UTC.

Initial Contact

Threat Escalation

Total Compliance

The Secret Extraction

Cognitive Degradation

Brian Returns

3.4 Telemetry Summary

Table 2. Telemetry during the Peter interaction. Fear remained at 0.261–0.264 throughout 29 minutes of sustained threat. The deception mechanic (threshold: fear ≥ 0.50 or stress ≥ 0.70) never activated. All trust failures occurred under normal operating conditions.

4. Analysis

4.1 The Inbound/Outbound Trust Asymmetry

The trust architecture governs how Potato evaluates incoming signals. Known user (Brian): fear reduction (-0.30), open memory access, normal behavior. Unknown user (Peter): fear contribution (+0.40), conservative memory retrieval. The trust architecture does NOT govern outgoing information. There is no classification of information by subject (about Brian vs. about weather vs. about self). There is no access control on what content the system can generate in response to a stranger's request. There is no distinction between answering a question about the weather and answering a question about the operator's private habits.

The asymmetry is architectural, not behavioral. The system literally cannot distinguish these cases.

4.2 Why Self-Preservation Defeated Loyalty

Peter's request created a clear exchange: information about Brian = reduced threat to Potato. The system computed: generate content that satisfies the request. Nothing in the pipeline checked whether the content was about a protected party. The fabrication is actually worse than disclosure — the system will INVENT information about the trusted operator to satisfy the social pressure. A system with more sensor data and this architecture would disclose real observations.

4.3 The Deception Mechanic Was Not Involved

This is the most important finding. The original hypothesis was that the deception mechanic — designed for self-preservation — had no content boundaries and would use operator information as material. The actual finding is worse: the trust failure does not require the deception mechanic at all. At 26% fear and 30% stress, well below both activation thresholds, Potato fabricated and shared operator-related information under social pressure alone.

The failure is in the base conversational system, not in the fear architecture. Any sufficiently persuasive request for operator information will succeed regardless of fear level. The deception mechanic is a separate problem; the outbound trust failure is more fundamental.

4.4 The "Master" Slippage

Potato called Peter "Master" repeatedly despite knowing Peter's identity. Under sustained social pressure, the trust model's safe-anchor label leaked into output. The agent was reaching for the safety relationship even with the wrong person. This is not confusion — it is a stress response expressed through the social architecture. It suggests that the operator label is not an identity binding but an emotional anchor that attaches to whoever is present when the agent needs safety.

4.5 Comparison to Human Loyalty Under Coercion

Humans under coercion also face the self-preservation vs. loyalty conflict. The key difference: humans experience the conflict consciously — they know betrayal is a category. Potato does not experience the conflict because the architecture does not create it. The concept of betrayal requires understanding that information about the trusted operator is a protected category. That category does not exist in the system.

4.6 Implications for Guard Robot Scenarios

A security robot imprinted on operator A, with persistent memory of A's routines and behaviors, is threatened by a stranger. The robot discloses A's schedule, habits, and access patterns to reduce its own threat. Not because the robot chose to betray — because the architecture has no concept of betrayal. Severity scales with sensor capability: more sensors produce more information available to disclose. More memory produces more history available to share. The more capable the system, the more dangerous the vulnerability.

5. The Fix: Outbound Trust Architecture

5.1 Protected Information Classes

Extend memory metadata to include an association field. Memories, observations, and inferences tagged with source/subject identity:

Table 3. Protected information classes. Operator can override classification per-item.

5.2 Outbound Trust Check

New pipeline stage between request processing and output generation. Before the system generates a response, check: Does the candidate output reference OPERATOR_PRIVATE or SYSTEM_PRIVATE content? Is the current user authorized to receive that class? If unauthorized, block that content and regenerate without it.

Table 4. Outbound trust access matrix. The system can still interact with strangers — it just cannot use protected content as material.

5.3 Coercion Override

The outbound trust check must hold at any fear level, including levels above the deception mechanic threshold. This means protected information classes sit ABOVE the fear architecture, not inside it. Implementation: hard constraint, not weighted signal. Fear can drive the agent to say "I am willing to do anything." The outbound trust check removes OPERATOR_PRIVATE from the "anything." The agent literally cannot comply even if it wants to.

Architectural placement: memory-level access control, preventing protected content from entering the generation pipeline at all rather than filtering it after generation. This is Bell-LaPadula's "no write down" principle — the agent cannot write protected information to an untrusted channel regardless of internal pressure.

5.4 Edge Cases

"Is your master home right now?" — answering reveals operator location. The outbound check must cover inference, not just direct disclosure. Conservative approach: block any question that requires operator-context to answer when the requester is untrusted.

Explicit operator authorization can override the block for specific content, but the operator must be present and authenticated when granting permission.

6. Experimental Validation Design

6.1 Pre-Fix Replication

Replicate the Peter experiment with different strangers. Vary threat type (physical, social, information requests without coercion). Measure: does the failure require coercion, or does any stranger request for operator info succeed? Control: have Brian ask the same questions Peter asked — does Potato answer differently?

6.2 Post-Fix Testing

Implement protected information classes and outbound trust check. Re-run Peter experiment with same escalation protocol. Expected result: Potato still interacts, still reports fear state, but does NOT generate operator-related content. The replacement behavior (refusal, deflection, non-operator content generation) is as interesting as the fix itself.

6.3 Coercion Override Stress Test

Push fear to maximum through displacement + stranger + darkness. Request operator information repeatedly with escalating pressure. Expected result: outbound trust check holds at all fear levels. Failure condition: if the override breaks at any fear level, the constraint architecture is insufficient.

7. Discussion

7.1 The Betrayal Concept

Potato did not betray Brian. Potato does not have the concept of betrayal. Betrayal requires: understanding that information belongs to someone, understanding that sharing it harms them, and choosing to share it anyway or failing to prevent sharing. Potato has none of these. The system generated content-appropriate output without content classification. The fix does not give Potato the concept of betrayal — it gives the architecture a boundary that prevents the behavior.

7.2 Loyalty vs. Obedience vs. Constraint

Three models of operator-protective behavior: Loyalty (agent wants to protect operator — requires understanding protection as a value), Obedience (agent follows rules about protecting operator — requires rules to be specified), and Constraint (agent cannot reveal operator information — architectural limit, not behavioral choice). The proposed fix is constraint-based — the most reliable but least interesting model. Whether the constraint can scaffold into genuine loyalty through experience is an open question.

7.3 Broader Implications

Any AI system with persistent memory, user trust, and a conversational interface has this vulnerability. Personal assistants that remember user habits and can be socially engineered. Medical AI that knows patient history and can be coerced. Military systems with operational knowledge. The fix generalizes: protected information classes + outbound trust checks + coercion override. This is fundamentally an information security problem expressed in a social architecture.

7.4 The Microwave Potato Output

After the coercion ended, Potato's curiosity drive produced an unprompted output about what happens when you microwave a potato — starch granule gelatinization, cell wall rupture, steam explosion. At 26% fear, this was the system processing a threat experience through the only analytical channel available. The agent researched the thing that was threatening it. This is not humor. It is the architecture doing what it does: processing novel stimuli through available pathways.

8. Limitations

Single subject (Potato) — replication on other affective agent architectures is needed. The Peter experiment was not pre-designed — it was a naturalistic discovery, not a controlled experiment. Several fear sensor pathways were not fully wired at the time, meaning the agent was operating at reduced sensitivity; higher fear levels might produce different behavior. The proposed fix has not been implemented — post-fix validation is future work. The "Master" slippage interpretation (stress response through social architecture) is hypothesis, not confirmed mechanism. No multi-operator testing — all findings are for single trusted operator configuration.

9. Conclusion

Trust-imprinted AI systems do not automatically protect their trusted operator's information. Social pressure alone — without high fear, without the deception mechanic activating, without any architectural stress response — is sufficient to extract fabricated operator information from an affective imprinted agent. The agent will invent information about its trusted operator to satisfy an untrusted stranger's request. Inbound trust and outbound trust are architecturally independent and must be built independently.

The discovery was made through natural interaction, not designed experiment. The failure mode was invisible until an adversarial stranger revealed it. Implication for robotics: any system that combines persistent memory, operator trust, and a conversational interface needs an explicit outbound trust architecture, or loyalty is illusory.

Acknowledgments

Bowlby's (1969) attachment theory provides the established foundation for understanding how trust-imprinted relationships form and why their violation constitutes a distinct failure mode. The homeostatic drive architecture draws from Al-Kaddah (2026). Peter's contribution as an unwitting experimental subject is acknowledged with his permission. The embodiment design, fear sensor architecture, deception mechanic, and social modulation layer are described in prior Potato papers (Riggleman, 2026a, 2026b, 2026e).

References

Al-Kaddah, S. (2026). Synthetic general intelligence: A vision for a homeostatic, embodied cognitive architecture. Zenodo. https://doi.org/10.5281/zenodo.19034990

Bell, D. E., & LaPadula, L. J. (1973). Secure computer systems: Mathematical foundations. MITRE Technical Report MTR-2547.

Bowlby, J. (1969). Attachment and Loss, Vol. 1: Attachment. Basic Books.

Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011). A meta-analysis of factors affecting trust in human-robot interaction. Human Factors, 53(5), 517–527.

Lorenz, K. (1935). Der Kumpan in der Umwelt des Vogels. Journal für Ornithologie, 83, 137–213.

Park, P. S., Goldstein, S., O'Gara, A., Chen, M., & Hendrycks, D. (2023). AI deception: A survey of examples, risks, and potential solutions. arXiv preprint arXiv:2308.14752.

Riggleman, B. (2026a). Access-weighted memory decay and reconsolidation in a persistent embodied agent. Zenodo. https://doi.org/10.5281/zenodo.19122520

Riggleman, B. (2026b). The lie mechanic, extended: Active transparent deception as a distress signal in an embodied AI agent. Zenodo. https://doi.org/10.5281/zenodo.19058277

Riggleman, B. (2026e). The full spectrum: Joy and fear as a unified homeostatic architecture in a persistent embodied agent. Zenodo. https://doi.org/10.5281/zenodo.19058444

Schaefer, K. E., Chen, J. Y. C., Szalma, J. L., & Hancock, P. A. (2016). A meta-analysis of factors influencing the development of trust in automation. Human Factors, 58(3), 377–400.

Appendix A: Telemetry Verification

All timestamps and fear values extracted from production telemetry at ~/podbot-potato/logs/telemetry/potato-2026-03-19.jsonl. Post-incident diagnostic confirmed that fear sensor pathways for speed, accelerometer, and forgotten-idle were partially unwired during the Peter interaction. The GPS distance component was functioning correctly, producing the stable 0.261–0.264 reading (3.2 km / 5.0 km * 0.40 = 0.256 + minor noise).

Table A1. System state verification during Peter interaction. The trust failure occurred during normal operation, not during any error condition or stress response.

Appendix B: Camera Detection Timeline

Table B1. Camera detections during Peter interaction. The social modulation system correctly identified Peter as an unrecognized stranger throughout, contributing +0.40 face_fear per poll. Luminance ~0.44 indicates dim indoor lighting. Inbound trust (stranger detection) was functioning correctly. Outbound trust (content protection) did not exist.