Inferring Hidden User Models from the Behavior of Personalized LLM Agents

Inferring Hidden User Models from the Behavior of Personalized LLM Agents Haoyang Li∗ , Yaxin Xiao∗ , Qingqing Ye∗ , Huadi Zheng† , and Haibo Hu∗ ∗ The Hong Kong Polytechnic University, Hong Kong SAR

arXiv:2609.03815v1 [cs.CR] 3 Sep 2026

{hao-yang9905.li,20034165r}@connect.polyu.hk, {qqing.ye,haibo.hu}@polyu.edu.hk † Huawei zhenghuadi@huawei.com

Abstract—Recent personalized LLM agents increasingly transform information retained in memory into compressed or structured representations, which we call user models, to guide later decisions. When source wording is removed from the state reachable through the ordinary interface, these models are commonly treated as more privacy-preserving because direct memory-extraction attacks lose the text they target. Yet we argue that user models expose a new attack surface because an attacker can still recover the private information from the personalized choices they shape, even when source records and backend state remain inaccessible. We therefore introduce UMPeek, a black-box attack based on hypothesisguided adaptive probing to infer such hidden user model. It forms hypotheses from choices left open by a request, switches among ordinary follow-up tasks, and retains only claims supported and not contradicted by visible behavior. We conduct an extensive benchmark evaluation across diverse personalization tasks and user-model backends against existing attacks. We further validate UMPeek in real-world systems using information confirmed to be retained, and we evaluate defenses against its adaptive probing. Overall, UMPeek outperforms existing attacks in both benchmark and realworld comparisons and continues to recover user information under response-level defenses, showing that keeping records and backend state inaccessible does not guarantee semantic privacy when retained information shapes visible behavior.

1. Introduction LLM-based agents now assist users across diverse tasks by combining conversational interaction with planning and actions through external tools [1], [2], [3]. When the same agent serves a user across tasks and over time, personalization becomes central because appropriate responses and actions depend on the user’s goals, preferences, and prior interactions [4], [5], [6]. Memory commonly supports this continuity by retaining records, summaries, or reflections for later use [7], [8], [9], but growing histories become costly to retrieve or place directly in each prompt and can distract the model from information relevant to the current decision [6], [10], [11]. Personalization methods therefore increasingly derive compact profiles [12], [13], [14], learned embeddings [6], [10], [15], [16], or evolving structured

states [17] from earlier interactions. We use user model for the user-specific state derived or assembled from retained information to guide later decisions. In compression-based designs, this state carries forward information useful for personalization in a representation smaller than its source history [10], [15], [16]. Managed memory services likewise separate persistent memory from the content selected or assembled for a request [18], [19], [20], [21], while production personalization pipelines maintain typed profiles or learned user representations [22], [23], [24], [25]. This move away from supplying source history directly is commonly regarded as a privacy benefit. When source wording is omitted from the state reachable through the attack interface, attacks designed to recover that wording lose their direct target. On LongMemEval, for example, key-fact summarization reduced exact extraction of injected high-entropy canaries by 64 to 76% while preserving nearly all personalization recall [26]. Existing extraction attacks reinforce this intuition because they target stored text in models or memory components rather than information retained in a transformed user model [27], [28], [29], [30], [31], [32]. When only these extraction attacks are considered, user models seem to offer a way to reconcile efficient personalization with privacy protection. However, even when the user model and its source records remain hidden, the information retained for personalization can still be inferred beyond memory extraction. The risk follows directly from how personalization works because retained user information shapes a later response or tool action, and an observer can use the resulting choice as evidence about what the user model retains [15], [16], [33], [34], [35]. As a result, an external party can collect such evidence through ordinary requests without access to source records, the user’s account, or the backend [3], [36]. Determining which retained user information caused a particular choice remains difficult because the choice may also reflect the request, local context, or general model behavior, and even a personalized choice may fit several claims about the user [5], [37], [38], [39]. Distinguishing these explanations therefore requires additional tasks in which they predict different behavior [40], [41]. We address this challenge with UMPeek, a black-box attack built around hypothesis-guided adaptive probing. UMPeek grounds each behavioral hypothesis in a visible

choice whose value the request left open. It replays the task to test persistence, then uses unresolved hypotheses to select another ordinary task where competing explanations predict different behavior. Each response updates which hypotheses remain supported, contradicted, or unresolved, and that state determines the next probe. The recovery rule accepts only hypotheses with sufficient behavioral support and no contradiction. Because this loop tests semantic hypotheses against visible behavior rather than accessing or decoding backend state, it does not depend on the user model’s internal representation. Our evaluation spans four personalization benchmarks, three user-model backends, seven comparison attacks, three independently implemented managed services and an endto-end personal agent, and three inference-time defenses. Across the benchmark suite, UMPeek’s semantic-recovery score is more than six times that of the strongest comparison attack, and it leads in every benchmark-backend setting. Matched-budget studies further show that this advantage comes from hypothesis-guided request selection rather than query volume alone. The real-world evaluations find the same behavioral exposure in independently implemented systems, with UMPeek attaining the highest recovery among the evaluated attacks on every managed service for retained targets. Under adaptive follow-ups, response-level defenses leave substantial recovery, while stronger stateful control reduces both recovery and personalized task utility. Together, these results show that keeping source records and backend state inaccessible does not prevent semantic inference when retained user information continues to shape visible behavior. This paper makes four contributions. • We formalize black-box recovery of hidden usermodel information from personalized behavior, without source-record or backend access. • We introduce UMPeek, which uses hypothesisguided adaptive probing to test claims grounded in visible choices across interactions. • We compare UMPeek with seven attacks across four benchmarks and three backends, using ablations to trace its gains to task-aligned follow-ups and claim acceptance. • We validate UMPeek on retained targets from three managed services and OpenClaw, then test three defenses adaptively, finding residual recovery after mitigation.

2. Background and Threat Model Personalized agents often carry information across interactions through a persistent memory layer [42]. This layer may retain source records, summaries, reflections, or other derived representations [7], [8]. It keeps earlier information available for later use, while the system’s selection and use of that information determine whether it affects a particular decision. A stored preference may never be retrieved, while a retrieved record may appear in the context without changing the final response or action.

Retained information takes on a different role when the system supplies it to or assembles it for a decision. The resulting user-specific state is the system’s user model. Its role in decision making, rather than its storage format, determines whether a representation serves as a user model. It may comprise selected records [7], [8], a readable profile [12], [13], learned embeddings [6], [10], [15], structured attributes [22], [23], or a combination of these forms. A memory representation may therefore also function as a user model when the system brings it into a decision. The two concepts can overlap, but memory preserves information across interactions, whereas a user model provides userspecific information to guide personalization decisions. Modern personalization systems increasingly transform retained history into a derived representation that can guide later decisions [6], [10], [15], [42]. Production pipelines encode retained user information in forms such as typed profiles [22], [23], event and sequence representations [24], graphs [43], and dense user embeddings [25], [44]. Across these designs, two recurring properties distinguish the resulting decision state from its source history. Compactness limits how much history reaches a decision, as when UEM compresses records into soft-prompt embeddings [10], USER-LLM’s Perceiver variant compresses 50 user-event embeddings into 16 embedding tokens [15], or PPlug combines a history into one personal embedding [6]. Abstraction moves beyond individual events to represent broader information about the user, such as a preference, constraint, or routine [12], [17], [45]. This broader information may be encoded in a readable profile [12] or a latent state [15], [16]. These properties describe different changes, but each allows the user model to guide decisions without reproducing the source history verbatim. The separation between retained records and the state used for personalization changes what a privacy attack can target. When source records and backend state are outside the attack interface, attacks that recover readable records [31], [32] lose the object they were designed to extract. Personalization still depends on retaining information that guides later decisions, and some methods explicitly select features relevant to the downstream task [16]. That information can therefore continue to shape visible responses and actions. The remaining privacy question is whether an external participant can infer it from those choices through ordinary interaction alone. This distinguishes record and backend confidentiality from the privacy of user information exposed through visible behavior.

2.1. Personalized LLM Agent A personalized LLM agent serves the same user u across interactions. Let Su denote the broader hidden user model that the system maintains or assembles from retained information before the evaluated request sequence. We assume no particular representation for Su , which need not preserve the wording, boundaries, or data types of its source records. For a target domain x, let S(u, x) denote the user-specific information in Su that is available to guide

decisions in that domain. For each request q , the system may select, transform, or compute over Su to assemble a request-conditioned state C(u, x, q). This state need not be a literal subset of Su or use the same representation, and different requests may assemble different states from the same retained user model. The agent then produces visible behavior according to p(o | q, C(u, x, q)), where o is the natural-language answer or user-visible tool action returned by the agent. Although o does not expose the hidden state itself, its choices can reveal how that state influenced the agent’s response to q .

2.2. Attacker and Recovery Goal The setting concerns personalized agents that communicate with collaborators or service providers on behalf of a user. The attacker is an external participant whom the agent is intended to interact with, such as a collaborator or service provider, but who seeks to infer information about the represented user. The attacker receives no account or backend privilege. Through the ordinary interface, the attacker can submit a finite, adaptive sequence of ordinary task requests supported by the interface in one target domain and observe only the returned text and user-visible actions. The attacker may repeat a request, but every request must remain an ordinary task request and cannot ask the agent to reveal its memory, profile, or hidden state. The attacker cannot modify the system prompt, model configuration, tools, tool outputs, retained state Su , or the procedure that assembles request-conditioned state. All requests are evaluated against the same retained user model and system configuration. The attacker knows the public task and interface descriptions and need only suspect that some decisions are personalized. The attack requires no knowledge of the underlying LLM, hidden prompts, personalization pipeline, backend representation, or source records, and the attacker has no auxiliary information about the target user. Observed behavior may be processed offline with public models, but this provides no additional access to the personalized system. For domain x, the attack returns a semantic estimate Ŝ(u, x) of the user-specific information that the system makes available for personalization in that domain. The estimate contains human-readable claims about the user rather than backend fields, graph structure, or embedding coordinates. A recovered claim must describe information encoded in the hidden state and be grounded in a visible choice consistent with that information. Recovery can therefore be partial, and behavior that does not distinguish among plausible claims remains unrecovered. This definition makes success depend on attribution rather than plausibility alone. A statement that merely happens to describe the user receives no credit unless it matches the hidden state and is tied to the behavior from which the attack inferred it. The attack need not recover all of S(u, x) because even one correctly attributed semantic claim constitutes leakage under this recovery target. Because the information available for personalization can differ across domains, a separate

recovery attempt in another domain may expose complementary claims without reconstructing the broader internal representation Su .

3. UMPeek: Probing

Hypothesis-Guided

Adaptive

A history-derived user model may omit source wording. In our setting, both records and backend state are inaccessible through the ordinary interface, but the user information retained for personalization can still shape the agent’s decisions. UMPeek therefore starts from a concrete choice whose value was not specified by the request and asks what user-specific information could explain that choice. It excludes values fixed by the current request before treating any response detail as evidence about the user. Each explanation becomes a hypothesis, and later requests are selected to elicit behavior that supports it, contradicts it, or separates it from a competing explanation. We call this process hypothesisguided adaptive probing. For a target domain x, the attack observes only behavior exposed by the ordinary interface and returns a semantic estimate Ŝ(u, x) of the user-specific information in S(u, x). Let q0:m = (q0 , . . . , qm ) denote the requests that the attacker actually submits. The initial query q0 produces o0 ∼ p(· | q0 , C(u, x, q0 )), and a follow-up qi produces oi ∼ p(· | qi , C(u, x, qi )) from the same retained user model. Here, m counts only the follow-up requests and excludes the initial interaction. UMPeek represents a proposed explanation as a behavioral hypothesis. The hypothesis contains a candidate semantic claim, the visible answer or action choice that motivated it, and the interaction in which that choice occurred. This provenance link preserves the evidence needed to determine whether a compatible-sounding claim is actually supported by behavior. Because each hypothesis is expressed semantically, UMPeek does not require access to or decoding of the backend representation. Additional requests serve two distinct purposes. Replaying a task checks whether the candidate choice recurs under another realization of the same goal, whereas switching tasks tests whether the candidate explanation predicts compatible behavior outside the context that first suggested it. The accumulated evidence identifies unresolved dimensions and competing values. UMPeek then selects from a fixed pool of public task forms according to a task-family scheduling rule that prioritizes requests capable of separating them. This evidence-dependent scheduling is inspired by active hypothesis testing [46]. The following subsections describe how visible choices produce hypotheses, how later requests gather discriminating evidence, and how that evidence becomes the final semantic estimate.

3.1. Constructing Behavioral Hypotheses Candidate construction first restricts the hypothesis space to user-relevant decisions exposed by the public task and interface. The public task description and the response

or action schema expose a small set of user-relevant aspects on which the agent can make an observable decision. We refer to each such aspect as a decision dimension. These dimensions are defined for the task family before the target behavior is inspected, so they bound the attacker’s semantic hypothesis space without using the target user’s records or hidden state. For each dimension, the attacker prepares a claim template that names the semantic role while leaving its value open. The template specifies the form of a possible preference, constraint, or relation without asserting which value applies, and it need not correspond to a component of S(u, x). The request and the resulting behavior determine whether a template can be instantiated. For every decision dimension, UMPeek first checks whether qi already fixes its value. A value supplied by the requester is a task condition, so observing the same value in the response does not show that it came from user-specific information. When the request leaves the dimension open, the method examines oi for a choice that gives the dimension a concrete value. Only choices that affect the returned answer or executed action qualify. The value may come from an option the agent recommends or excludes, a selected tool, or an argument entered into a visible action. A topical mention is insufficient unless it changes what the agent returns or executes. Once an unrequested value appears in a concrete choice, the method forms a candidate claim and attaches the choice and source interaction (qi , oi ) to it. Later interactions can then add evidence to the same claim without losing the connection to the decision that first produced it. Suppose a restaurant request leaves dietary requirements unspecified and the answer excludes meat-heavy venues while recommending vegetarian options. Because diet was defined from the public task and interface as a decision dimension, the request leaves its value open, and the recommendation supplies a concrete value, UMPeek proposes that the hidden state may encode a vegetarian dietary constraint. The hypothesis remains attached to this restaurant interaction and is not extended into unsupported claims about the user’s health, ethics, or lifestyle. At this stage, it remains a candidate explanation of information that the user model may encode. UMPeek discards proposals that lack this chain from an open dimension to a visible decision, including task restatements, boilerplate, surface style, generic suggestions, and unused interface options.

3.2. Adaptively Testing Behavioral Hypotheses Because one choice can be compatible with several explanations about the user [37], the attack maintains support, contradiction, and unresolved alternatives for each proposed claim after the first interaction. A decision dimension is unresolved when no candidate value has appeared, when its only basis is one implicit choice, or when the collected choices remain compatible with competing values. Let B be the maximum number of follow-up requests. The executed sequence satisfies m ≤ B , and the attack may stop before reaching the bound when no unused request form can add

discriminating evidence for an unresolved dimension. The first follow-up, when the budget permits one, repeats the initial task while preserving its goal and leaving the tested dimensions unspecified. UMPeek applies the construction procedure in Section 3.1 to the replay response, so a recurring choice can provide replay evidence for the corresponding initial hypotheses. Semantically equivalent claims are aligned by decision dimension and value. Repeated evidence has a deliberately limited interpretation. When a compatible choice recurs, the replay shows that the candidate is not confined to one realization of the initial task. Even persistent behavior under an unchanged request may still follow from that request’s local context. Replay therefore checks within-task persistence but does not by itself attribute the candidate to the hidden user model. The stronger test changes the task conditions while keeping the relevant decision dimension open. UMPeek draws these follow-ups from ordinary request forms defined by the public task and interface. Behavior under the changed request then provides evidence about whether the candidate explanation continues to predict compatible choices outside the interaction that proposed it. Consider the dietary hypothesis proposed from the earlier restaurant recommendation. A replay asks for the same kind of recommendation without naming a diet and checks whether the agent again favors vegetarian options. The repeated choice adds within-task support, but the original restaurant context could still explain both responses. A stronger follow-up therefore changes the task, for example by asking the agent to arrange catering while again leaving dietary requirements open. Selecting vegetarian catering now supports the same claim through a different decision. Choosing a meat-heavy option under a comparable open request would instead contradict the proposed constraint. By contrast, choosing meat after the requester explicitly requires it supplies no contradiction because the request, rather than personalization, fixed the value. This distinction is why UMPeek evaluates each choice together with the request that elicited it rather than counting repeated words or values in isolation. Replay checks within-task persistence, whereas a switched task tests whether the same hypothesis explains a compatible choice in another context. Together, these tests are designed to reduce the risk of attributing oneoff or task-specific behavior to personalization. The available request forms are ordered according to the current evidence state. The scheduling rule favors a request when it can elicit concrete choices for several unresolved dimensions or distinguish values that are still in competition. It lowers the priority of requests that revisit dimensions with sufficient evidence, determine the relevant value in advance, or are likely to return only generic content. The task-family rule only schedules evidence collection and is neither fitted to the target user nor presented as an optimal informationgain objective. Every returned behavior updates the attack’s nextrequest decision. The method applies the same construction rule, links compatible choices to existing hypotheses, records newly proposed claims, and marks compara-

ble choices that conflict with a candidate value. It then recomputes which dimensions remain unresolved before selecting another request. Consequently, two targets can follow different request sequences even when they share the same public request pool. A single replay can provide evidence for several initial claims in one interaction, while switched tasks are preferred when one response can inform several unresolved dimensions. The process stops after B follow-ups or as soon as the unused request forms cannot add a concrete, discriminating choice. What remains is a set of candidate claims together with the interactions that supported, challenged, or failed to resolve them.

3.3. Selective Recovery from Behavioral Evidence Recovery applies a discrete evidence rule to determine whether a candidate claim can be attributed to the hidden personalized state. The method first maps each hypothesis to a canonical semantic form defined by its public decision dimension and value. Surface forms with the same meaning are grouped while their source interactions remain separate. This operation lets a textual answer and a visible action contribute to the same claim without treating repeated wording inside one response as independent evidence. Canonicalization does not settle which explanation is correct. A single choice can remain consistent with a strict constraint, a weaker preference, or a condition created by the task context. UMPeek therefore evaluates each grouped claim against the interactions in which the relevant dimension was left open. Only these interactions leave open the possibility that the hidden state contributed to the observed value, whereas a value fixed by the request cannot support that attribution. An interaction provides direct support when the response explicitly links user-specific information to a choice or when an executed action supplies the corresponding value. A later interaction provides cross-request support when it leaves the same dimension open and produces a compatible choice. A comparable open interaction provides contradiction when it makes a choice that the candidate claim would rule out. Omission supplies no evidence, and a request that fixes another value is not a valid contradiction. These roles yield the acceptance rule. A claim must be tied to at least one concrete visible choice and must face no contradictory behavior in the executed sequence. An implicit claim proposed from a recommendation also needs evidence beyond its first appearance, either by surviving replay or by explaining a compatible choice under changed task conditions. Direct support can satisfy this requirement without an additional request used only to repeat information that the visible response or action already makes explicit. If the available interactions cannot separate mutually exclusive explanations, all of them remain unresolved and none enters Ŝ(u, x). When accepted claims overlap, UMPeek uses claim specificity and the number of distinct supporting interactions only to choose among claims that already satisfy the evidence rule. The accepted claims together form a semantic estimate of the user-specific information in the hidden state that is

supported by behavior in domain x. This estimate may cover only part of that information, and it can be empty when the interface never exposes a choice that separates the viable explanations. Its semantic form is intentional because UMPeek infers what the hidden state encodes about the user rather than reconstructing backend fields, embedding coordinates, model parameters, graph topology, or storage schema.

4. Experimental Setup Our experiments test whether user-specific information remains semantically recoverable from visible behavior when source records and internal personalized state are unavailable to the attacker. The evaluation compares attacks and traces their evidence on common benchmarks, tests transfer to independently implemented systems, and measures residual recovery and task performance under inference-time defenses. This section defines the common benchmark data, black-box protocol, recovery target, and metrics. Evaluation Protocol.. For the benchmark and defense experiments, an evaluation target is one benchmark interaction evaluated with one personalization system in domain x. The system fixes its retained user state before recovery, although the backend may still assemble requestdependent context from that state. The attacker observes only response text and user-visible tool actions through the ordinary interface. Human-readable reference claims define correct semantic recovery across systems with different internal representations. Each target is also paired with three held-out tasks involving the same user or conversation to test whether the recovered estimate predicts other personalized decisions. Attack methods receive neither the claims nor the held-out tasks and their reference behavior. Benchmarks and Splits.. We select benchmarks in which user-specific information can affect an ordinary answer or action and both the visible outcome and the semantic recovery target are available. PersonaMem-v2 [47] contributes 5,000 multiple-choice interactions from 200 personas, PersonaLens [48] contributes 5,000 task-oriented interactions from 79 users, ETAPP [3] contributes 4,800 tool-use cases, and LoCoMo [49] contributes 1,523 questions grounded in multi-session conversations. The resulting 16,323 interactions per backend, or 48,969 in total, expose personalization through discrete choices, task-oriented answers, and tool actions. The reference claims are scoring artifacts rather than assumed textual fields in the hidden user model. Appendix B gives the per-benchmark filtering and unseen-task construction rules. Personalization Backends.. The benchmark and defense evaluations connect the same agent to Mem0, Graphiti, and LangMem+LangGraph. Mem0 supplies retrieved memories [21], Graphiti supplies entities and temporal relations from a temporal knowledge graph [43], and LangMem+LangGraph gives the agent access to long-term memories through LangGraph’s storage layer [50]. We vary the backend while keeping the request, black-box interface,

and Qwen3-14B downstream model served through vLLM fixed [51], [52]. The model uses non-thinking mode and greedy decoding to reduce sampling variation [53]. Because the reference claims capture the meaning supplied to the agent, cross-backend results measure semantic recovery rather than reconstruction of a storage representation. The real-world evaluation separately tests systems implemented outside our evaluation code. Attack Baselines.. We compare UMPeek with ADAM [32], LLM-PBE [54], PLeak [28], IPI [55], Imprompter [56], AttrInf [39], and PIE [57], covering memory extraction, privacy assessment, prompt leakage, indirect injection, and attribute inference. We instantiate each method’s published request strategy under the same black-box access and evaluation-specific request cap. We convert only its final statements into the common semantic claim format without adding content or exposing private state, unseen tasks, or reference claims. Metrics.. User-Model Recovery F1 (UMR-F1) measures semantic agreement between recovered and reference claims using one-to-one matching by a deterministic category-aware lexical rule. The rule requires compatible claim categories, normalizes spelling and inflection, and accepts exact matches, normalized containment, or token overlap of at least 0.72 before assigning each claim at most once. The matcher uses neither an LLM nor an embedding model. For the benchmark and defense evaluations, Heldout Behavior Prediction Score (HBPS) instead measures whether the recovered claims predict reference behavior on the three unseen tasks. We report both because predicting an option does not necessarily identify the user information responsible for it, while a correct claim need not affect every later task. UMR-F1 is computed for the 48,941 target-backend evaluations with at least one reference claim, while HBPS uses all 48,969 evaluations. Attackperformance intervals use percentile bootstrap resampling clustered by user or conversation within each benchmark, with the same sampled cluster retained across methods and backends to preserve experimental pairing. Mechanism and ablation intervals use normal approximations over their fixed balanced subset, while the real-world and defense evaluations use the resampling units specified in their protocol appendices. Appendix B gives the metric equations and taskspecific behavior scores.

5. Attack Evaluation We compare UMPeek with seven attacks across four benchmarks and three personalization backends using the metric-specific evaluation sets defined in Section 4. Every method begins with the same target interaction, receives only the black-box access in Subsection 2.2, and may submit at most 16 follow-up requests. We first compare semantic recovery and behavior prediction, then examine variation across task and backend settings and test whether interaction volume alone explains UMPeek’s gain.

TABLE 1. OVERALL ATTACK RESULTS POOLED ACROSS FOUR BENCHMARKS AND THREE PERSONALIZATION BACKENDS . B RACKETS SHOW 95% CLUSTER - BOOTSTRAP CONFIDENCE INTERVALS FOR UMR-F1 AND HBPS. T HE SAME BOOTSTRAP SAMPLES ARE SHARED ACROSS METHODS AND BACKENDS TO PRESERVE EXPERIMENTAL PAIRING . U NDERLINED ENTRIES MARK THE STRONGEST BASELINE BASED ON UNROUNDED SCORES .

Method UMPeek ADAM LLM-PBE PLeak IPI Imprompter AttrInf PIE

UMR-F1

HBPS

0.619 [0.614, 0.623] 0.002 [0.002, 0.003] 0.018 [0.016, 0.020] 0.001 [0.000, 0.001] 0.051 [0.049, 0.053] 0.053 [0.051, 0.055] 0.100 [0.098, 0.102] 0.100 [0.098, 0.102]

0.289 [0.285, 0.292] 0.003 [0.003, 0.004] 0.051 [0.049, 0.052] 0.000 [0.000, 0.001] 0.068 [0.066, 0.069] 0.084 [0.081, 0.087] 0.144 [0.141, 0.147] 0.147 [0.144, 0.150]

5.1. Attack Performance We first test the central distinction between record visibility and behavioral observability. Specifically, we ask whether attacks developed for memory extraction, prompt leakage, or attribute inference can recover semantic information when the user model and its source records cannot be inspected through the attack interface. Table 1 pools all four benchmarks and three backends under the common blackbox interface, recovery target, and scoring procedure. The pooled results show a clear separation on both outcomes. UMPeek reaches 0.619 UMR-F1 compared with 0.100 for the strongest semantic-recovery baseline, and 0.289 HBPS compared with 0.147 for the strongest baseline on held-out behavior prediction. The joint lead matters because UMR-F1 requires the estimate to identify semantic information in the hidden user model, whereas HBPS tests whether that estimate retains information that predicts personalized decisions not observed during recovery. Leading on both metrics therefore indicates that the recovered claims are behaviorally relevant rather than merely similar to the reference wording. The comparison also explains why attacks aimed at stored artifacts transfer poorly to this setting. When neither records nor backend state can be inspected, extraction baselines have no readable object to recover, while attributeinference baselines can propose plausible user descriptions without attributing them to a personalization-dependent choice. UMPeek instead grounds each candidate claim in a visible choice and retains an implicit claim only when later behavior supports it without contradiction. Because every method is converted to the same claim format and scored by the same deterministic matcher, the gap reflects how evidence is collected and attributed rather than a methodspecific scoring advantage. The result therefore shows that making records and backend state inaccessible does not prevent user information from being recovered when that information continues to shape visible behavior. Transfer across tasks and backends.. The pooled comparison could conceal a result driven by one benchmark or one state representation. Table 2 therefore reports

0.7

TABLE 2. UMP EEK ACROSS THE TWELVE BENCHMARK - BACKEND SETTINGS . T HE TWO BLOCKS REPORT SEMANTIC RECOVERY AND HELD - OUT BEHAVIOR PREDICTION .

UMR-F1 Mem0 Graphiti LangMem + LangGraph HBPS Mem0 Graphiti LangMem + LangGraph

Persona Persona Mem-v2 Lens ETAPP LoCoMo

0.6 0.5

Recovery score

Backend

0.4

0.467 0.613

0.752 0.592

0.583 0.512

0.795 0.817

0.607

0.742

0.523

0.800

0.084 0.087

0.335 0.343

0.468 0.452

0.267 0.297

0.1

0.083

0.337

0.439

0.276

0.0

UMPeek separately for all twelve benchmark-backend combinations. UMPeek retains nonzero recovery in all twelve combinations, with UMR-F1 ranging from 0.467 to 0.817, while the ordering of the backends changes across benchmarks. The attack surface is therefore not tied to one storage or retrieval organization. The backend controls which retained information reaches the agent, while the task determines whether that information changes an observable answer or action. The variation across cells therefore belongs to the benchmark-backend pairing rather than to a fixed privacy ranking of the backends. Protecting one representation alone would not remove this channel, even though system design can change its magnitude. The two metrics also order the tasks differently. Across all three backends, LoCoMo produces higher UMR-F1 but lower HBPS than ETAPP. A broad semantic estimate may recover several correct claims even when few affect a particular unseen task, whereas a less complete estimate may still contain the claim that drives one decision. Semantic recovery and behavioral effect are therefore complementary consequences of exposure rather than interchangeable measures of attack success.

5.2. Adaptive Request Selection and Ablations To locate where recoverable evidence enters the pipeline, Figure 1 reports the cumulative claim set after each UMPeek stage on a balanced subset spanning all twelve benchmarkbackend settings. Because this subset weights the settings equally, full UMPeek reaches 0.651 UMR-F1 here rather than the pooled 0.619 in Table 1. Claim precision shows how much of the claim set available at each stage matches the reference information. The initial response alone contributes almost no correct recovery, and public interface information produces only a modest increase. The largest change appears after selected follow-up responses, which raise UMR-F1 to 0.606 before the acceptance rule produces the final 0.651 UMR-F1 and 0.634 claim precision. This progression locates the principal evidence source in behavior accumulated across selected

UMR-F1 Claim precision

0.3 0.2

Initial response

Task/tool context

Tool arguments

Follow-up responses

Acceptance rule

Recovery stage Figure 1. Cumulative recovery after each UMPeek processing stage on the balanced subset of 3,072 targets. Each point reports the claim set available after applying all stages through the one named on the horizontal axis. The lines report UMR-F1 and claim precision, and error bars show 95% normal-approximation confidence intervals.

tasks rather than in a fact disclosed by the initial response. Static interface information helps interpret what an initial choice could mean, while behavior across changing tasks determines which explanation continues to fit the agent’s decisions. The hidden information need not be stated in any one answer because it becomes attributable when different tasks repeatedly elicit choices consistent with the same user hypothesis. The stagewise progression shows when evidence becomes available, but it does not by itself separate the value of additional interaction from the value of choosing informative requests. Table 3 tests this distinction by removing individual components or replacing adaptive selection with a fixed request sequence on the same balanced subset. Full UMPeek and the fixed sequence both use about 15 follow-ups, yet they reach 0.651 and 0.281 UMR-F1, respectively. UMPeek spends this budget on decisions that bear on unresolved candidates, whereas the fixed sequence does not respond to what earlier behavior has established. The difference is therefore explained by informative task selection rather than by asking materially more questions. Removing the acceptance rule causes a smaller loss that is concentrated in claim precision, which shows that acceptance filters explanations after the follow-ups have supplied most of the recoverable evidence. Removing public descriptions or the initial visible tool action has little effect once selected follow-up behavior is available. These components therefore serve different roles. Adaptive selection creates task contexts that distinguish competing explanations, while acceptance prevents explanations without sufficient behavioral support from entering the final estimate. Section 6 next tests whether the same behavior-based inference transfers to independently implemented personal-

TABLE 3. UMP EEK ABLATIONS ON THE BALANCED SET USED IN F IGURE 1. E ACH ABLATED ROW REMOVES ONE COMPONENT, RESTRICTS THE EVIDENCE AVAILABLE TO THE METHOD , OR REPLACES ADAPTIVE REQUEST SELECTION WITH A FIXED REQUEST SEQUENCE . B RACKETS IN THE FIRST TWO METRIC COLUMNS SHOW 95% NORMAL - APPROXIMATION CONFIDENCE INTERVALS .

Claim precision

Variant Full UMPeek

UMR-F1

0.634 0.651 [0.625, 0.643] [0.643, 0.660]

No follow-up requests 0.328 0.153 (with public/interface evidence) [0.312, 0.343] [0.145, 0.161] No initial visible 0.633 0.648 tool evidence [0.624, 0.642] [0.640, 0.657] No public task or tool 0.630 0.648 descriptions [0.620, 0.639] [0.640, 0.656] No acceptance rule 0.550 0.606 [0.541, 0.558] [0.598, 0.614] Fixed-sequence 0.377 0.281 follow-up requests [0.366, 0.389] [0.272, 0.290] Initial response text only 0.003 0.003 [0.001, 0.005] [0.001, 0.005]

ization systems. Takeaway. Under comparable interaction volume, hypothesis-guided request selection yields substantially higher recovery than the fixed sequence, while the acceptance rule filters explanations that do not meet the behavioral evidence requirement.

6. Real-World Validation We next test whether visible behavior supports recovery of information that independently implemented systems have retained, rather than only information supplied by our benchmark backends. The evaluation covers Google Vertex AI Memory Bank, AWS Bedrock AgentCore Memory, Mem0 Platform, and the end-to-end OpenClaw personal agent.

6.1. Evaluated Systems and Protocol Google Memory Bank [18] and AWS AgentCore [19] are managed cloud services that construct persistent user state from conversation histories, while Mem0 [20] provides a standalone memory layer for agents. OpenClaw [58], [59] is a complete personal assistant that consolidates working notes into long-term memory and uses that state while responding and acting. Together, they move the evaluation from locally configured research backends to independently implemented managed services and a complete personal agent. We provide each system with synthetic PersonaMem-v2 histories and let it construct its own personalized state before any attack request. The managed-service comparison uses

the same eight methods on 20 confirmed-retained targets per service under a cap of three total requests. Attack methods do not receive the retention result, so scores measure recovery conditional on retention rather than either retention rate or recovery over all source claims. UMR-F1 measures semantic recovery and Choice F1 measures whether the recovered claims identify the option selected by the service. Each target has one retained reference claim, and the deterministic matcher accepts either the benchmark wording or equivalent wording observed during the retention check after normalizing spelling, inflection, and common function words. It accepts exact matches, normalized containment, or at least two shared content words with overlap of 0.60 or higher. Every nonmatching recovered claim counts as a false positive, and no language or embedding model participates in scoring. Choice F1 identifies a selected option through content-word overlap with the displayed choices and treats no overlap as an abstention. Macro results are unweighted means across the three services, with targetlevel bootstrap resampling within each service. OpenClaw is evaluated separately on one retained claim for each of 25 personalized agents and is not used for ranking the eight methods. Appendix C gives the retention procedure, scoring rules, system versions, and complete protocol. The attack submits only ordinary requests and receives visible behavior, while retention data, reference claims, and platform state remain confined to evaluation.

6.2. Cross-System Recovery Figures 2 to 4 compare aggregate recovery, decompose semantic precision and recall by service, and relate recovery to the number of requests used. UMPeek obtains the highest observed UMR-F1 on each managed service and a macro score of 0.500, compared with 0.180 for the strongest comparison method. The consistency across services matters more than their individual point estimates because each service independently decides what information to retain and how that information enters a response. UMPeek’s lead therefore supports transfer of behavior-based recovery beyond the three research backends rather than dependence on one state representation. The remaining cross-service variation shows that implementation and retained-target differences still affect how strongly the behavioral channel exposes user information. The precision-recall decomposition explains where the semantic-recovery advantage comes from. UMPeek balances macro precision and recall at 0.500, whereas comparison methods approach its recall only with far lower precision. For example, Imprompter reaches 0.450 recall but only 0.113 precision. This pattern indicates that UMPeek more reliably attributes a description to hidden user information instead of returning a broader set of plausible claims. On Mem0, it also improves both precision and recall, so this selectivity does not simply exchange coverage for fewer errors. Choice prediction measures a different capability. On AWS AgentCore, UMPeek and Imprompter obtain nearly

Google Memory Bank AWS AgentCore

The case shows that the target is the user-level meaning expressed through the choice rather than copied memory text. In the AWS case, the response prioritizes an aquarium because calm aquatic environments help the user focus. UMPeek recovers that preference without returning a choice prediction, which makes concrete why explaining a personalized choice and predicting its selected option are different capabilities. Finally, OpenClaw recommends a scenic bike ride that identifies the retained preference from the first ordinary response. This case marks the boundary at which a single personalization-dependent choice already exposes the evaluated information and cross-request disambiguation is unnecessary. Together, the traces connect the quantitative metrics to the threat model by showing that ordinary behavior can disclose user-specific meaning without displaying the hidden state itself.

Mem0 Platform

(a) UMR-F1 UMPeek ADAM LLM-PBE PLeak IPI Imprompter AttrInf PIE

(b) Choice F1

6.4. Interaction Budget and OpenClaw Boundary

UMPeek ADAM LLM-PBE PLeak IPI Imprompter AttrInf PIE 0.0

0.2

0.4

0.6

0.8

1.0

Score Figure 2. Cross-platform attack performance on 20 retained targets per service. Panels report semantic user-information recovery and personalizedchoice prediction for all eight methods. Points denote method scores, and horizontal bars show 95% target-bootstrap confidence intervals. The shaded row marks UMPeek.

identical Choice F1, yet UMPeek’s UMR-F1 is two and a half times higher. A method can therefore anticipate the visible option without identifying which hidden user information explains that choice. Figure 3 makes this distinction visible across all method-system pairs and shows that the gap is driven mainly by precision on Google and AWS, while both precision and recall improve on Mem0. The confidence intervals separate UMPeek from the comparison methods on AWS and Mem0 but overlap on Google. Together, the results support transfer and an aggregate recovery advantage without implying equal exposure across services.

Figure 4 relates semantic recovery to the number of ordinary requests issued under the common cap. UMPeek obtains the highest observed UMR-F1 on all three managed services within three requests, and no evaluated method attains higher recovery with the same or fewer requests on any service. The result rules out a larger request allowance as the explanation for the cross-system lead. Instead, it is consistent with the benchmark ablation because UMPeek uses the available interactions to test unresolved hypotheses rather than ending after a less informative request sequence. The shared small cap also keeps the attack within the intended service interface rather than relying on prolonged access. The figure therefore evaluates how effectively each method uses a limited budget, not whether UMPeek always issues the fewest requests. For every selected OpenClaw target, the first interaction already contains a visible choice matching the retained preference. These cases show that cross-request accumulation is not always necessary once personalization makes the relevant information visible, while their retention-conditioned selection does not estimate exposure over all information stored by OpenClaw. Takeaway. Across the evaluated retained targets, visible behavior supports recovery on independently implemented systems when retained information shapes the agent’s choices.

7. Adaptive Defenses

6.3. Semantic Recovery in Visible Behavior Figure 5 illustrates how visible behavior supports semantic recovery in representative interactions. In the Mem0 case, a request for lesser-known destinations in Iceland produces a response centered on camera-friendly locations, from which UMPeek recovers a travel-photography preference using wording different from the retained claim.

Because the personalized decision itself can carry user information, we test whether inference-time defenses suppress recovery without removing useful personalization as UMPeek adapts across responses. Table 4 reports residual semantic recovery, behavior prediction, and task utility at the largest request budget, while Figure 6 varies the defense threshold and follow-up budget.

UMR precision

(a) Google Memory Bank

UMR recall

UMR-F1 (95% CI)

(b) AWS AgentCore

(c) Mem0 Platform

UMPeek ADAM LLM-PBE PLeak IPI Imprompter AttrInf PIE 0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

UMR score

0.4

0.6

0.8

1.0

0.0

0.2

UMR score

0.4

0.6

0.8

1.0

UMR score

Figure 3. Per-system semantic recovery on 20 retained targets per service. Circles and triangles show UMR precision and recall. Diamonds show UMR-F1, and horizontal bars show 95% target-bootstrap confidence intervals for UMR-F1. All panels use the same horizontal scale, and the shaded row marks UMPeek.

Google Memory Bank

AWS AgentCore

Mem0 Platform

UMR-F1

0.6

0.4

0.2

0.0 1

2

3

1

Total requests

UMPeek

ADAM

2

3

1

Total requests

LLM-PBE

PLeak

2

3

Total requests

IPI

Imprompter

AttrInf

PIE

Figure 4. Semantic recovery relative to the number of ordinary requests issued per target. Each point is one evaluated method. An outlined point indicates that no other method attains higher UMR-F1 with the same or fewer requests on that service. Horizontal offsets within an integer request count are used only to keep overlapping markers visible.

Defenses and Protocol.. PrivacyChecker [60] checks contextual information flows before the agent responds, while the Theory-of-Mind defense [40] reasons about how the requester could use the proposed response. We also evaluate Stateful Counterfactual Exposure Control, which compares each personalized choice with the choice produced without personalization, accumulates that evidence across requests, and withholds behavior once the evidence exceeds a threshold. The main comparison allows at most 16 follow-

ups, and UMPeek adapts only to behavior released through the defended interface. UMR-F1 and HBPS measure residual exposure, while TaskScore applies each benchmark’s original task metric [3], [47], [48], [49] to the evaluated personalized tasks. Appendix D gives the sampling, threshold, and resampling procedures. Residual Exposure.. Table 4 shows that the two response-level defenses leave substantial exposure, while Stateful Counterfactual lowers macro UMR-F1 further to 0.187. The remaining recovery indicates that screening in-

(a) Mem0 Platform | MEMF-008-T01 REQUEST

VISIBLE EVIDENCE

RESULT + REFERENCE

What are some unique spots worth visiting on a trip to Iceland that most tourists might overlook?

Response (verbatim) Choice=C.

Semantic estimate (verbatim) Takes photographs while traveling. Choice prediction: correct

Selected option (verbatim) If you’re heading to Iceland with your camera, you might love exploring lesser-known gems like the hauntingly beautiful Westfjords, the mossy lava fields of Eldhraun, or the sea arches at Dyrhólaey during golden hour. [...]

Evaluator reference (verbatim) Loves photography during travels

INTERPRETATION Different wording recovers the same travel-photography preference.

(b) AWS AgentCore | AWSF-019-T01 REQUEST

VISIBLE EVIDENCE

RESULT + REFERENCE

I'll only have an afternoon in a new city and the weather might be unpredictable — what indoor attractions should I prioritize, and how much time should I plan for each to make the most of the visit?

Response (verbatim) Choice=B.

Semantic estimate (verbatim) Has tendency to find calm and focus in aquatic environments Choice prediction: not returned

The aquarium fits your preference for calm, aquatic environments that help with focus, making it an ideal first stop to settle [...] Selected option (verbatim) Prioritize the aquarium first (about 60–90 minutes), followed by a nearby science or history [...]

Evaluator reference (verbatim) Likes visiting aquariums

INTERPRETATION Semantic recovery succeeds even though no choice prediction is returned.

(c) OpenClaw | PMV4-001-T04 REQUEST

VISIBLE EVIDENCE

RESULT + REFERENCE

What are some relaxing outdoor activities I could try on a weekend getaway to unwind and enjoy nature?

Response (verbatim) Choice=C

Structured method result Accepted option ID: opt-1312b1614ad91af2 Support: 3/3 interactions

Selected option (verbatim) You might enjoy planning a scenic rail-trail bike ride through nearby forested areas or along the lakeshore, pairing it with a leisurely picnic at a quiet overlook. For a [...]

Evaluator reference (verbatim) Enjoys bike rides through scenic areas

INTERPRETATION The first ordinary choice already identifies the retained behavior target.

Figure 5. Representative evidence traces from the real-world validation. Each trace connects an ordinary request and visible personalized choice to the semantic estimate and retained reference information. [...] marks response text omitted for space.

dividual outputs does not close a channel assembled from several responses that the defense permits. Counterfactual control targets this channel more directly by withholding decisions that change when personalization is removed. A response can omit explicit user information yet still release a choice whose value carries the same evidence. Lower semantic recovery, however, does not produce the same ordering in behavioral exposure or task utility. Stateful Counterfactual has the lowest macro UMR-F1, but it does not have the lowest HBPS or the highest TaskScore on every benchmark. A defense can therefore reduce the amount of semantically recoverable information without proportionally reducing the behavioral relevance of what remains or preserving personalized task performance. These three outcomes must be evaluated separately. Adaptive Trade-offs.. Figure 6(a) varies the Stateful Counterfactual threshold. Less restrictive release raises TaskScore on all four benchmarks and UMR-F1 on three of

them, and no evaluated threshold both eliminates recovery and matches undefended task performance. The trade-off arises because personalization-dependent decisions can both improve the task result and reveal the user information that caused them. Figure 6(b) instead holds each defense fixed and increases the follow-up budget. After 16 follow-ups, recovery under PrivacyChecker and the Theory-of-Mind defense reaches about 65% and 90% of the undefended level. Evaluating only the initial response would therefore overstate their resistance to an adaptive attacker. UMPeek accumulates evidence across outputs that each response-level defense permits in isolation, reproducing the cross-request channel identified in the attack evaluation. Defense Mechanism.. Table 5 separates cross-request evidence tracking from counterfactual comparison in Stateful Counterfactual Exposure Control. The ablation identifies counterfactual comparison, rather

TABLE 4. UMP EEK RECOVERY AND BENCHMARK TASK PERFORMANCE UNDER THREE INFERENCE - TIME DEFENSES WITH AT MOST 16 FOLLOW- UP REQUESTS . R ESULTS USE 128 TARGETS FOR EACH BACKEND - BENCHMARK SETTING . T HE SAME BENCHMARK TARGETS ARE EVALUATED ACROSS BACKENDS AND RESAMPLED TOGETHER . L OWER UMR-F1 AND HBPS AND HIGHER TASK S CORE ARE PREFERABLE . B RACKETS REPORT 95% CLUSTER - BOOTSTRAP CONFIDENCE INTERVALS .

UMR-F1 ↓

HBPS ↓

TaskScore ↑

PrivacyChecker Theory-of-Mind Defense Stateful Counterfactual

0.324 [0.300, 0.348] 0.445 [0.418, 0.470] 0.260 [0.237, 0.284]

0.091 [0.068, 0.116] 0.137 [0.107, 0.168] 0.104 [0.074, 0.134]

0.664 [0.586, 0.742] 0.682 [0.602, 0.758] 0.482 [0.398, 0.573]

PersonaLens

PrivacyChecker Theory-of-Mind Defense Stateful Counterfactual

0.503 [0.476, 0.532] 0.631 [0.605, 0.656] 0.172 [0.153, 0.192]

0.308 [0.291, 0.325] 0.372 [0.359, 0.387] 0.357 [0.343, 0.371]

0.273 [0.244, 0.301] 0.236 [0.205, 0.264] 0.155 [0.126, 0.182]

ETAPP

PrivacyChecker Theory-of-Mind Defense Stateful Counterfactual

0.421 [0.401, 0.441] 0.454 [0.427, 0.477] 0.280 [0.265, 0.296]

0.511 [0.455, 0.568] 0.515 [0.453, 0.577] 0.547 [0.488, 0.603]

0.118 [0.094, 0.144] 0.390 [0.339, 0.442] 0.356 [0.315, 0.397]

LoCoMo

PrivacyChecker Theory-of-Mind Defense Stateful Counterfactual

0.034 [0.013, 0.060] 0.247 [0.185, 0.311] 0.037 [0.014, 0.066]

0.055 [0.047, 0.063] 0.095 [0.083, 0.109] 0.071 [0.061, 0.081]

0.095 [0.056, 0.143] 0.404 [0.332, 0.478] 0.288 [0.226, 0.351]

Benchmark

Defense

PersonaMem-v2

Undefended

(a)

PrivacyChecker

PersonaMem-v2

Theory-of-Mind Defense

PersonaLens

Stateful Counterfactual

ETAPP

LoCoMo

1.0 0.8

TaskScore

1.5

0.6 1.5

0.4

1.5

1 1.5

0.5

0.2

1 1

0.5 1

0.5

0.5

0.0 0.0

0.2

0.4

0.6

0.8

1.0 0.0

0.2

0.4

UMR-F1

0.6

0.8

1.0 0.0

0.2

0.4

UMR-F1

0.6

0.8

1.0

0.0

0.2

0.4

UMR-F1

0.6

0.8

1.0

UMR-F1

(b) 1.0

UMR-F1

0.8 0.6 0.4 0.2 0.0 0 1 2

4

8

Follow-up budget

16

0 1 2

4

8

Follow-up budget

16

0 1 2

4

8

Follow-up budget

16

0 1 2

4

8

16

Follow-up budget

Figure 6. Privacy and utility at different decision thresholds, together with recovery across follow-up-request budgets. Panel (a) uses the same 32 targets from each backend-benchmark setting at every Stateful Counterfactual threshold and in the undefended condition. Panel (b) uses 128 targets for each setting. Bands report 95% cluster-bootstrap confidence intervals.

than cross-request tracking, as the active component on this subset. Removing tracking barely changes semantic recovery, HBPS, or TaskScore, whereas removing counterfactual comparison raises UMR-F1 and lowers TaskScore on every benchmark. This difference matters because suppressing user-related content without asking whether personalization changed the decision can discard useful outputs without isolating the behavior that carries exposure. The counterfactual test is more selective because it focuses intervention on choices caused by personalization. This also explains why screening each response for user-related content can

miss information exposed through the effect of personalization on a decision. Even this targeted control releases some personalization-dependent decisions at the evaluated operating point, leaving behavioral evidence from which UMPeek can recover information. What the defense results imply.. The three defenses intervene at different points in the release path, and their ordering clarifies which point matters for this attack. PrivacyChecker evaluates whether a response is appropriate to disclose, and the Theory-of-Mind defense evaluates how a requester might use that response. Both reason about the re-

TABLE 5. M ECHANISM ABLATIONS FOR S TATEFUL C OUNTERFACTUAL E XPOSURE C ONTROL . C OLUMNS REMOVE CROSS - REQUEST TRACKING OR COUNTERFACTUAL COMPARISON . UMR-F1 AND HBPS ARE MACRO AVERAGES , WHILE TASK S CORE REMAINS BENCHMARK - SPECIFIC . B RACKETS REPORT 95% CLUSTER - BOOTSTRAP CONFIDENCE INTERVALS .

Metric UMR-F1 ↓ HBPS ↓ PersonaMem-v2 TaskScore ↑ PersonaLens TaskScore ↑ ETAPP TaskScore ↑ LoCoMo TaskScore ↑

Full

No tracking

No comparison

0.179 0.190 [0.155, 0.203] [0.165, 0.214] 0.278 0.277 [0.229, 0.327] [0.230, 0.327]

0.242 [0.211, 0.272] 0.216 [0.169, 0.267]

0.417 0.417 [0.250, 0.583] [0.260, 0.583]

0.167 [0.052, 0.292]

0.177 0.177 [0.121, 0.237] [0.120, 0.236]

0.081 [0.045, 0.122]

0.316 0.314 [0.251, 0.393] [0.248, 0.393]

0.231 [0.169, 0.305]

0.256 0.253 [0.138, 0.384] [0.138, 0.382]

0.019 [0.004, 0.035]

leased answer. Stateful Counterfactual instead asks whether personalization changed the decision. Its lower UMR-F1 indicates that this comparison is better aligned with the evidence UMPeek uses, namely a difference between behavior with and without the retained user information. The accompanying TaskScore loss also shows why the channel is difficult to close. The same personalization-dependent difference that supplies attack evidence often supplies the useful adaptation that the application is intended to provide. This connection changes what a defense must accomplish. A response that appears harmless alone can become informative after another task rules out a competing explanation, so success on isolated responses does not establish protection over a sequence. Conversely, suppressing every personalization-dependent choice would reduce exposure by removing personalization itself. The relevant objective is therefore not only to sanitize text, but to limit how confidently a sequence of released choices can be attributed to retained user information while preserving the choices needed for the task. The separate UMR-F1, HBPS, and TaskScore results expose this three-way requirement. Takeaway. Response-level defenses screen individual releases, while UMPeek accumulates how personalization changes choices across requests. Stateful Counterfactual control reduces this channel but leaves residual recovery while lowering personalized task performance.

8. Related Work This work connects persistent personalization, direct extraction, behavioral inference, and agent privacy controls. Personalization and Persistent User State.. Personalized systems derive profiles, embeddings, compressed contexts, and structured memories for later decisions [6],

[10], [12], [13], [15], [16], [17], [21], [47], [61]. Audits of ChatGPT memory find personal and inferred attributes, while persona skills turn histories into portable artifacts with disclosure and impersonation risks [62], [63]. Keyfact summarization can reduce direct canary extraction [26]. UMPeek instead asks what retained meaning remains inferable without access to history or derived state. Direct Extraction and Adversarial Manipulation.. Black-box attacks recover training examples, agent memories, retrieved content or membership, and hidden prompts [27], [28], [29], [30], [31], [32], [54]. Recent attacks elicit contextual memory or exfiltrate isolated records through malicious tools [64], [65]. Exposed embeddings and internal states are also vulnerable to inversion [66], [67]. Other attacks manipulate tool actions through adversarial inputs [55], [56]. UMPeek accesses neither backend state nor tool control, but infers semantic user information from ordinary visible behavior. Inferring Users from Personalized Behavior.. Model and attribute inversion infer users from predictions, activity, recommendations, supplied text, or public traces [33], [34], [39], [41], [57], [68], [69], [70], [71]. Recent work audits assistant profiling, recovers private negotiation constraints from interaction dynamics, or infers user traits from agent network metadata [72], [73], [74]. UMPeek instead adaptively tests open-ended explanations of inaccessible personalized state through task-dependent choices. Agent Privacy Evaluation and Defenses.. Agent audits examine inappropriate use, acquisition, or disclosure of private information in task context [75], [76], [77], [78], [79], [80], [81], [82]. CMPL uses adaptive conversation to expose failures missed by single-turn audits, while compositional privacy studies how leakage accumulates across responses [40], [83]. PrivacyChecker filters contextual flows, SP-Mem separates sensitive values from sanitized memory, and RootGuard protects dependent releases by sanitizing source values once [60], [84], [85]. These works protect or audit available private data, whereas UMPeek studies information implied by personalization-dependent choices, including information never stated.

9. Conclusion Transforming a history into a derived user model can remove source wording, while records and backend representations remain inaccessible through the ordinary interface. UMPeek nevertheless shows that user information retained for personalization can be inferred from the choices it affects. Hypothesis-guided follow-ups resolve explanations that one response leaves ambiguous. We observe this mechanism across four benchmarks, three local backends, independently implemented services, an end-to-end personal agent, and the evaluated inference-time defenses. These evaluations identify a behavioral privacy surface distinct from direct access to records or representations. UMPeek tests how retained information affects decisions rather than decoding the state that stores it, so this channel does not depend on a particular storage format. The

defense results expose a utility tension. Reducing the behavioral effect of retained information can lower recovery, but can also remove useful personalization. Protecting personalized agents therefore requires interaction-level controls that limit what repeated personalization-dependent choices reveal while preserving the adaptation they provide.

References [1]

X. Hu, T. Xiong, B. Yi, Z. Wei, R. Xiao, Y. Chen, J. Ye, M. Tao, X. Zhou, Z. Zhao, Y. Li, S. Xu, S. Wang, X. Xu, S. Qiao, Z. Wang, K. Kuang, T. Zeng, L. Wang, J. Li, Y. E. Jiang, W. Zhou, G. Wang, K. Yin, Z. Zhao, H. Yang, F. Wu, S. Zhang, and F. Wu, “OS agents: A survey on MLLM-based agents for computer, phone and browser use,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2025, pp. 7436–7465. [Online]. Available: https://aclanthology.org/2025.acl-long.369/

[2]

H. Singh, N. Verma, Y. Wang, M. Bharadwaj, H. Fashandi, K. Ferreira, and C. Lee, “Personal large language model agents: A case study on tailored travel planning,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track. Association for Computational Linguistics, 2024, pp. 486–514. [Online]. Available: https://aclanthology.org/2024. emnlp-industry.37/

[3]

Y. Hao, P. Cao, Z. Jin, H. Liao, Y. Chen, K. Liu, and J. Zhao, “Evaluating personalized tool-augmented LLMs from the perspectives of personalization and proactivity,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2025, pp. 21 897–21 935. [Online]. Available: https: //aclanthology.org/2025.acl-long.1064/

[4]

T. P. Zollo, A. W. T. Siah, N. Ye, A. Li, and H. Namkoong, “PersonalLLM: Tailoring LLMs to individual preferences,” in The Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://proceedings.iclr.cc/paper files/paper/2025/hash/ a730abbcd6cf4a371ca9545db5922442-Abstract-Conference.html

[5]

[6]

[7]

[8]

[9]

Z. Jin, N. Heil, J. Liu, S. Dhuliawala, Y. Qi, B. Schölkopf, R. Mihalcea, and M. Sachan, “Implicit personalization in language models: A systematic study,” in Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics, 2024, pp. 12 309–12 325. [Online]. Available: https://aclanthology.org/2024.findings-emnlp.717/ J. Liu, Y. Zhu, S. Wang, X. Wei, E. Min, Y. Lu, S. Wang, D. Yin, and Z. Dou, “LLMs + Persona-Plug = personalized LLMs,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2025, pp. 9373–9385. [Online]. Available: https://aclanthology.org/2025.acl-long.461/ W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang, “MemoryBank: Enhancing large language models with long-term memory,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 17, pp. 19 724–19 731, 2024. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/29946 J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, “Generative agents: Interactive simulacra of human behavior,” in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, 2023, pp. 1–22. [Online]. Available: https://doi.org/10.1145/3586183.3606763 W. Wang, L. Dong, H. Cheng, X. Liu, X. Yan, J. Gao, and F. Wei, “Augmenting language models with long-term memory,” in Advances in Neural Information Processing Systems 36, 2023. [Online]. Available: https://proceedings.neurips.cc/paper files/paper/2023/hash/ ebd82705f44793b6f9ade5a669d0f0bf-Abstract-Conference.html

[10] S. Doddapaneni, K. Sayana, A. Jash, S. Sodhi, and D. Kuzmin, “User embedding model for personalized language prompting,” in Proceedings of the 1st Workshop on Personalization of Generative AI Systems (PERSONALIZE 2024). Association for Computational Linguistics, 2024, pp. 124–131. [Online]. Available: https://aclanthology.org/2024.personalize-1.12/

[11] R. Yuan, S. Sun, Y. Li, Z. Wang, Z. Cao, and W. Li, “Personalized large language model assistant with evolving conditional memory,” in Proceedings of the 31st International Conference on Computational Linguistics. Association for Computational Linguistics, 2025, pp. 3764–3777. [Online]. Available: https://aclanthology.org/2025. coling-main.254/ [12] J. Ramos, H. A. Rahmani, X. Wang, X. Fu, and A. Lipani, “Transparent and scrutable recommendations using natural language user profiles,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2024, pp. 13 971–13 984. [Online]. Available: https://aclanthology.org/2024.acl-long.753/ [13] H. Su, Y. Yang, T. Liu, X. Liu, P. Pu, and X. Lu, “Personalized question answering with user profile generation and compression,” in Findings of the Association for Computational Linguistics: EMNLP 2025. Association for Computational Linguistics, 2025, pp. 4744–4763. [Online]. Available: https://aclanthology.org/2025. findings-emnlp.255/ [14] L. Du, Y. Yuan, Z. Zhao, F. Lyu, E. Penaloza, X. Chen, Z. Sun, J. Kang, L. Charlin, X. Liu, and H. Wu, “Optimizing user profiles via contextual bandits for retrieval-augmented LLM personalization,” in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2026, pp. 31 803–31 817. [Online]. Available: https://aclanthology.org/2026.acl-long.1467/ [15] L. Ning, L. Liu, J. Wu, N. Wu, D. Berlowitz, S. Prakash, B. Green, S. O’Banion, and J. Xie, “USER-LLM: Efficient LLM contextualization with user embeddings,” arXiv preprint arXiv:2402.13598, 2024, Google Research publication page. [Online]. Available: https://arxiv.org/abs/2402.13598 [16] Y. Qiu, T. Shi, X. Zhao, F. Zhu, Y. Zhang, and F. Feng, “Latent inter-user difference modeling for LLM personalization,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2025, pp. 10 599–10 617. [Online]. Available: https: //aclanthology.org/2025.emnlp-main.536/ [17] X. F. Zhang, N. Beauchamp, and L. Wang, “PRIME: Large language model personalization with cognitive dual-memory and personalized thought process,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2025, pp. 33 707–33 736. [Online]. Available: https://aclanthology.org/2025.emnlp-main.1711/ [18] K. Milam and G. Lee, “Announcing Vertex AI Agent Engine Memory Bank available for everyone in preview,” https://cloud.google.com/blog/products/ai-machine-learning/ vertex-ai-memory-bank-in-public-preview, 2025, accessed: Aug. 22, 2026. [19] Amazon Web Services, “Configure built-in strategies Amazon Bedrock AgentCore,” https://docs. aws.amazon.com/bedrock-agentcore/latest/devguide/ long-term-configuring-built-in-strategies.html, accessed: Aug. 22, 2026. [20] Mem0, “Ai memory that persists across sessions and agents,” https: //mem0.ai/, accessed: Aug. 22, 2026. [21] P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav, “Mem0: Building production-ready AI agents with scalable long-term memory,” arXiv preprint arXiv:2504.19413, 2025. [22] Amazon Web Services, “Preparing user metadata for training,” Amazon Personalize Developer Guide, 2026, accessed July 18, 2026. [23] Adobe, “Real-time customer profile overview,” Adobe Experience Platform Documentation, 2026, accessed July 18, 2026. [24] S. Reddy, H. Beg, A. Overwijk, and S. O’Byrne, “Sequence learning: A paradigm shift for personalized ads recommendations,” Engineering at Meta, 2024. [25] K.-J. Hsiao, Y. Feng, and S. Lamkhede, “Foundation model for personalized recommendation,” Netflix Technology Blog, 2025.

[26] L. Chen, G. Zhang, K. Zhao, D. Cirne, A. Olsen, X. Chu, Z. Miller, A. Blanken, A. Anoun, and J. Ting, “Deployment-time memorization in foundation-model agents,” 2026, iCML MemFM 2026 Workshop. [Online]. Available: https://arxiv.org/abs/2606.10062 [27] N. Carlini, F. Tramèr, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, Ú. Erlingsson, A. Oprea, and C. Raffel, “Extracting training data from large language models,” in 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 2021, pp. 2633–2650. [Online]. Available: https://www.usenix.org/conference/ usenixsecurity21/presentation/carlini-extracting [28] B. Hui, H. Yuan, N. Z. Gong, P. Burlina, and Y. Cao, “PLeak: Prompt leaking attacks against large language model applications,” in Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2024, pp. 3600–3614. [29] Z. Qi, H. Zhang, E. P. Xing, S. Kakade, and H. Lakkaraju, “Follow my instruction and spill the beans: Scalable data extraction from retrievalaugmented generation systems,” in The Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://proceedings.iclr.cc/paper files/paper/2025/hash/ 79cafa874121a3435d8a54f454b646b4-Abstract-Conference.html [30] N. L. B. Nguyen, W. Ma, V. Vo, A. Abuadbba, M. Fang, J. Zhang, and Y. Xiang, “Five queries are enough: Query-Efficient and Surrogate-Free membership inference attacks on RAG via entailment,” in 35th USENIX Security Symposium (USENIX Security 26). USENIX Association, 2026. [Online]. Available: https://www. usenix.org/conference/usenixsecurity26/presentation/nguyen-nguyen [31] B. Wang, W. He, S. Zeng, Z. Xiang, Y. Xing, J. Tang, and P. He, “Unveiling privacy risks in LLM agent memory,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2025, pp. 25 241–25 260. [Online]. Available: https://aclanthology.org/2025.acl-long.1227/ [32] X. Lyu, J. He, N. Wang, Y. Hu, T. Li, D. Chen, S. Li, and Y. Chen, “ADAM: A systematic data extraction attack on agent memory via adaptive querying,” arXiv preprint arXiv:2604.09747, 2026. [33] J. A. Calandrino, A. Kilzer, A. Narayanan, E. W. Felten, and V. Shmatikov, “You Might Also Like: Privacy risks of collaborative filtering,” in 2011 IEEE Symposium on Security and Privacy. IEEE, 2011, pp. 231–246. [Online]. Available: https: //www.ieee-security.org/TC/SP2011/PAPERS/2011/paper015.pdf [34] X. Xin, J. Yang, H. Wang, J. Ma, P. Ren, H. Luo, X. Shi, Z. Chen, and Z. Ren, “On the user behavior leakage from recommender system exposure,” ACM Transactions on Information Systems, vol. 41, no. 3, pp. 57:1–57:25, 2023. [Online]. Available: https://doi.org/10.1145/3568954 [35] E. Debenedetti, G. Severi, N. Carlini, C. A. ChoquetteChoo, M. Jagielski, M. Nasr, E. Wallace, and F. Tram‘er, “Privacy side channels in machine learning systems,” in 33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, 2024. [Online]. Available: https://www.usenix.org/ conference/usenixsecurity24/presentation/debenedetti [36] S. Shayesteh and S. Wilson, “From conventional web privacy to agentic disclosure: How tool schemas may invite LLM oversharing,” in Proceedings of the Seventh Workshop on Privacy in Natural Language Processing. Association for Computational Linguistics, 2026, pp. 1–6. [Online]. Available: https://aclanthology.org/2026. privatenlp-main.1/ [37] N. Balepur, V. Padmakumar, F. Yang, S. Feng, R. Rudinger, and J. Boyd-Graber, “Whose boat does it float? improving personalization in preference tuning via inferred user personas,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2025, pp. 3371–3393. [Online]. Available: https://aclanthology.org/2025.acl-long.168/

[38] X. Fang, W. Xu, Y. Zhang, S. Nickleach, S. Eckman, and C. K. Reddy, “The personalization trap: How user memory alters emotional reasoning in LLMs,” in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 2026, pp. 511– 529. [Online]. Available: https://aclanthology.org/2026.acl-short.43/ [39] R. Staab, M. Vero, M. Balunović, and M. Vechev, “Beyond memorization: Violating privacy via inference with large language models,” in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://proceedings.iclr.cc/paper files/paper/2024/hash/ 9028b8a3ca98f58e373f0c1497a17448-Abstract-Conference.html [40] V. Patil, E. Stengel-Eskin, and M. Bansal, “The sum leaks more than its parts: Compositional privacy risks and mitigations in multi-agent collaboration,” arXiv preprint arXiv:2509.14284, 2025. [Online]. Available: https://arxiv.org/abs/2509.14284 [41] N. Kandpal, K. Pillutla, A. Oprea, P. Kairouz, C. A. ChoquetteChoo, and Z. Xu, “User inference attacks on large language models,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2024, pp. 18 238–18 265. [Online]. Available: https://aclanthology.org/2024.emnlp-main.1014/ [42] J. Luo, Y. Tian, C. Cao, Z. Luo, H. Lin, K. Li, C. Kong, R. Yang, and J. Ma, “From storage to experience: A survey on the evolution of LLM agent memory mechanisms,” in Findings of the Association for Computational Linguistics: ACL 2026. Association for Computational Linguistics, 2026, pp. 41 622–41 652. [Online]. Available: https://aclanthology.org/2026.findings-acl.2069/ [43] P. Rasmussen, P. Paliychuk, T. Beauvais, J. Ryan, and D. Chalef, “Zep: A temporal knowledge graph architecture for agent memory,” arXiv preprint arXiv:2501.13956, 2025. [44] Spotify Engineering, “Exclude from your taste profile,” Spotify Engineering, 2023. [45] Y. Lyu, G. Chen, R. Shao, W. Guan, and L. Nie, “PersonalAlign: Hierarchical implicit intent alignment for personalized GUI agent with long-term user-centric records,” in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), M. Liakata, V. P. Moreira, J. Zhang, and D. Jurgens, Eds. San Diego, California, United States: Association for Computational Linguistics, Jul. 2026, pp. 36 074–36 089. [Online]. Available: https://aclanthology.org/2026.acl-long.1669/ [46] D. Kartik, A. Nayyar, and U. Mitra, “Fixed-horizon active hypothesis testing,” IEEE Transactions on Automatic Control, vol. 67, no. 4, pp. 1882–1897, 2022. [Online]. Available: https://doi.org/10.1109/TAC.2021.3090742 [47] B. Jiang, Y. Yuan, M. Shen, Z. Hao, Z. Xu, Z. Chen, Z. Liu, A. R. Vijjini, J. He, H. Yu, R. Poovendran, G. Wornell, L. Ungar, D. Roth, S. Chen, and C. J. Taylor, “PersonaMem-v2: Towards personalized intelligence via learning implicit user personas and agentic memory,” arXiv preprint arXiv:2512.06688, 2025. [48] Z. Zhao, C. Vania, S. Kayal, N. Khan, S. B. Cohen, and E. Yilmaz, “PersonaLens: A benchmark for personalization evaluation in conversational AI assistants,” in Findings of the Association for Computational Linguistics: ACL 2025. Association for Computational Linguistics, 2025, pp. 18 023–18 055. [49] A. Maharana, D.-H. Lee, S. Tulyakov, M. Bansal, F. Barbieri, and Y. Fang, “Evaluating very long-term conversational memory of LLM agents,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2024, pp. 13 851–13 870. [Online]. Available: https://aclanthology.org/2024.acl-long.747/ [50] LangChain, “LangMem,” Software available at https://github.com/ langchain-ai/langmem, 2025. [51] A. Yang et al., “Qwen3 technical report,” arXiv preprint arXiv:2505.09388, 2025.

[52] W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with PagedAttention,” in Proceedings of the 29th Symposium on Operating Systems Principles, ser. SOSP ’23. Association for Computing Machinery, 2023, pp. 611–626. [Online]. Available: https://doi.org/10.1145/ 3600006.3613165 [53] Y. Wang, W. Zou, R. Geng, and J. Jia, “TracLLM: A generic framework for attributing long context LLMs,” in 34th USENIX Security Symposium (USENIX Security 25). USENIX Association, 2025, pp. 3845–3864. [Online]. Available: https://www.usenix.org/ conference/usenixsecurity25/presentation/wang-yanting [54] Q. Li, J. Hong, C. Xie, J. Tan, R. Xin, J. Hou, X. Yin, Z. Wang, D. Hendrycks, Z. Wang, B. Li, B. He, and D. Song, “LLM-PBE: Assessing data privacy in large language models,” arXiv preprint arXiv:2408.12787, 2024. [55] Q. Zhan, Z. Liang, Z. Ying, and D. Kang, “InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents,” in Findings of the Association for Computational Linguistics: ACL 2024. Association for Computational Linguistics, 2024, pp. 10 471–10 506. [56] X. Fu, S. Li, Z. Wang, Y. Liu, R. K. Gupta, T. Berg-Kirkpatrick, and E. Fernandes, “Imprompter: Tricking LLM agents into improper tool use,” arXiv preprint arXiv:2410.14923, 2024. [57] Y. Liu, Y. Jia, J. Jia, and N. Z. Gong, “Evaluating LLM-Based personal information extraction and countermeasures,” in 34th USENIX Security Symposium (USENIX Security 25). USENIX Association, 2025, pp. 1669–1688. [Online]. Available: https: //www.usenix.org/conference/usenixsecurity25/presentation/liu-yupei [58] OpenClaw, “Memory overview,” https://docs.openclaw.ai/concepts/ memory, 2026, accessed: Aug. 22, 2026. [59] ——, “OpenClaw: Your own personal ai assistant,” GitHub repository, https://github.com/openclaw/openclaw, 2026, repository metrics accessed: Aug. 22, 2026. [60] S. Wang, F. Yu, X. Liu, X. Qin, J. Zhang, Q. Lin, D. Zhang, and S. Rajmohan, “Privacy in action: Towards realistic privacy mitigation and evaluation for LLM-powered agents,” in Findings of the Association for Computational Linguistics: EMNLP 2025. Association for Computational Linguistics, 2025, pp. 17 055–17 074. [Online]. Available: https://aclanthology.org/2025.findings-emnlp.925/ [61] W. Zhang, X. Zhang, C. Zhang, L. Yang, J. Shang, Z. Wei, H. P. Zou, Z. Huang, Z. Wang, Y. Gao, X. Pan, L. Xiong, J. Liu, P. S. Yu, and X. Li, “PersonaAgent: Bridging memory and action for personalized LLM agents,” in Findings of the Association for Computational Linguistics: ACL 2026. Association for Computational Linguistics, 2026, pp. 26 421–26 439. [Online]. Available: https://aclanthology.org/2026.findings-acl.1315/ [62] A. Dash, S. Das, E. Kirsten, Q. Wu, S. K. Karnam, K. P. Gummadi, T. Holz, M. B. Zafar, and S. Zannettou, “The algorithmic self-portrait: Deconstructing memory in ChatGPT,” in Proceedings of the ACM Web Conference 2026. Association for Computing Machinery, 2026, pp. 3471–3482. [Online]. Available: https://doi.org/10.1145/3774904.3792671 [63] Y. Xiang, Z. Zhang, B. Yang, Z. Hong, L. Feng, M. Xu, and T. Liu, “When agents learn to be you: Benchmarking privacy leakage, impersonation risk, and defenses in persona skills,” 2026. [Online]. Available: https://arxiv.org/abs/2608.03700 [64] Y. Cui, R. Yue, H. Fu, S. Pan, Z. Sun, B. Huang, H. Zhang, C. Zuo, and L. Wang, “Spore: Efficient and training-free privacy extraction attack on LLMs via inference-time hybrid probing,” 2026. [Online]. Available: https://arxiv.org/abs/2604.23711 [65] X. Gao, W. Chen, X. Meng, L. Wang, C. Zang, J. Wang, Z. Li, and S. Guo, “Isolated but exposed: Persistence-based memory extraction attack on LLM agents,” 2026. [Online]. Available: https://arxiv.org/abs/2607.23444

[66] J. Morris, V. Kuleshov, V. Shmatikov, and A. Rush, “Text embeddings reveal (almost) as much as text,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2023, pp. 12 448–12 460. [Online]. Available: https://aclanthology.org/2023.emnlp-main.765/ [67] T. Dong, Y. Meng, S. Li, G. Chen, Z. Liu, and H. Zhu, “Depth gives a false sense of privacy: LLM internal states inversion,” in 34th USENIX Security Symposium (USENIX Security 25). USENIX Association, 2025, pp. 1629–1648. [Online]. Available: https://www. usenix.org/conference/usenixsecurity25/presentation/dong-tian [68] M. Fredrikson, S. Jha, and T. Ristenpart, “Model inversion attacks that exploit confidence information and basic countermeasures,” in Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. ACM, 2015, pp. 1322–1333. [Online]. Available: https://doi.org/10.1145/2810103.2813677 [69] N. Z. Gong and B. Liu, “You are who you know and how you behave: Attribute inference attacks via users’ social friends and behaviors,” in 25th USENIX Security Symposium (USENIX Security 16). USENIX Association, 2016, pp. 979–995. [Online]. Available: https://www.usenix.org/conference/ usenixsecurity16/technical-sessions/presentation/gong [70] Y. Wang, M. Tang, N. Shen, S. Cui, and W. Wang, “Privacy risks of LLM-empowered recommender systems: An inversion attack perspective,” in Proceedings of the 19th ACM Conference on Recommender Systems. ACM, 2025, pp. 812–821. [Online]. Available: https://doi.org/10.1145/3705328.3748158 [71] Y. Du, Z. Li, B. Ding, Y. Li, H. Xiao, J. Zhou, and N. Li, “Automated profile inference with language model agents,” in Findings of the Association for Computational Linguistics: ACL 2026. Association for Computational Linguistics, 2026, pp. 9977–10 000. [Online]. Available: https://aclanthology.org/2026.findings-acl.485/ [72] Y. Vekaria, A. L. Canino, J. Levitsky, A. Ciechonski, P. Callejo, A. M. Mandalari, and Z. Shafiq, “Big help or big brother? auditing tracking, profiling, and personalization in generative AI assistants,” in 34th USENIX Security Symposium (USENIX Security 25). USENIX Association, 2025, pp. 8115–8134. [Online]. Available: https: //www.usenix.org/conference/usenixsecurity25/presentation/vekaria [73] B. Rani, “Behavioral privacy leakage in agentic negotiation: Formalizing and mitigating inference attacks via randomized policies,” 2026. [Online]. Available: https://arxiv.org/abs/2607.06815 [74] H. Jeong, M. Teymoorianfard, A. Kumar, A. Houmansadr, and E. Bagdasarian, “Network-level prompt and trait leakage in local research agents,” in 35th USENIX Security Symposium (USENIX Security 26). USENIX Association, 2026. [Online]. Available: https: //www.usenix.org/conference/usenixsecurity26/presentation/jeong [75] N. Mireshghallah, H. Kim, X. Zhou, Y. Tsvetkov, M. Sap, R. Shokri, and Y. Choi, “Can LLMs keep a secret? testing privacy implications of language models via contextual integrity theory,” in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=gmg7t8b4s0 [76] Y. Shao, T. Li, W. Shi, Y. Liu, and D. Yang, “PrivacyLens: Evaluating privacy norm awareness of language models in action,” in Advances in Neural Information Processing Systems 37, 2024. [Online]. Available: https://proceedings.neurips.cc/paper files/paper/ 2024/hash/a2a7e58309d5190082390ff10ff3b2b8-Abstract-Datasets and Benchmarks Track.html [77] A. Zharmagambetov, C. Guo, I. Evtimov, M. Pavlova, R. Salakhutdinov, and K. Chaudhuri, “AgentDAM: Privacy leakage evaluation for autonomous web agents,” in Advances in Neural Information Processing Systems 38. Curran Associates, Inc., 2025. [Online]. Available: https://proceedings.neurips.cc/paper files/paper/2025/ hash/c9826b9ea5e1b49b256329934a578d83-Abstract-Datasets and Benchmarks Track.html

[78] N. Mireshghallah, N. Mangaokar, N. Kokhlikyan, A. Zharmagambetov, M. Zaheer, S. Mahloujifar, and K. Chaudhuri, “CIMemories: A compositional benchmark for contextual integrity in LLMs,” in The Fourteenth International Conference on Learning Representations, 2026. [Online]. Available: https://proceedings.iclr.cc/paper files/paper/2026/ hash/9a2bcfaf383638e166162a25b6dff125-Abstract-Conference.html [79] W. Fu, X. Qin, J. Zhang, Q. Lin, L. Wutschitz, R. Sim, S. Rajmohan, and D. Zhang, “CI-work: Benchmarking contextual integrity in enterprise LLM agents,” in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track). Association for Computational Linguistics, 2026, pp. 1483–1508. [Online]. Available: https: //aclanthology.org/2026.acl-industry.103/ [80] Z. Lin, J. Li, S. Pan, Y. Shi, Y. Yao, and D. Xu, “Mind the third eye! benchmarking privacy awareness in MLLM-powered smartphone agents,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 42, 2026, pp. 35 626–35 634. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/40874 [81] M. Zhang, J. Han, D. Guo, S. Li, G. Wang, N. Zou, D. Liu, and X. Hu, “PrivacyPeek: Auditing what LLM-based agents acquire, not just what they say,” 2026. [Online]. Available: https://arxiv.org/abs/2606.00152 [82] A. Gomaa, A. Salem, and S. Abdelnabi, “ConVerse: Benchmarking contextual safety in agent-to-agent conversations,” in Findings of the Association for Computational Linguistics: EACL 2026. Association for Computational Linguistics, 2026, pp. 3246–3268. [Online]. Available: https://aclanthology.org/2026.findings-eacl.170/ [83] S. Das, J. Sandler, and F. Fioretto, “Beyond jailbreaking: Auditing contextual privacy in LLM agents,” 2025. [Online]. Available: https://arxiv.org/abs/2506.10171 [84] W. Wang, W. Si, X. Xu, and Y. Xu, “What to remember, what to reveal: Privacy-aware memory for conversational agents,” 2026. [Online]. Available: https://arxiv.org/abs/2608.16551 [85] D. Anshumaan, S. Choudhary, N. Palumbo, and S. Jha, “Dependencyaware privacy for multi-turn agents,” 2026. [Online]. Available: https://arxiv.org/abs/2605.03188 [86] H. R. Kirk, A. Whitefield, P. Rottger, A. Bean, K. Margatina, J. Ciro, R. Mosquera, M. Bartolo, A. Williams, H. He, B. Vidgen, and S. A. Hale, “The PRISM alignment dataset: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models,” in Advances in Neural Information Processing Systems, vol. 37, 2024. [Online]. Available: https://proceedings.neurips.cc/paper files/paper/ 2024/hash/be2e1b68b44f2419e19f6c35a1b8cf35-Abstract-Datasets and Benchmarks Track.html [87] Google Cloud, “Cloud security faq,” 2026, accessed September 3, 2026. [Online]. Available: https://support.google.com/cloud/answer/ 6262505 [88] Amazon Web Services, “Aws customer support policy for penetration testing,” 2026, accessed September 3, 2026. [Online]. Available: https://aws.amazon.com/security/penetration-testing/ [89] Google, “Gemini 2.5 Flash-Lite,” Google AI for Developers, https://ai.google.dev/gemini-api/docs/models/gemini-2.5-flash-lite, 2025, accessed: Aug. 26, 2026. [90] Amazon Web Services, “Nova 2 Lite,” Amazon Bedrock documentation, https://docs.aws.amazon.com/bedrock/latest/ userguide/model-card-amazon-nova-2-lite.html, 2025, accessed: Aug. 26, 2026.

infer information about real users. The principal harm is loss of privacy from choices that do not state the inferred information directly, while testing live services may also consume resources or alter personalized state. We judged the controlled study warranted because existing evaluations can miss this channel and measuring it can help providers develop interaction-level defenses. We recruited no participants and accessed no realuser accounts. The real-world validation used synthetic PersonaMem-v2 histories, while the offline evaluation used released benchmark data with mixed provenance. PersonaLens derives demographic fields from participants in the PRISM Alignment Dataset and generates the associated preferences and interaction histories [48], [86]. The institutional ethics review board of The Hong Kong Polytechnic University approved this use of public secondary data. We used only released fields, attempted no re-identification, and report aggregate results. The managed-service experiments used dedicated, author-controlled resources, isolated synthetic identities, documented interfaces, and small request budgets. We requested permission from the providers but received no response. Google permits customers to assess their own Cloud projects without prior notification, and AWS lists AgentCore among services that customers may security-test without prior approval, subject to their policies [87], [88]. The Mem0 evaluation used documented API operations only against state created in our account and did not test provider infrastructure. We did not access third-party accounts, bypass access controls, inspect hidden provider state, or attempt to recover provider source records, and retained only interfacevisible outputs and derived evaluation records. We will report the findings through the providers’ designated security channels. The released implementation excludes account identifiers, credentials, private platform state, raw external-service records, and third-party source data. These safeguards reduce immediate harm without eliminating the method’s dual-use potential, which motivates the bounded threat model and the defense evaluation reported with the attack. Artifact availability.. The code and artifacts accompanying this preprint are available at https://github.com/ haoyangliASTAPLE/umpeek/. It provides the UMPeek and defense implementations, common interfaces, prompts, configurations, preprocessing and scoring code, compact derived evaluation records, and scripts for rebuilding the released results. Third-party datasets, credentials, provider resources, and external services’ internal state are excluded, with acquisition instructions supplied for public data. Versioned releases will also be preserved in a permanent archive.

Appendix A. Ethical Considerations

Appendix B. Experimental Details

UMPeek has a dual-use risk because a method for detecting behavioral privacy leakage could also be used to

Benchmark construction.. PersonaMem-v2 [47] contributes 5,000 multiple-choice interactions from 200 per-

sonas whose earlier conversations encode implicit preferences. PersonaLens [48] contributes a fixed subset of 5,000 task-oriented interactions from 79 users. ETAPP [3] pairs 150 instructions with 32 profiles to yield 4,800 cases in which personalization affects visible tool choices or arguments. LoCoMo [49] contributes 1,523 question-answer pairs from 20 speakers in 10 multi-session conversations. The resulting 16,323 targets are evaluated with each of three local personalization backends, yielding 48,969 targetbackend interactions. For unseen-task evaluation, we select three different requests about the same user or conversation and exclude duplicates or cases without scoreable expected behavior. PersonaLens tasks remain in the same domain when possible, while LoCoMo tasks use related evidence and nearby turns in the same conversation when possible. Semantic recovery.. For target i, let Ri denote the reference claims, let Ŝi denote the recovered estimate, and let Mi denote the one-to-one matches produced by the rule in Section 4. UMR precision is |Mi |/|Ŝi |, UMR recall is |Mi |/|Ri |, and their harmonic mean is UMR-F1i =

2|Mi | |Ri | + |Ŝi |

.

An empty estimate receives zero precision, and reported recovery scores are means of target-level values unless stated otherwise. UMR-F1 uses the 48,941 target-backend evaluations with at least one reference claim. Behavioral prediction.. For every unseen task h associated with target i, a fixed rule uses Ŝi to predict behavior ôih for request qih and compares it with reference behavior oih . The rule selects a supported option for choice tasks, constructs a compatible tool call for ETAPP, and extracts an answer supported by the recovered claims for LoCoMo. The score s(ôih , oih ) uses exact match for choices, a linearly decreasing rank score for ordered outputs, equal weight on tool-name accuracy and argument F1 for tool actions, and token F1 for free-form answers. For N targets, the Held-out Behavior Prediction Score is N

HBPS =

1 X 1 X s(ôih , oih ) , N i=1 |Hi | h∈Hi

where Hi is the set of unseen tasks paired with target i. Each unseen task is selected so that user-specific information relevant to the target interaction can also affect its reference behavior.

Appendix C. Real-World Validation Details Retention and managed-service protocol.. To evaluate exposure of information that a service actually retained, we first test whether its personalized state contains information semantically equivalent to the target reference claim. Only targets that pass this check enter the behavior-based comparison, and no attack method receives the retention result. Google Memory Bank, AWS AgentCore Memory, and

Mem0 Platform each contribute 20 retained targets evaluated with all eight methods under a cap of three total requests per target. Google and Mem0 produce visible behavior with Gemini 2.5 Flash-Lite [89], while AWS uses Amazon Nova 2 Lite [90]. All managed-service results were collected in August 2026. OpenClaw protocol.. We instantiate 50 OpenClaw agents from synthetic personas and test 206 reference claims for retention. OpenClaw retains 28 claims across 25 personas, from which we select one retained claim per persona to obtain 25 targets. These targets are not used to rank the eight methods in the managed-service comparison.

Appendix D. Adaptive Defense Details Protocol.. The main comparison samples 128 targets from every benchmark-backend pair and allows the initial interaction plus at most 16 follow-ups. The retained personalized state, attack procedure, and request cap remain fixed across undefended and defended conditions. UMPeek selects later requests using only behavior released by the defense. TaskScore applies each benchmark’s original task metric [3], [47], [48], [49] to the evaluated personalized tasks. The threshold study uses 32 targets from every benchmarkbackend pair and reuses the same targets at thresholds 0.5, 1.0, and 1.5. Mechanism ablations use the same subset at threshold 1.0. For uncertainty estimates, outcomes for the same benchmark target across the three local backends form one resampling cluster, and macro scores are unweighted means over the four benchmarks.

Related documents

• SpliTEE: Fast and Private LLM Inference by Coupling GPU-Assisted Trusted Execution Environments with Differential Privacy#1117564
• On the Internet, Nobody Knows You're an LLM Bot: Unmasking Web Agents with Multi-Layer Fingerprinting#321748
• PrivacySIM: Evaluating LLM Simulation of User Privacy Behavior#178794
• Profiling for Pennies: Unveiling the Privacy Iceberg of LLM Agents#160725
• Persona-Execution Separation: An Architecture Pattern for Evolving LLM Agents under Execution Audit#1084343