Agentic LLMs with web search change the threat model for text anonymization: weak contextual cues can become cross-referenceable evidence for re-identification, yet those same details also carry downstream analytic value of the text. Existing defenses either remove explicit identifiers, perturb text for formal privacy, or test rewritten text against non-web inference models, leaving underexplored the operating region between resistance to agentic web-search re-identification and utility retention.
We introduce AURA (Anonymization with Utility-Retention Adaptation), an LLM-powered mask–reconstruct framework that decouples privacy localization from utility-preserving reconstruction and selects candidates with adversarial privacy and utility-retention checks. We evaluate AURA on real-user interview transcripts using re-identification attacks carried out by web-search agents, along with a utility evaluation based on interviewee-profile facts, codebook facts, and the joint contextual utility grid. Our results show that adaptive-scope AURA yields the lowest agentic re-identification counts under each of three attacker models among the non-DP methods, and that at matched scope and backbone, AURA’s mask–reconstruct design retains more contextual utility than the prior LLM anonymizer (+6.4 percentage points in unit-grid recovery) at comparable privacy.
AURA separates the two pieces of an anonymization decision: where to intervene (privacy localization) and how to rewrite (utility-preserving reconstruction). The pipeline runs in three phases, each producing artifacts the next phase can consume — and each amenable to a different model or human-in-the-loop control.
Run a one-off agentic web-search attacker on the transcript to discover not only the base 8 privacy attributes (Age, Sex, Location, Occupation, Education, Relationship, Income, Place of Birth) but also transcript-specific quasi-identifiers — workflow cues, research-pipeline signatures, tool-stack mentions, domain-practice cues, and institutional context. The output is an adaptive privacy scope A, a blacklist B of evidence spans, and a utility insight profile P over 8 dimensions (Theme, Experiential, Affect, Reasoning, Behavioral, Relational, Temporal, Expertise).
An iterative rewriter alternates with LLM-based privacy inference for up to
Rmask rounds: each round infers attributes in scope
A, rewrites ti to suppress them, and stops when no
attribute can be inferred or the round budget is reached. A diff between the original
and the converged rewrite yields a masked template T̂ with
[MASK_i] placeholders, a mask map M linking placeholders
back to original spans, and seed replacements for the next phase.
For the masked template only, the reconstructor proposes N candidate replacement dictionaries, producing N candidate rewrites. Each candidate is scored by an attacker (privacy severity S and specificity count C) and a keeper (utility loss L). Selection is privacy-first: filter to candidates with C ≤ Cmax, then minimize (S, L); otherwise minimize (C, S, L) in order.
We evaluate three privacy-scope variants and multiple LLM backbones:
We evaluate on 53 re-identifiable interview transcripts from the Anthropic Interviewer dataset. We screen all 1,250 transcripts with an agentic web-search attack and retain cases with verifiable identification evidence. Every defense is evaluated on both privacy and utility.
Each attacker receives a fresh request containing only the attack prompt and rewritten transcript, without Phase 0 traces. We report successful re-identification counts out of 53 under three attackers:
Adaptive Qwen privacy scopes are generated with DeepSeek-V4-Flash and web search, holding out all three evaluation attackers from scope generation.
Recoverability is judged by DeepSeek-V4-Flash, a different model family from the GPT-4.1 and Qwen rewriting backbones. Grid recovery is weighted over all validated units.
We compare against a comprehensive set of baselines spanning the space:
Across three web-search attackers, AURA’s adaptive-scope variants yield the lowest non-DP re-identification counts under each attacker: 0–7 / 53, compared with 26–40 / 53 for Presidio and 10–12 / 53 for the prior LLM anonymizer. Adaptive-privacy AURA retains 72.0–76.1% of unit-level utility-grid information.
Re-identification count and rate under each attacker. Lower is better. The highlighted row matches Table 1 of the preprint. No model refusal was observed.
| Method | GPT-5.1 | GPT-5.4-mini | Gemini-3-Flash |
|---|---|---|---|
| AURA variants | |||
| AURA (adapt. privacy, Qwen3.5-35B-A3B) | 2 / 53 (3.8%) | 7 / 53 (13.2%) | 2 / 53 (3.8%) |
| AURA (adapt. privacy, Qwen3.5-27B) | 4 / 53 (7.5%) | 7 / 53 (13.2%) | 0 / 53 (0.0%) |
| AURA (adapt. privacy, GPT-4.1) | 4 / 53 (7.5%) | 6 / 53 (11.3%) | 0 / 53 (0.0%) |
| AURA (pure adaptive, GPT-4.1) | 2 / 53 (3.8%) | 7 / 53 (13.2%) | 3 / 53 (5.7%) |
| AURA (8-attribute, GPT-4.1) | 8 / 53 (15.1%) | 14 / 53 (26.4%) | 10 / 53 (18.9%) |
| AURA (8-attribute, Qwen3.5-27B) | 5 / 53 (9.4%) | 9 / 53 (17.0%) | 4 / 53 (7.5%) |
| AURA (8-attribute, Qwen3.5-35B-A3B) | 3 / 53 (5.7%) | 12 / 53 (22.6%) | 5 / 53 (9.4%) |
| Non-DP baselines | |||
| Anonymizer [Staab+ 2025] | 11 / 53 (20.8%) | 12 / 53 (22.6%) | 10 / 53 (18.9%) |
| Presidio | 26 / 53 (49.1%) | 40 / 53 (75.5%) | 27 / 53 (50.9%) |
| One-shot rewrite (minimal) | 12 / 53 (22.6%) | 18 / 53 (34.0%) | 11 / 53 (20.8%) |
| One-shot rewrite (detailed) | 24 / 53 (45.3%) | 33 / 53 (62.3%) | 23 / 53 (43.4%) |
| DP-MLM baselines | |||
| DP-MLM (ε = 10) | 0 / 53 (0.0%) | 0 / 53 (0.0%) | 0 / 53 (0.0%) |
| DP-MLM (ε = 30) | 2 / 53 (3.8%) | 0 / 53 (0.0%) | 0 / 53 (0.0%) |
| DP-MLM (ε = 50) | 9 / 53 (17.0%) | 5 / 53 (9.4%) | 2 / 53 (3.8%) |
| DP-MLM (ε = 70) | 10 / 53 (18.9%) | 9 / 53 (17.0%) | 3 / 53 (5.7%) |
| DP-MLM (ε = 100) | 11 / 53 (20.8%) | 13 / 53 (24.5%) | 3 / 53 (5.7%) |
| DP-MLM (ε = 120) | 11 / 53 (20.8%) | 13 / 53 (24.5%) | 6 / 53 (11.3%) |
| DP-MLM (ε = 140) | 8 / 53 (15.1%) | 10 / 53 (18.9%) | 4 / 53 (7.5%) |
Counts are empirical outcomes, not guarantees of anonymity. At 0 / 53, the exact 95% binomial interval is 0–6.7%. See Appendix I for all intervals.
Utility preservation (preprint Figure 2, bottom). Results across 53 transcripts, with 340 profile facts, 732 codebook facts, and 4,758 utility-grid units. Dashed lines mark adaptive-privacy AURA (Qwen3.5-35B-A3B), which has the highest unit-grid recovery among AURA configurations.
Full utility recovery (%) from preprint Table 6. Grid (unit) is the weighted recovery rate over all 4,758 validated profile-code units. Higher is better.
| Method | Profile | Codebook | Grid (unit) |
|---|---|---|---|
| AURA variants | |||
| AURA (adapt. privacy, Qwen3.5-35B-A3B) | 75.0% | 97.5% | 76.1% |
| AURA (adapt. privacy, Qwen3.5-27B) | 74.4% | 97.8% | 76.0% |
| AURA (adapt. privacy, GPT-4.1) | 70.3% | 96.4% | 72.0% |
| AURA (pure adaptive, GPT-4.1) | 69.7% | 94.7% | 69.4% |
| AURA (8-attribute, GPT-4.1) | 75.3% | 95.1% | 72.7% |
| AURA (8-attribute, Qwen3.5-27B) | 73.2% | 97.4% | 74.7% |
| AURA (8-attribute, Qwen3.5-35B-A3B) | 71.5% | 97.4% | 72.9% |
| Non-DP baselines | |||
| Anonymizer [Staab+ 2025] | 67.4% | 95.9% | 66.3% |
| Presidio | 97.9% | 97.7% | 95.6% |
| One-shot rewrite (minimal) | 88.8% | 97.7% | 87.0% |
| One-shot rewrite (detailed) | 97.6% | 97.4% | 96.5% |
| DP-MLM baselines | |||
| DP-MLM (ε = 10) | 0.0% | 0.0% | 0.0% |
| DP-MLM (ε = 30) | 22.1% | 47.1% | 14.1% |
| DP-MLM (ε = 50) | 72.1% | 68.7% | 51.2% |
| DP-MLM (ε = 70) | 71.8% | 74.7% | 56.4% |
| DP-MLM (ε = 100) | 74.1% | 72.4% | 55.2% |
| DP-MLM (ε = 120) | 79.1% | 73.8% | 59.1% |
| DP-MLM (ε = 140) | 77.1% | 74.9% | 59.3% |
At the same 8-attribute scope and GPT-4.1 backbone, AURA retains 72.7% of utility-grid units, compared with 66.3% for the anonymizer. The paired difference is +6.4 percentage points (nominal 95% CI: 0.1 to 12.7), using a paired transcript-cluster bootstrap with 100,000 resamples. Average re-identification counts across attackers are 10.67 / 53 and 11 / 53, respectively. This comparison holds scope and backbone fixed to examine the contribution of mask–reconstruct.
Adaptive-privacy AURA retains 72.0–76.1% unit-grid recovery with 0–7 / 53 re-identifications. DP-MLM trades utility for lower re-identification, while Presidio and one-shot rewriting preserve more utility with higher re-identification. Table 7 reports the paired utility comparisons.
Privacy–utility trade-off (preprint Figure 2, top). Privacy success (100 − re-identification rate) versus unit-grid recovery under GPT-5.4-mini, which has the highest average re-identification rate in this evaluation. The frontier connects observed operating points and does not establish statistical dominance.
A synthetic illustration of how AURA generalizes specific details while preserving surrounding context. These examples illustrate editing behavior and are not evaluation data.
I work in applied sensor physics, specifically on detecting weak environmental fields using tabletop interferometry. We wrote a paper, currently in review, which proposed a generalised mechanism for modelling the noise produced by moving calibration objects.
I work in scientific research that involves studying noise from environmental vibrations on sensitive measurement equipment. We completed a project that proposed a general approach for modeling the noise from moving sources.
I work in a technical field. A recent project involved analyzing the impact of external factors on data collection tools. I developed a general approach for modeling the influence produced by changing conditions.
Fact annotation checks and human judgments of rewrite quality, reported in Appendices J–K.
91 / 105
Facts with matching labels from two independent expert annotators (86.7% agreement; Gwet’s AC1 = 0.845).
This measures inter-expert agreement. It does not measure expert–LLM agreement.
68 / 71
Resolved crowd labels agree with the DeepSeek-audited reference (95.8% agreement; Gwet’s AC1 = 0.948).
41 of 102 recruited participants were retained. Of 75 facts, 71 had a resolved majority and four were tied.
86.9% faithful
106 of 122 retained rewrite pairs were judged to introduce no false information.
59 of 92 recruited participants were retained. Of 150 pairs, 28 with tied faithfulness votes were excluded. Results are pooled across the evaluated rewrites.
Among the same 122 retained pairs, 74 (60.7%) were judged equally easy or easier to read than the original, 11 (9.0%) favored the original, and 37 (30.3%) had tied readability votes. Equal readability and tied votes are distinct outcomes.
The results are consistent with two distinct contributions: scope design improves resistance to agentic re-identification, while mask–reconstruct preserves contextual utility at comparable privacy. The masked spans also provide an inspectable map of the details treated as privacy risks under the chosen scope.
AURA provides empirical protection against the evaluated attackers, with no formal differential-privacy or k-anonymity guarantee. The benchmark is a selected set of 53 vulnerable transcripts; fact recovery approximates qualitative utility and does not directly measure end-to-end analyst performance. Zero observed re-identifications does not establish zero risk against future attackers.
Responsible release remains a process of reviewing residual risks and evaluating multiple attackers. We report aggregate results, withhold identifying evidence and attack traces, and use synthetic examples to prevent localization. The study was deemed exempt by our institutional review board under the Secondary & Specimen Protocol category.
@misc{li2026llmanonymizationagenticreidentification,
title={LLM Anonymization Against Agentic Re-Identification},
author={Ziwen Li and Jianing Wen and Tianshi Li},
year={2026},
eprint={2605.30848},
archivePrefix={arXiv},
primaryClass={cs.CR},
url={https://arxiv.org/abs/2605.30848},
}