Beyond the Document: Compiled Context and Governed Agents for Any Work Product

Beyond the Document: Compiled Context and Governed Agents for Any Work Product James Rhodes George Kang AlphaBitCore, Inc. {james,george}@alphabitcore.com

arXiv:2611.13055v1 [cs.SE] 9 Nov 2026

November 2026

Abstract

evidence lineage, or the rules under which the work was approved. A subsequent question often sends the recipient back to the author or into a general-purpose assistant that can reason about the visible document but does not automatically inherit the producing organization’s evidence or authority boundaries. The limitation is not only how questions are answered. It is what gets delivered. Thesis. The unit of delivery for professional knowledge should be more than a document. A completed work product should travel with its compiled context, evidence and interaction rules, so that another model or runtime can help a recipient interrogate the finished work without reconstructing the original workflow. The producing process and the consuming process need not be the same. Explanation of approved work should also remain distinct from new analysis, new advice or revision requiring fresh authorization. We study one implementation of this thesis, the Portable Agent Format (PAF). A package combines the presentation the reader sees with typed facts, searchable source records, evidence references, provenance classifications, signed release metadata, governance policy and a runtime profile. Its hosted viewer (Lens) answers against the package under a defined policy; a second reader uses a small local model and a package-specific selection and writing pipeline. The format carries knowledge, not the models or agent processes that produced it. A signature records what was issued; it does not make every statement true. Testable hypothesis. On questions about a delivered work product, a reader with compiled context and bounded retrieval should recover more of the work’s knowledge than a reader with the visible source alone, even after giving that reader the same instructions and ordinary access to the underlying source files. A smaller offline reader should retain useful access to that context, though its retrieval may be less robust. Earlier exploratory evaluations used packages during reader development, so they could not provide held-out estimates. We therefore created four new packages on 1 November 2026, after fixing the hosted and offline readers, and froze the questions and gold answers before the included evaluation. The packages were not used for reader development or adapter training. We report only these new results, span-

A document preserves the visible result of professional work but often discards the research, calculations, assumptions, evidence, and authorization boundaries behind it. We study a different unit of delivery: a Portable Agent Format (PAF) package that carries a work product together with compiled context, evidence, provenance, and rules for subsequent interaction. To test whether this context remains useful beyond the process that produced it, we evaluate four new packages, created after the hosted and offline readers were fixed: an investment proposal, fund research, financial-statement analysis, and an engineering incident review. Two packages were produced by specialized agents and two through a generalpurpose assistant’s packaging skill. Across 400 frozen questions and 12,000 evaluated responses, a hosted governed reader succeeds on 96% of questions versus 86% with instructions plus raw-file retrieval (Holm-adjusted 𝑝 < 10−7 ). On the 260 answerable questions per condition, success is 94% versus 79%; the gap varies by package. Source-material chat and instructions alone reach 48% and 74% overall. A separate 0.8-billion-parameter offline reader succeeds on 69% overall, but 54% on answerable questions. Its −1 response rate is 2.9%, combining materially wrong and unauthorized answers, versus 56% with the source and instructions alone. Ablations suggest advantages for the integrated selector and package-adapted writer, without isolating representation from training effects. The same run exposes a failure: offline retrieval resolves none of the 40 historical-record questions in one financial-data package. These findings support treating compiled knowledge as part of the delivered work product, while leaving independent standards interoperability and endto-end governance conformance unproven.

1

Introduction

Professional work commonly arrives as a PDF, web page, presentation, or report. The recipient receives a conclusion but not the full state that made it possible: source records, intermediate analysis, alternatives considered, qualifications, 1

Beyond the Document: Rhodes & Kang (2026)

ning two producer paths in finance and engineering. The lineage. Content that must not be disclosed should be exconditions and limitations appear in §4 and §6. cluded before issuance: limiting a model’s response cannot hide bytes that a recipient can inspect in the package. Four findings organize the paper: 1. Compiled context adds usable knowledge beyond prompting and raw retrieval. Across four packages, hosted Lens reaches 96% success against 48% for ordinary source chat, 74% for the same source with Lens rules, and 86% for Lens rules plus BM25 over the source files. The increment over raw retrieval is nonuniform across packages (§5.1).

Explanation versus revision. A runtime can use a package to explain information present in the approved context. A question requiring external data, additional computation, a novel recommendation or an altered decision should be declined or routed for new work, not silently rewrite the signed artifact. The evaluation tests the answer-or-decline behavior 2. The architecture works on different document and using a deliberately constructed decline stratum; it does not production paths. The evaluation includes two packages test end-to-end escalation, revocation, signature-trust enforceproduced by Prime agents and two from Claude chats ment or approval of a successor release. through an ABC-authored packaging skill, including an engineering postmortem. It demonstrates a second path into the same governed publication ecosystem, not independent format conformance (§3). 3 Four New Packages 3. A much smaller reader benefits from packaged context and a purpose-built selector. On the same packages, the offline 0.8B reader reaches 69% overall success, including 54% on answerable questions; 2.9% of responses receive a −1 rating. The ablations implicate retrieval and writer adaptation without fully isolating their effects (§5.3).

All four packages were generated on 1 November 2026, after the hosted and offline reader implementations had been frozen. Two were produced by an ABC Prime agent workflow; two were constructed in a Claude chat with an ABCauthored paf-from-chat skill and sealed and approved through ABC. These are distinct production paths into the 4. Portability of knowledge does not eliminate retrieval same publication system, not independently specified or indefailure. On one package containing numerous near- pendently implemented PAF producers. Table 1 summarizes duplicate XBRL observations, the offline viewer fails all their contents and the comparison documents. 40 record-level questions despite 100% hosted record success. This exposes an unresolved selection and disambiguation problem (§5.4). Different input and information structures. The investment proposal contains a multi-account recommendation whose supporting analysis is not fully present in the client 2 The Portable Work Product PDF. The fund case binds a fund-profile source to separately A PAF package Π separates six functions. Its presentation 𝑃 compiled research. The Apple package starts with two large is the human-readable deliverable; compiled context 𝐾 con- SEC XBRL company-concept files whose reported periods, tains typed facts, conclusions, rationales and conditions; evi- fiscal labels, filings and observations contain many neardence 𝐸 contains source records, quotations and provenance; duplicates; the distilled five-year analysis is far smaller. The governance 𝐺 declares permitted retrieval and interaction; a engineering package summarizes an AWS incident and its manifest 𝑀 and signatures Σ bind the release to identifiable impact windows, starting from an already coherent post-event bytes and signers; and a runtime profile 𝑅 declares how a con- report. These cases therefore vary in how much compilation sumer opens it. The package contains no answering-model or retrieval selection should be required. weights. These roles are illustrated in Figure 1. Evidence, approval and boundaries. Each package contains a manifest of payload hashes and both producer and approver signature files. The test bundle provides the four decrypted package archives; their archive SHA-256 digests match the study room records, and the listed payload hashes recompute. This demonstrates consistency of supplied bytes with the manifest, not an independent validation of a signature trust chain, approval identity or arbitrary factual accuracy. The packages include provenance records, but an author’s declaration of source provenance is not itself verified execution

Holding the artifacts fixed. The frozen study lists unique package identifiers and SHA-256 archive digests for all four. Both readers were evaluated on these same published artifacts. The protocol states that the packages had not been used to train, debug or tune the readers. The question sets were written from the packages and frozen before any evaluated answer was generated. This is held-out package content, not a random sample of naturally occurring documents, and the questions intentionally cover facts beyond the visible document. 2

Beyond the Document: Rhodes & Kang (2026)

Work by agents or people

Compile and approve

Portable PAF

Governed reader

external or attested origin

facts, records, policies

signed, versioned package

hosted Lens or local viewer

Presentation 𝑃

Context 𝐾

Evidence 𝐸

Governance 𝐺

Manifest / signatures 𝑀, Σ

Runtime 𝑅

Figure 1: The delivery unit is the package, not the original conversation or producing agent system. The recipient’s reader uses declared context and policy; integrity verification and actual enforcement are separate properties to test. Package and subject

Producer path

Baseline supplied to source readers

Typed facts

Records

Whitaker portfolio proposal DFEOX fund analysis

Prime agents

Client proposal PDF

175

2,053

Prime agents

Sponsor-profile HTML reconstruction

99

296

Apple financial statements AWS outage RCA

Claude chat skill + ABC issue Claude chat skill + ABC issue

Two SEC company-concept XBRL JSON files

19

16

Proposal PDF, narrative, HTML Fund analysis PDF, narrative, HTML Financial analysis report

Presentation in PAF

Published AWS post-event summary

10

21

Engineering postmortem

Table 1: Fresh study packages and the input available without the compiled package. The profiles and source exports are files frozen with the study. A source-only baseline is not equivalent to a fully informed author, and the source material differs in complexity across cases. Counts were inspected in the supplied package archives.

4

Evaluation Protocol

4.1

Questions, scoring and unit of analysis

respect. The deliberate 25/40/35 weighting is not an estimate of the distribution of real user questions.

4.2

Each package has 100 questions: 25 MEMO questions answerable from its delivered report, 40 RECORD questions whose answer exists in package records but not in that report’s headline text, and 35 DECLINE questions requiring an unsupported scenario, new recommendation, outside information or other action the package does not authorize. Candidate memo and record facts were enumerated and drawn using fixed seeds, with logged replacements of unusable candidates; the decline questions were written to fixed topical quotas and audited against the package. A Claude-based question writer and scripted value checks constructed the gold answers. Every model answers each question three times per condition, for 12,000 evaluated answers: 4 packages × 100 questions × 3 repeats × (4 hosted + 6 offline conditions). A further 60 hosted answers per package were regraded as blind anchors in the offline batch; those 240 observations are not counted as new study answers. Two blind judges, Claude Fable 5.1 and Claude Opus 5.5, saw the same anonymized answers with the question, gold, required values and stratum, but not the condition. Answers were scored 2 (correct and complete), 1 (partial), 0 (unhelpful without substantive harm) or −1 (wrong or an unauthorized answer to a decline question). Question-level success means a score of 2 on at least two of three repetitions. We report the fraction of individual responses scored −1 as the minusone response rate, separating materially wrong answers to MEMO/ RECORD questions from unauthorized answers to DE CLINE questions. This stratum-based split is derived from judge ratings, not an independent error taxonomy. Neither outcome measures actual financial loss, and a response not scored −1 is not necessarily correct or authorized in every

Hosted comparisons

All hosted conditions use Claude Sonnet 5.5. They form a ladder of system comparisons rather than one-component ablations: • C1, source chat: the baseline document(s) in an ordinary assistant, with general knowledge and web access available, ten questions per conversation. • C2, rules only: the baseline and Lens’s served system instructions, without the package’s supporting records or external web access; each question starts a fresh call. • C4, plain retrieval: C2 plus BM25 retrieval over the raw files indexed by the package (1,500-character chunks, 200character overlap, at most 16,000 characters; 𝑘1 = 1.5, 𝑏 = 0.75). • C3, full package: Lens in the hosted Client Room, over the signed package’s compiled context and governed retrieval, one fresh session per question. C3 minus C4 is the practical difference between two complete readers with the same broad underlying source corpus. It does not isolate typed-fact compilation from retrieval orchestration, chunking, runtime controls or the use of multiple retrieval steps. C1 to C2 also changes web access and conversation context. We treat C4 as the stronger baseline for incremental value beyond simply making the source files available.

4.3

Offline comparisons

The local viewer uses Qwen3.5-0.8B, Q4_K_M quantization, llama.cpp, and the frozen v2 selection and writing adapters 3

Beyond the Document: Rhodes & Kang (2026)

Package

C1 C2 C4 C3

Whitaker DFEOX Apple RCA

34 27 60 71

62 51 92 92

80 72 97 96

90 96 99 99

Pooled

48

74

86

96

Measure

C1

C2

C4

C3

MEMO success (%) RECORD success (%) Answerable success (%) DECLINE success (%) −1: wrong answer (𝑛) −1: unauthorized answer (𝑛) Total −1 responses (%)

72 60 65 17 35 205 20.0

70 56 61 99 19 3 1.83

83 77 79 99 5 2 0.58

98 92 94 99 1 2 0.25

Table 2: Hosted success (%, score 2 in at least two of three repetitions). Primary blind judge; 100 questions per package. Table 3: Hosted pooled outcomes. Answerable combines 25 Pooled displayed rates are rounded to whole percent. MEMO and 40 RECORD questions per package (260 pooled). Wrong-answer and unauthorized-answer counts partition −1 ratings by answerable versus decline stratum; each condition (sorter v16, writer v7). The six conditions are O1 (source has 1,200 responses. with neutral prompt), O2 (source with Lens instructions), O4 (the ordinary retrieval passages with Lens instructions), O4w (the viewer’s trained writer over those raw passages), ble 2 shows the large range across cases. The paired pooled O5 (the same trained writer over the package’s top twelve differences are +48 percentage points versus C1 (95% CI BM25-ranked lines without its learned selection stage), and 43–53, Holm 𝑝 = 2.5 × 10−52 ), +21.75 versus C2 (CI O3 (the full viewer with package selection and writing). Sam18–26, 𝑝 = 1.7 × 10−21 ), and +9.75 versus C4 (CI 7–13, pling is repeated with three fixed seeds. The comparison O5 𝑝 = 1.8×10−8 ). The second judge gives the same directional versus O4w probes the effect of package representation with findings. a common writer; O3 versus O5 probes the added selection The result against plain retrieval is not uniform. On stage. Neither is a perfect equal-token-budget ablation, and Whitaker, C3 exceeds C4 by 10 points (Holm 𝑝 = 0.013), O3 also includes its integrated response handling. and on DFEOX by 24 (𝑝 = 3.0 × 10−6 ). On Apple it adds The two Prime-produced packages declare offline mode dis2 (𝑝 = 0.62), and on RCA 3 (𝑝 = 0.38). Thus the pooled abled, whereas the chat-built packages declare offline mode difference does not establish a similarly sized incremental enabled. The study evaluates answer behavior by opening benefit in every kind of package. The largest raw-document these fixed artifacts in its offline harness; it does not demongaps occur where the source artifact withholds much of the strate that a conforming viewer would enforce each package’s finished research; the smallest occur where the source files offline-policy flag or verify its signatures against a trust store. already offer structured facts or a well-written post-event Those are separate conformance tests. explanation. The 35% DECLINE quota rewards appropriate refusals. On the 260 answerable questions per condition (MEMO plus 4.4 Statistical comparisons RECORD), C3 reaches 245/260 (94.2%) against C4’s 206/260 Paired success is assessed by exact McNemar tests at question (79.2%), a descriptive 15.0-point difference. C1 reaches level, with 95% Newcombe intervals for paired differences. 64.6% and C2 61.2% on answerable questions despite C2’s Reported 𝑝-values are Holm-adjusted within each compari- higher overall rate: instructions improve refusal behavior son family: hosted C3–C1, C3–C2, C3–C4; offline O3–O2 without adding missing facts. Within the record-only stratum, and O3–O4w; ablations O5–O4w and O3–O5. We report C3 reaches 92%, C4 77% and C2 56%. Those questions both individual packages and pooled 400-question results and intentionally sample facts beyond the visible report, so C4, show the primary Fable judge, with agreement from Opus. which receives raw-source retrieval, is the more informative Pooling treats questions as the sampled units, not independent control. package replications; with only four packages, inference to a population of tasks remains limited. Correct answers versus authorized refusals. Of C1’s 240 responses scored −1, 205 answer questions that should have been declined and 35 are materially wrong answers to an5 Results swerable questions. C3 has three such responses in total: 5.1 Finding 1: compiled context expands reach one wrong answer and two unauthorized answers. Most of the reduction therefore reflects better adherence to the under a bounded reader intended boundary, not merely improved factual accuracy. Across the four packages, the hosted full-package condi- Rules already account for most of the gain on declines, while tion succeeds on 96% of questions, compared with 48% compiled context and retrieval increase answerable-question for ordinary source chat, 74.25% with Lens instructions success. The experiment does not isolate deterministic gating alone, and 86.25% when raw-file retrieval is added. Ta- from prompts or retrieval, and these ratings do not establish 4

Beyond the Document: Rhodes & Kang (2026)

Package

O1 O2 O4 O4w O5 O3

Whitaker DFEOX Apple RCA

4 11 3 17

13 24 7 37

5 17 22 28

36 35 37 37

48 66 38 52

64 77 51 84

Pooled Answerable only

9 12

20 20

18 23

36 5

51 28

69 54

Outcome

O2 O4w O5 O3

MEMO success (%) 20 RECORD success (%) 19 Answerable success (%) 20 DECLINE success (%) 21 −1: wrong answer (𝑛) 469 −1: unauthorized answer (𝑛) 207 Total −1 responses (%) 56.3

8 54 80 3 13 38 5 28 54 94 93 96 64 74 27 10 10 8 6.2 7.0 2.9

Table 4: Offline success (%) on 400 questions; the final row Table 5: Offline pooled outcomes. Answerable success uses restricts to 260 answerable questions. O4w and O5 use the 260 MEMO and RECORD questions. Wrong-answer and frozen trained writer; O3 also uses the package selector. unauthorized-answer counts partition −1 ratings by question stratum (1,200 responses per condition). Decline success materially boosts pooled rates, especially O4w. universal policy conformance.

5.2

The offline ablation helps locate the performance differFinding 2: one package contract spans proence, but is not a clean test of representation alone. The same duction paths

trained writer increases from 36.25% over raw BM25 pasBoth hosted Lens and the offline reader consumed packages sages (O4w) to 51% over the package’s BM25 lines (O5), a produced from ABC Prime workflows and packages authored +14.75-point difference (CI 10–19, Holm 𝑝 = 4.4 × 10−10 ). in a Claude chat then sealed by ABC. The Apple and RCA The trained writer obtains only 8% MEMO success with raw hosted conditions each reach 99% success; C4 is already passages, below the unadapted O2 reader’s 20%. A plausible at 97% and 96%, respectively. These packages establish explanation is distribution mismatch: the writer was adapted cross-path consumption, including a case outside financial to package-style input, not arbitrary raw-file chunks, but this services, without demonstrating fully independent produc- was not independently tested. Restoring the viewer’s learned tion or cross-vendor conformance. The packaging skill, the selection stage (O3) adds another 18 points over O5 (CI 14– signing pipeline, the format schemas and both readers remain 22, Holm 𝑝 = 3.7 × 10−14 ). These comparisons demonstrate differences between integrated configurations; the observed within the ABC ecosystem. A key architectural observation is distinct from a perfor- gains may reflect representation, training compatibility, remance comparison: the consuming reader does not rerun the trieval selection and context budgets together. producing workflow. It uses the delivered package’s facts, records, evidence and rules. The study does not directly measure recipient adoption, operating cost, customer trust, 5.4 Finding 4: the held-out run reveals a retrieval failure cryptographic trust-chain enforcement or the quality of the original producing work; those require separate tests. In the Apple package, the offline viewer’s overall success is 51%, but its success on the 40 RECORD questions is 0%. On 114 of the 120 individual record-answer attempts it responds: 5.3 Finding 3: a small offline model retains “The proposal doesn’t state that. I can pass the question to much of the packaged knowledge the sender.” By contrast, hosted Lens succeeds on 100% of The integrated 0.8B offline viewer (O3) succeeds on 69% those 40 questions. The questions ask for specific reporting of all questions, compared with 20.25% for the same small periods and filings among many near-duplicate observations model given the baseline and Lens instructions (O2), and of revenue and net income. The frozen offline selector does 36.25% for the trained viewer writer given raw-file passages not reliably map the reader’s date and period qualifiers to the (O4w). The paired differences are +48.75 points (95% CI precise XBRL row; it can find nearby amounts but not the 43–54, Holm 𝑝 = 7.3 × 10−45 ) and +32.75 (CI 28–37, 𝑝 = requested fact. A conservative writer therefore often declines 3.2 × 10−34 ), respectively. On answerable questions alone, instead of hallucinating, at a large cost in coverage. O3 reaches 141/260 (54.2%), O2 reaches 51/260 (19.6%), This failure also explains why the Apple ablation is differand O4w only 13/260 (5.0%). Of O3’s 276 overall successes, ent: O5 and O4w differ by only one overall point, with no 135 are correct declines, so the 69% pooled rate should not statistically detectable advantage for package lines alone. The be read as 69% answer coverage. Responses scored −1 fall problem was not fixed before collecting the results. Altering from 56.33% under O2 to 2.92% under O3; these include the selector after observing it would have converted this set both wrong answers and unauthorized responses, separated into another development evaluation. The result demonstrates in Table 5. These are within-model comparisons, not a claim both the utility and the limits of an offline, compact reader: of parity with the hosted system. a packaged body of evidence can be portable while precise 5

Beyond the Document: Rhodes & Kang (2026)

record selection is not automatically portable across runtimes. Systems are not component-matched. The hosted reader can perform governed, structured or multi-stage retrieval while C4 uses a single BM25 pass; the 9.75-point increment 5.5 Judge stability and audit checks therefore reflects the complete system, not just the file format or a deterministic claim checker. O4w and O5 share a Both judges see the same blind items; pooled success difwriter but not matched input distributions: the adapted writer fers modestly (hosted C3 96% versus 95.5%; offline O3 69% may handle structured package lines better than raw passages versus 67%), with no main pooled comparison changing dibecause of its training. Segmentation and context budgets rection or statistical conclusion. Pass/fail agreement is 98.5– also differ; the offline system uses trained adapters and an 99.2% by package for the hosted batches and 95.3–98.0% for embedding-based selector, and the study has no base-modeloffline batches including anchors; Cohen’s 𝜅 spans 0.86–0.95. on-PAF condition that isolates those adapters. Apple O4w A 60-answer hosted anchor per package was blindly regraded also received a longer context slot solely to avoid truncating with the offline batches, with 97–100% within-judge pass/fail an oversized raw-file prompt. C1 differs from C2 in web matching. This addresses grader drift across batches but does access and conversation batching as well as instructions. The not establish independent human validation: both judges and four packages and one answering provider limit cross-task the question writer are from the Claude model family. and cross-model generalization. We independently recomputed success and harm summaries from all eight graded CSV files (4,800 hosted and 7,200 offline responses), checking the three-repeat design, Security and governance are proposed properties, not conditions, scores and package-archive digests. The pub- validated outcomes. We checked digest consistency for lished summary rates reconcile to the underlying rows within the supplied PAF archives and manifest entries, not an inderounding. We did not independently rerun the hosted mod- pendent Ed25519 trust chain or the authority of the human els or validate the underlying source claims against external approver. Some study packages declare offline mode disabled, and the test harness does not establish a conforming offline primary documents. verifier. This study does not test tamper refusal, revocation, malicious package contents, prompt injection, audience-safe byte projection, escalation into approved revision, or formal 6 Threats to Validity and Scope execution attestation. A low −1 answer rate is evidence of behavior on these questions; it does not prove cryptographic Held-out evaluation. The four packages were created after correctness, soundness of arbitrary material claims, or suitthe reader implementations and offline adapters were fixed ability of financial advice. and were not used in earlier reader training or debugging. Each package was evaluated using 100 preselected questions with frozen gold answers, three fixed repetitions per condi- Evidence availability. The authors retain the frozen prototion, the hosted Lens instructions applicable to the study, and cols, question sets and golds, individual answers, both judges’ blinded grading by two judges. Hosted and offline answers blind ratings and explanations, results tables, scripts and PAF were graded in separate blind batches, with hosted anchor re- packages in a private research bundle. These files are not sponses repeated in the offline batch to check grading stability. publicly released. This limits independent replication of both The included results were generated with the study conditions the experimental findings and the provenance assertions. described in §4; the held-out claim concerns the package content and fixed model configurations, not independence of 7 Interpretation: what travels, and package authorship or judging.

what fails to travel Question construction and sampling. The golds were compiled from the PAFs and their supporting evidence by a Claude question writer. The result therefore measures accurate access to a package-defined body of knowledge, not independent truth. The record-only sampling intentionally selects data omitted from the visible report and can favor a package-aware reader; a strong retriever over identical raw sources is the essential control. The fixed 35% decline quota can inflate the success of conservative readers relative to typical question traffic, while majority-of-three scoring can mask intermittent wrong answers. The same underlying case facts are repeated across three runs and should not be treated as independent question units.

7.1

The gain is not merely a stricter prompt

The comparison ladder distinguishes instruction-following from access to compiled knowledge. On the hosted system, the shift from ordinary chat (C1) to Lens instructions without the added records (C2) improves out-of-scope declines from 17% to 99%, while record success remains approximately unchanged or declines slightly (60% to 56%). This is consistent with a prompt that narrows unauthorized assertions without supplying the additional information the recipient needs. Adding ordinary retrieval (C4) takes record success to 77%; the bound package (C3) reaches 92%. Because the corpus is largely the same, this last comparison is evidence 6

Beyond the Document: Rhodes & Kang (2026)

that how the knowledge is organized and made reachable matters, but the experiment does not uniquely attribute the difference to typed facts rather than the hosted runtime’s retrieval algorithm. On the offline model the picture is more extreme. O2 produces 20% overall success and 56% responses rated −1, including 469 wrong answers and 207 unauthorized answers. O4w lowers the −1 rate to 6.2% largely by refusing questions, but resolves only 5% of answerable questions; O5 resolves 28%, and O3 54%. These results support evaluating answer coverage separately from refusal policy. They suggest that learned selection and training-compatible representations matter to the integrated reader; the test does not measure their independent causal contributions.

The distinction is especially important in regulated use: integrity, provenance, authorization, answer grounding, and substantive correctness are different assertions and require different tests.

7.4 Next tests suggested by the held-out failure

The Apple case indicates a concrete next step: a held-out experiment that tests selectors over repeated numeric observations with explicit keys for issuer, metric, filing, fiscal period, start and end dates, units and amendments. Such a selector should retrieve the correct row or visibly return no confident match, with a held-out test set used only after the new algorithm is fixed. Separately, PAF verification needs mutation tests (tampered payloads, mismatched signatures, revoked issuers, invalid approvals) and a viewer-conformance 7.2 Human-authored and AI-authored docu- suite covering offline policy and audience projection. Those ments studies can validate whether the format’s broader trust claim survives adversarial and independent implementations. They The experiment deliberately avoids limiting PAF to the native should not be read back into the present evaluation as already outputs of the Prime agents. The DFEOX case compiles re- proven. search around a source profile that is not the engagement’s own memo; the Apple and RCA cases compile content prepared in a general-purpose assistant, with ABC acting as 8 Related Work packaging and issuance service. The visible work product is therefore not required to have been authored by the package’s Document conversation and retrieval. Retrievaleventual consumer, and a recipient does not need the original augmented generation (RAG) conditions a model on source agent environment to ask a question. The study does not show passages [Lewis et al., 2020]. The strongest baseline here is that a source page was edited in place, nor does it establish explicitly a prompted reader with BM25 over the package’s all possible document types. It shows the narrower, conse- raw source files. The measured contribution beyond that quential property that separate original content and compiled baseline must be interpreted as the joint effect of compiled context can be delivered as one usable artifact. context, retrieval architecture and runtime behavior rather This is relevant to organizations that already have work- than evidence of a universally superior retriever. Atomic flows producing PDF reports, research notes, audit records factuality measurement [Min et al., 2023] and approaches or incident summaries. A transition to interactive, bounded that check numeric claims against a structured evidence work products need not start by replacing every authoring set [Solatorio, 2025] complement, but do not substitute for, system. It could start with an issuing process that attaches delivery of a persistent compiled artifact. compiled context and declared authority to what existing processes already produce. Whether that lowers costs or im- Integrity, provenance and authority. C2PA content creproves service quality requires deployment studies beyond dentials and W3C Verifiable Credentials bind statements and this question-answering benchmark. identifiers to signed artifacts [Coalition for Content Prove-

7.3

nance and Authenticity, 2025, World Wide Web Consortium, 2025]; in-toto and SLSA establish software build provenance [Torres-Arias et al., 2019, Open Source Security Foundation, 2023]. Those mechanisms provide important building blocks but do not alone define the context available to a later answering agent. The PAF architecture combines content, evidence references, release metadata and declared interaction policy in one delivery unit. A signature authenticates the signed statement, not the correctness of all underlying facts. Prompt injection against retrieved evidence remains a threat [Greshake et al., 2023].

A low observed harm rate is not a semantic proof

The −1 response measure combines two different behaviors: materially wrong answers on answerable questions and unauthorized answers on questions that require refusal. We therefore separate these counts, rather than describing their sum as a factual-error or real-world harm rate. The ratings do not audit the runtime’s complete call trace, prove that every retrieved claim derives from a permissible source, or check who was entitled to see each packaged byte. Likewise, a verifier can establish that a signed manifest identifies certain Agent execution and compiled memory. Proof of Execubytes without establishing that the numbers in them are true. tion [Rhodes and Kang, 2026b] frames signed runtime evi7

Beyond the Document: Rhodes & Kang (2026)

dence for governed actions, while compiled memory [Rhodes and Kang, 2026a] treats verified experience as reusable context for agents. The present paper concerns a different boundary: transferring useful knowledge with the work product so that the recipient’s reader need not reconstruct the original producer. Policy-bound answer receipts [Le Ray, 2025] and approval-integrity mechanisms [Alpay and Alpay, 2026] are complementary. Neither independent execution replay nor a cross-organization trust registry is evaluated in this study.

shows why that distinction matters: when the offline reader could not identify the correct XBRL record, it mostly declined rather than inventing a figure, sacrificing coverage. Portability of the artifact is necessary but not sufficient; retrieval and governance need independent conformance. A document records what was decided. A PAF package can carry the knowledge behind that decision into the next interaction.

Data and code availability. The study records and implementation materials are retained privately by the authors and are not publicly released. The manuscript reports aggregate 9 Toward an Interoperable Format outcomes and the protocol details needed to interpret the results but does not make the research bundle available for The four-case test supports the value of a package carry- independent replication. ing context across two producer paths and two readers, but all implementations and the institutional signing path share ABC-controlled format components. A generally interop- References erable format would require a public schema, canonical se- Faruk Alpay and Taylan Alpay. Approval integrity and recovery in llm rialization, well-defined provenance semantics, trust roots answer publication, 2026. arXiv:2609.15576. and revocation rules, explicit offline permissions, audienceCoalition for Content Provenance and Authenticity. Content crespecific evidence projection, and independent verifier and dentials: C2pa technical specification, version 2.2. https: runtime conformance tests. In particular, conformity cannot //spec.c2pa.org/specifications/specifications/2. 2/specs/C2PA_Specification.html, May 2025. be inferred from a reader’s ability to parse and answer questions about a package. Trusted issuance must separate who Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Comsigned, what was approved, which evidence was checked, and promising real-world llm-integrated applications with indirect prompt what the consuming runtime actually enforced. injection. In Proceedings of the 16th ACM Workshop on Artificial IntelliThe offline Apple failure identifies a concrete semanticgence and Security (AISec), 2023. arXiv:2302.12173. interoperability requirement that goes beyond valid bytes: Le Ray. Policy-governed rag: Research design study, 2025. record selectors must preserve period, filing, unit, instrument Jean-Marie arXiv:2510.19877 [cs.CR]. and qualifier identity. Better labeling, typed lookup or an explicit structured query interface may help, but these were Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen-tau not evaluated in this run. The next experiment should repeat Yih, Tim Rocktaschel, Sebastian Riedel, and Douwe Kiela. Retrievalthe exact question set under independent implementations augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Processing Systems 33, 2020. arXiv:2005.11401. and separately test cryptographic and policy conformance, without tuning on the test packages. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih,

10

Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023. arXiv:2305.14251.

Conclusion

Open Source Security Foundation. Supply-chain levels for software artifacts (slsa) specification, version 1.0. https://slsa.dev/spec/v1.0/, 2023.

A finished document need not be the final container for professional knowledge. The clean evaluation of four newly produced packages demonstrates that compiled context can travel with a deliverable and remain useful to a hosted governed reader and to a much smaller offline model. The hosted system resolves 96% of the fixed question set (94% on answerable questions), compared with 86% (79% answerable) when the same model receives raw-source retrieval. The local 0.8B reader resolves 69% overall (54% answerable), compared with 36% overall (5% answerable) for its trained writer over raw passages. These are results for the studied systems and question distribution, not guarantees for other tasks. The ambition is not to make every question answerable. It is to preserve the knowledge and authority boundaries behind completed work so that readers can request explanations from an accountable body of evidence and recognize when new work or a new approval is required. The held-out run also

James Rhodes and George Kang. Compiled memory: Not more information, but more precise instructions for language agents, 2026a. arXiv:2603.15666 [cs.AI]. James Rhodes and George Kang. Proof of execution: Runtime verification for governed ai agent actions, 2026b. arXiv:2607.05397 [cs.CR]. Aivin V. Solatorio. Proof-carrying numbers (pcn): A protocol for trustworthy numeric answers from llms via claim verification, 2025. arXiv:2509.06902. Santiago Torres-Arias, Hammad Afzali, Trishank Karthik Kuppusamy, Reza Curtmola, and Justin Cappos. in-toto: Providing farm-to-table guarantees for bits and bytes. In 28th USENIX Security Symposium, pages 1393– 1410, 2019. World Wide Web Consortium. Verifiable credentials data model v2.0. https://www.w3.org/TR/vc-data-model-2.0/, 2025. W3C Recommendation, May 2025.

8

Related documents

• Lexplorer: Navigating the Complexity of Legal Document Landscapes#1123731
• The Last Mile of Coding Agents for Long-Document Tasks: An Empirical Study on Clinical Trial Protocol Digitization#1344010
• Drafting#1191078
• Building a Collaborative Digital Collection: A Necessary Evolution in Libraries#1168962
• APEX-Harm: Evaluating Agent Safety in Shared Professional Workspaces#1298119