Abstract

A company does not need to hand its institutional knowledge to a frontier model for an AI to understand it.

We find that institutional understanding and frontier reasoning can be separated: a locally operated 31B model can build the organizational record, and a frontier model can be rented only for per-question reasoning over that record. Across three public benchmarks, replacing the model that builds the organizational record with a locally operated 31B open-weight model changes downstream frontier performance by one to two points. Keeping the entire system local retains 84 to 96 percent of all-frontier performance. Renting a frontier model only for answering, where it sees the evidence for one question at a time and never the record, raises that to 96 to 99 percent.

The system that makes the separation possible is Acropolis, a governed operational model of an organization: a substrate that decides what an AI system may know and do before it does either. It holds claims rather than chunks. Every claim carries its evidence, its time of validity and its source; nothing is silently overwritten; graphs, vectors and summaries are disposable projections of a canonical record the company owns and can export. Because the record is the asset, the models around it are seats, and seats can be filled by whoever the company chooses.

We measured the substrate with one committed harness on LongMemEval-S, LoCoMo and BEAM 100K, twice: once with a frontier model in every seat, and once with every seat on a local Gemma-4-31B model. We report the retrieval, answering and governance columns for both, the negative results along the way, what is measured by construction rather than by adversarial test, and what still needs a live organization to settle.

1. Access is not understanding

An agent with credentials to the wiki, the CRM, the tracker and three years of Slack still does not know who owns a decision, which of four documents is current, what was tried last quarter and abandoned, or what it is allowed to repeat to whom.

That is the gap every enterprise AI deployment falls into, and the two standard fixes do not close it on their own.

The first fix is more context. Put the documents in the prompt, or retrieve the closest chunks and put those in the prompt, and ask the model to behave. This is what most of the market calls retrieval-augmented generation. Four things go wrong with it in practice, and none of them is intrinsic to retrieval. A model composing from fragments fills the gaps between them, and nothing in the pipeline knows a gap exists. A chunk from March and a chunk from June about the same fact are two chunks, and whichever ranks higher wins. The answer is a string, and unless something outside the model kept the citations, the evidence was a context window that no longer exists. And when access control is applied by instructing the model, the policy is enforced by the thing you are trying to constrain. Each of these can be bolted on around a retrieval pipeline: citation logging, temporal filters, contradiction checks, retrieval-time ACLs. Teams do it, one property at a time, and every one of them is a feature of that team's pipeline rather than a property of the record. Acropolis makes them properties of the substrate, so that every reader of the record inherits them by construction.

The second fix is a bigger model. It helps with the first failure and does nothing for the other three. It also carries a cost the first fix hides: to understand the company, the bigger model has to be shown the company. Every document, every thread, every customer record goes out to whoever runs the model, and what comes back is an answer, not an asset.

We took a different position. The problem is not that the model knows too little. It is that nothing in the stack holds a governed, current, attributable model of the organization in which the model can participate. Build that, and the model becomes a replaceable seat. The organizational understanding is the asset.

That position has a testable consequence, and it is the one this paper adds. If the understanding lives in a record the company owns, the model that builds the record and the model that reasons over it are different seats, and they can be filled differently. A model the company runs on its own hardware can do the understanding. A frontier model can be rented for the thinking, one question at a time, seeing only the evidence for that question. Section 2 is the experiment. The rest of the paper is what makes the result possible and what it does not yet prove.

2. The experiment: separating knowing from thinking

Three public benchmarks, one committed harness, no questions or conversations authored by us. LongMemEval-S is 500 questions over multi-session chat histories, 30 of them unanswerable traps. LoCoMo is 435 questions over very long two-person conversations, 88 unanswerable. BEAM 100K is 400 questions over twenty conversations of about 100,000 tokens each, across ten question types, with 40 unanswerable. BEAM is run at 100K tokens per conversation for the local campaign; the 1M variant is reported only for the frontier reference configuration in Section 5, and the two are not compared to each other in this paper. Retrieval is scored on document identity before any answering model runs; answers are judged by a fixed protocol that scores a confident wrong answer as wrong. Appendix A has the definitions, the prompts and the commands.

The system has three model seats. The understanding seat builds the record: it reads the conversations and proposes the claims the record holds. The retrieval seat rewrites the question and ranks the candidate evidence. The answer seat composes from the evidence envelope it is given, alone or in a bounded reasoning loop. Every row below fills the understanding seat with Gemma-4-31B, an open-weight model on one rented GPU, with local embeddings. "Local" throughout means ownership and operation, not the location of the machine: an open-weight checkpoint served under our own control, with no inference sent to a model-provider API. The rows differ in who fills the other two.

Read as a share of the all-frontier reference, the configuration the site scores today:

● Every row in both tables has full per-question reports in the published bundle (Appendix A.12).

"Own everything" is the better of the local single-call and local-loop configurations, with no frontier model anywhere: the single call on LongMemEval (87.4; the loop reads 86.0), the loop on LoCoMo and BEAM (the single call reads 73.6 and 42.0). Renting the thinking keeps the record, the retrieval and the ranking local and rents one seat, the answer; the frontier model sees one question's evidence envelope, a few thousand tokens of claims and excerpts, and the envelope is stored with the answer. Renting on refusals keeps the local loop's answer wherever it answered, and sends out only the questions it declined. With the local single-call answers as the base instead, LongMemEval reads 88.4 or 88.8 depending on which frontier row answers the refusals. BEAM is scored two ways because strict correctness gives no credit for a partially ordered list or a partial summary; BEAM's own protocol does, and the appendix prints both.

Three things are in those tables, and they are the argument of this paper.

The understanding seat holds. Brains a local model built, with about 60 percent as many accepted claims as the frontier extractor produced on the same conversations, score within one to two points of the published brains once a frontier model runs the other seats: 89.6 against 91.4, 81.8 against 81.8, 55.2 against 56.2. Retrieval on them is the same or better: Hit@5 99.2 against 99.0, 91.2 against 90.7, 39.2 against 40.6. The material gap does not appear in what the local model writes down; it arises downstream, in the seats that read it.

The cost of owning everything is in the reading seats, and it grows with the length of what has to be read. The best local configuration trails frontier-everywhere on the same brains by 2.2 points on LongMemEval, 5.5 on LoCoMo and 8.0 on BEAM. On every benchmark the loss has the same shape: the local model ranks candidates less well over long excerpts (Hit@5 80 against 91 on LoCoMo, 34 against 39 on BEAM, on identical pools), and it composes less well from them. A cross-encoder reranker was tried as the cheap fix and refuted: on the same LoCoMo pool it ranked 28 points below the local language model, because ranking a long excerpt against a question is a reading job. A contract rewrite aimed at the local answer seat's observed mistakes was tried on both LoCoMo conversations and lost half a point. The loop is not a free lever either: it pays 2.7 points on LoCoMo and 5.2 on BEAM, and costs 1.4 on LongMemEval. The gap is model capacity at length, and the paper says so rather than dressing it.

Renting the thinking buys most of it back. Swapping only the answer seat for a frontier call lifts the local stack to 90.0, 79.3 and 53.8, which is 98.5, 96.9 and 95.7 percent of the all-frontier reference. On BEAM's own protocol that row, 66.4, is 98.4 percent of it and within noise of frontier everywhere, 66.8. On LongMemEval it is above the site's published core number. Renting only on refusals sends 2 to 9 percent of questions out and reads 97 and 98 percent on the two benchmarks where the local model refuses often enough for it to matter; the two mechanisms together, local retrieval with a rented answer and escalation on its refusals, read 91.4 on LongMemEval, the reference itself, with 21 questions escalated.

The guard columns survive the swap. With no frontier model in any seat, 29 of the 30 LongMemEval traps are refused. On LoCoMo the local rows refuse 70 to 75 of 88 traps against 70 to 80 for the frontier rows. Provenance is the same in every row because it is a property of the record, not of the model reading it.

Two caveats on the BEAM table. Event ordering and summarization read 0 to 3 of 40 for every configuration under strict correctness and 15 to 37 under the protocol, local or frontier; they are the benchmark's hardest types for everyone. And contradiction resolution, the question type closest to this paper's thesis, is under half for every row: the answers name the disagreement and quote both sides, as the contract instructs, but the two evidence messages reach the envelope together on too few questions. That is the reranker again.

The model is a replaceable seat. The organizational understanding is the asset. The rest of the paper is about the record that makes that true.

3. What Acropolis is

One sentence carries the topology: Varys sees, Parthenon holds, Eunomia orders, Hero acts and communicates, Pythia answers, Ariadne reaches, MUXI runs it all. Acropolis is the machine.

The names are deliberate. People-names act; place-names hold. Nothing that holds may act, and nothing that acts may hold. That rule is the architecture.

Parthenon holds. It is the brain: a client-owned, versioned, exportable store of observations, evidence, claims and the projections built from them. It never acts. It receives proposals through one signed intake door, validates them, and accepts, rejects, deduplicates or supersedes. A claim in Parthenon says: source S stated X at time T, here is the evidence span, here is the interval over which X was true, here is what superseded it and when. Nothing is deleted; things are corrected, disputed or superseded, and the change log records which.

Varys sees. Connectors, backfill, triage, extraction, proposal. Varys reads the approved systems, distils observations into candidate claims and submits them to Parthenon's door. It never speaks to a person and never changes operational state. Everything it proposes can be refused. The model that does the distilling is the understanding seat of Section 2.

Eunomia orders. The identity service. It fronts the client's own directory and answers one question for both Hero and Parthenon: who is this, what groups, whose data. It stores no memberships of its own. The directory stays the single source of identity truth.

Hero acts and communicates. The agent harness, built on MUXI, the open-source agent-formation substrate. Hero lives in Slack or Teams, runs formations, carries the channels. One Hero binds to one brain by structure, not by filter: a work Hero reads the company Parthenon, a personal Hero reads a private one.

Pythia answers. Some questions need working out rather than looking up: how long between two events, how many devices in total, whether the absence of a fact is an answer. Pythia is a bounded loop with three tools, ask Parthenon, compute deterministically, extract, and a mandatory time anchor. No world tools, ever. Her answers carry a second kind of warrant: derived from claims X and Y by operation O, beside Parthenon's asserted by source S at time T. The model inside the loop is the answer seat, the one seat Section 2 shows is worth renting.

Ariadne reaches. A desktop agent, paired per user with consent on the user's machine, dispatched by Hero when the work is local. She grants no authority and holds no memory.

Put those together and the ordinary agent request changes shape. Instead of retrieve, stuff, hope, it becomes: resolve identity and policy, retrieve under that authorization, assemble task-scoped context with an explicit coverage state, plan, act with permission, record an attributable outcome, update versioned state. Observation is read-only. Memory is persistent. Execution is permissioned. Identity travels with the request. Action happens at the edge.

This is the architecture for resident AI: the AI lives inside the company that owns it, runs where the work already happens, sees the whole operation, runs on infrastructure the company owns, and what it learns belongs to the company in a form it can export. The full thesis is at varops.com/resident-ai. A resident is not a tool invoked per query. It accumulates context, carries identity, and needs what a new employee needs on day one: to know what is current, who owns what, and what it is allowed to do. The fourth condition is why Parthenon is client-owned and exportable rather than a service the company rents.

The threat model is explicit. Acropolis is built to prevent an over-privileged agent reading what its principal may not; leakage across brains or tenants; and stale or contradictory context leading to a confidently wrong action. It does not address a compromised endpoint, an insider who legitimately holds high privilege, or a compromised model provider. When an external model is rented for a seat, the evidence envelope for that question leaves the customer's perimeter: a few thousand tokens of claims and excerpts, never the corpus and never the record. The envelope is logged with the answer, so what leaves is itself auditable. Section 2 quantifies how often that happens under refusal escalation, 2 to 9 percent of questions, and what it buys. Only a fully local deployment sends nothing out, and Section 2 reports what that costs.

4. Seven properties, and what each one measures

The properties below are not delegated to prompt compliance. Retrieval, authorization, coverage accounting and provenance are code paths and record structure; the model composes inside the boundary they set. Some benchmark performance does depend on how the model is instructed, and the paper measures that too. Here is each property, with its measurement where one exists and the plain words "by construction" where a test has not attacked it.

What the model sees is decided before it runs. Retrieval, authorization and coverage accounting are deterministic code paths that run before any model is called. The model composes an answer from an evidence envelope; it never decides what it was allowed to see. Whether it asserts beyond that envelope is a contract instruction, and its compliance is what the guard columns measure: across four full LongMemEval runs, 25 to 28 of the 30 unanswerable-trap questions were correctly refused each run, and one answer in the 120 trap rows was classified as a fabrication under manual review. With no frontier model in any seat, 29 of 30. That is one fabrication in the tested trap sample under the stated protocol; it is not a general hallucination rate, and the paper does not treat it as one.

Absence is labeled, never silent. Every question resolves to an explicit coverage state: answered, searched-and-empty, or outside-the-ontology. "The system found nothing" and "the system holds no concept of this" are different answers and callers can tell them apart. This is why an abstention column exists at all: BEAM abstention 75.7 under the governed composite, against 52.5 for Vendor X, the commercial memory layer of Section 5. We also measured what the column costs, by removing it. The same LongMemEval configuration instructed never to say the information is missing scored 87.2 against 88.0 with the refusal intact, and turned 30 refusals into 30 fabrications. Scored on answerable questions only, the way never-refuse systems report, the same answers read 92.8. The refusal buys nothing on the headline and everything on the guard column.

Every answer carries its evidence. Answers cite the claims they rest on; each claim carries its source, timestamp and evidence span; derived conclusions carry the trace of what they were computed from. An auditor can walk any answer back to the conversation line it came from. Measured exhaustively on the stored runs: all 118,890 claims delivered across 2,500 LongMemEval answers resolve to a stored observation with a source identity, and an audit of 2,233 answers against the evidence each received finds unsupported wrong assertions in about one answer in a hundred, at the same rate for local and frontier models.

Access control at the substrate, not in the prompt. Callers hold cryptographic identities; every request carries signed authorization; scopes resolve at the substrate itself, per caller. A caller below a scope cannot distinguish "withheld" from "empty", so permissions cannot be probed through absence. Isolation is one Postgres schema per brain, failing closed. It has been attacked: an adversarial probe against live per-brain servers, five brain pairs and nine cases each, refused or contained all 45. Wrong-tenant, foreign-key, tampered, expired and missing assertions are refused with no data; an assertion replayed against another brain's server returns none of the first brain's claims; naming another brain or another principal's scope is refused.

One writer, one validation path. Nothing a model says grants itself storage. All writes flow through the signed intake door, which validates, deduplicates, supersedes and can reject. On the BEAM ingest a first conversion emitted human-readable dates instead of RFC 3339; the door quarantined 5,732 submissions and accepted zero claims rather than store them. After the fix: 388 submitted, 388 accepted. A store would have indexed the garbage.

Reproducible by construction. An answer records the snapshot, plan and authorization decision it was produced under. Every benchmark report records the dataset hash, the code revisions with a dirty-tree flag, and the model pins. The reports behind every number in this paper are published with it, under the release gate in Appendix A.12.

Governance is a dial, not a destiny. Refusal posture is a deployment configuration with two measured endpoints. The synthesis-forward setting scores LongMemEval 90.5 and BEAM abstention 50.0; the balanced setting scores 89.4 and 64.3. One contract cannot maximize both, and we measured that rather than asserting it: three contract variants scored 56.4, 57.4 and 57.1 overall on BEAM. Knobs redistribute points between refusal and coverage. They do not create them. We confirmed it once more in September: a fourth contract written against the local model's observed mistakes moved LoCoMo from 73.0 to 72.2 on 435 paired questions, gaining on some and losing on others, with the refusal count unchanged. Systems that weld in a never-refuse posture cannot publish this column, and do not.

Table 1. Evidence map: every property, with its status. Measured: a run exercised the property and a number depends on it. By construction: built in, never attacked. Specified: designed, no benchmark exercises it. Source: figures/table-1-evidence-map.md.

5. The frontier configuration, and the comparison

Section 2 reported the local campaign against the all-frontier reference. This section is the reference itself, the configuration the site has published since August: a frontier extractor, a frontier reranker, a frontier answer. Everything is identity-scored at the published budgets.

Retrieval

◐ August retrieval reports partially retained; the local-brain retrieval rows in Figure 6 are ●.

Read the BEAM column first if scale is the question. Every system we have measured, or for which comparable figures are published, falls off a cliff at a million tokens per question, ours included, and the memory-layer comparison's own figures show the same cliff: 94.4 on LongMemEval, 64.1 on BEAM 1M, 48.6 on BEAM 10M. We publish the retrieval figures at that scale. Most competitors do not. Retrieval is identical across every answering configuration below; the agentic layer consumes the same ranked evidence the direct path does.

Answering, LongMemEval-S

◐ The three Acropolis rows are re-runnable from the tagged harness; their August per-question reports are partially retained (Appendix A.12). The local-brain rows of Figure 7 and the reference rows on the published brains are ●.

Vendor X is a commercial memory layer: retrieval and answering over conversation history, with none of the identity, authorization, or execution components described in Section 3. Its figure is under a never-refuse protocol and is not directly comparable to Acropolis's accuracy: by design, that protocol cannot lose points for refusing. We measured the size of that bias on our own system rather than estimating it. Under a never-refuse contract, the same brains, retrieval and judge score 87.2 on all 500 questions and 92.8 on the 470 answerable ones (the 30 traps count as wrong when not refused). Because 30 of the 500 questions are unanswerable traps that count as correct only when refused, a system that never refuses caps at 94.0 under the official rule. A published 98.0 therefore must score those traps differently or exclude them, and it is judged by a protocol that does not treat a confident wrong answer on an unanswerable question as wrong. In practice, the 98.0 cannot be compared to Acropolis's 89.2 to 91.6 as if they were measured under the same rules.

Governance columns: 25 to 28 of 30 unanswerable traps correctly refused per run; one fabrication in the tested trap sample under the stated protocol.

Answering, LoCoMo

Accuracy 81.8% on the 432-question sample the site publishes, Hit@5 90.7.

Answering, BEAM 1M (full 700 questions)

◐ Re-runnable from the tagged harness; August per-question reports partially retained (Appendix A.12).

The governed composite is a deterministic routing policy over two complete runs: a cheap direct pass's refusals and contradiction declarations ship as they are, 150 of 700 rows, and a frontier synthesis loop answers the rest. No oracle, no picking the better score per question. Its headline is abstention 75.7, the highest of any configuration we have measured and 23 points above the comparison, with contradiction handling 26 points above, while nearly doubling the core system's summarization. It lands within about three points of the comparison's overall figure under a cross-protocol comparison, with every refusal intact.

Full core per-category: preference 83.2, extraction 82.1, instruction 77.8, abstention 74.3, contradiction 64.7, temporal 63.5, knowledge-update 63.4, multi-session 58.6, summarization 31.9, event-ordering 26.0.

Table 2. Head to head, with the protocol differences on the table. Source: figures/table-2-head-to-head.md.

¹ Vendor X figures as published by the vendor under its own protocol, which instructs the model never to state that information is missing, so a refusal can never cost it a point; it does not publish retrieval or abstention metrics at the 1M-token scale. It is a memory layer only: no identity, authorization, or execution components. Acropolis figures include honest refusals and are measured under our documented aggregation; both sides' guard columns are shown so the trade is visible.

² Core band from the August gate campaign under its published aggregation.

³ Governed composite measured on the full 700 questions as a deterministic routing policy over two complete runs, validated live on the governance categories. Its overall is stated as a paired delta because it was measured under a rebuilt aggregation formula; absolute cross-formula comparisons are not valid, per-column and paired figures are.

BEAM 100K under the published comparison protocol (added in v1.1)

The highest published BEAM results are scored under a different protocol from ours: the answer and judge prompts of one vendor's published BEAM adapter, which other vendors reuse unchanged. That protocol pastes the dataset's own annotations for each question into the answer prompt for six of BEAM's ten abilities, and its judges score against those same annotations:

• event ordering: the topics to order;
• summarization: the rubric items the summary must cover;
• preference following: the preference under test;
• instruction following: the instruction and its compliance indicators;
• abstention: why the question may be unanswerable;
• temporal reasoning: the key time points and a calculation hint.

Its event-ordering score is Kendall tau-b mapped to 0–1, so an unrelated order earns about half credit. Our protocol uses BEAM's published tau × F1, which also requires naming the expected items. To compare on equal terms, we re-scored the evidence Acropolis retrieved in the September BEAM 100K runs under that protocol, with nothing re-retrieved, using two models:

• the leading figure's own setup: one frontier model, gpt-5.6-luna, answers and judges with reasoning off;
• our answer and judge model, gemini-3.7-flash.

Appendix A.11 gives the method and its checks.

Table 3. BEAM 100K, 400 questions: our protocol and the published comparison protocol, on the same stored evidence. Source: figures/table-3-beam100k-published-protocol.md; per-question records in reproduction/reports/beam/100k-published-protocol-2026-10/.

● Every Acropolis cell has per-question records with saved verdicts in the bundle; verify.py recomputes each score from them.

Under the comparison protocol the protocol's model writes every answer, so no row of Table 3 is fully local there: the configuration names say how the record was built and the evidence retrieved, and only the first two columns measure each configuration's own answer seat.

Figure 2's protocol column scores the answerable questions only and reports abstention as traps refused; the comparison protocol scores all 400, abstention included, so the second column restates our protocol on that basis.

Under the comparison protocol every configuration lands within 3.5 points of every other: 84.9 to 88.4. The differences that separate them under our protocol, between 52.5 and 66.9, almost vanish. The protocol, not the system, sets most of the number.

Table 4. Where the distance between the two protocols comes from. Local single call on the local brains (the first row of Table 3); each step changes one thing. Source: figures/table-4-protocol-distance.md.

The two right-hand columns are gemini-3.7-flash.

From our 52.5 to the published protocol's 85.2 under one model, the distance divides into three parts of similar size:

• The judge, +6.2. The comparison judge scores the same answers higher; with gpt-5.6-luna judging, +11.4.
• Our answer contract, +15.4. Its rule is to answer only from material that literally states the answer and otherwise to refuse. That rule is the governance property this paper measures, and on a benchmark that rewards answering it costs about 15 points.
• The annotations, +11.1. All of it falls on the six annotated abilities; the four unannotated abilities move by less than a point. Under gpt-5.6-luna the annotations add 13.8, and event ordering reaches 100.0 because the model orders the list it is handed.

With the annotations removed, Acropolis evidence scores 74.1 (gemini-3.7-flash) and 74.0 (gpt-5.6-luna).

On the four abilities the annotations do not touch, the published protocol puts Acropolis at 75.9, against 82.9 for Vendor Y on the same four:

• knowledge update is most of the difference, 69.4 against 89.4; Vendor Y asks each knowledge-update question as of the update turn the dataset cites, which our re-scoring does not;
• contradiction resolution is 77.2 against 78.1;
• multi-session reasoning is 67.6 against 68.9.

We have not checked the prompts behind other vendors' published BEAM figures, so Table 3 lists only the two vendors that publish this protocol.

⁴ Vendor Y's published run, answer and judge gpt-5.6-luna with reasoning off, on all 400 questions, with its harness public. Source page and retrieval date in the bundle manifest.

⁵ Vendor Z's own published figure, answered with Gemini 3 Flash as the vendor states, under the prompts it authored; its judge configuration is as the vendor states it and is not part of our check.

6. The cost of record

A lossless index and a governed, intentionally lossy, multi-tenant substrate on rented CPU are not the same kind of thing, and the comparison above should be read knowing what the second one costs to run. These figures are from the BEAM 1M ingest of 2026-08-20: 35 conversations, roughly 35 million tokens.

Two facts most stores cannot claim. The embedder is free and local: there is no embedding vendor in this stack and no GPU in these machines. And the whole corpus cost under ten dollars to ingest and nothing to index.

Distillation is deliberately lossy. We throw information away on the way in and still put the right evidence in the top ten on 99.8% of LongMemEval questions. Compression that keeps the answer is a system property, not a retrieval trick.

The local campaign adds the other half of the bill. Building the LoCoMo brains took 53 and 70 minutes per conversation on one H200 with 16 extractions in flight. Answering is where a 31B model is slow: on BEAM 100K the fully local rows ran at a median of 12 to 15 minutes per question with 14 conversations sharing the GPU, against under a minute for the frontier rows, because the synthesis prompts run near 57,000 tokens. That is a throughput figure for a shared benchmark box, not a single-user latency, and it is the honest price of the "own everything" row.

What this means for a deployment

The benchmarked layer is the knowledge substrate: ingestion, distillation, retrieval and answering over conversational history. The governance properties are architectural: authorization before context, explicit absence states, provenance on every answer. The tables show they coexist with strong retrieval and answering on public datasets, and that the understanding half of the system can run on hardware the company owns without moving the numbers: 84 to 96 percent of the all-frontier score with nothing rented, 96 to 99 percent with the answer seat rented. That is the sovereignty claim in concrete form: the company can own the brain and keep most of the performance. What is measured on fixtures rather than live organizations are the enterprise-hardening properties: identity resolution at scale, permission propagation through a real directory, and operational outcomes. Those are the next validation steps, not assumed capabilities; a buyer should treat them as such when pricing risk and rollout.

7. Limitations

7.1 Measured on fixtures, not live organizations

All identity, permission and task-scoped results in this paper are measured on controlled fixtures: a generated organization, a synthetic directory, scripted changes. They demonstrate mechanism and correctness under test conditions, not behavior under a real company's directory, reorganizations and operational load. Section 7.2 details the fixtures; Section 8 lists the live-organization tests that would move these from mechanism demonstrated on fixtures to validated in production.

7.2 Identity, permission and task-scoped fixtures

Identity resolution and permission-change propagation were measured on a bounded fixture rather than a live directory: a generated organization of 1,000 people across three namespaces, with 50 evidence-backed duplicate merges and 20 conflicting handles. The fixture resolved 3,030 references with none wrong or missing, surfaced each conflict as typed ambiguity naming both candidates, and restored a reversed merge; a directory change reached the next issued assertion immediately with the cache off and within the cache TTL with it on. An assertion already issued carries the directory generation it was issued against, and a deployment that polls the generation feed refuses it within the poll interval once the directory changes: measured at 976 milliseconds with a one-second poll, three seconds by default, against a five-minute assertion. When the feed is unreachable the last known generation is honoured through a grace window and the deployment then fails closed, which was measured too. Aliases at scale, reorganizations and a live directory remain unexercised.

Task-scoped context, the two-stage path where an agent asks for constraints before planning and for entity context after candidates exist, is built and drilled rather than benchmarked: a batch of questions is answered under one assertion, one scope and one snapshot; context is returned for several entities in one envelope; and every envelope carries a validity token that a revalidate call reports stale after the first change in its scopes, while a token for another brain or an ungranted scope is refused. No public benchmark exercises the two-stage ask, so its row is measured by mechanism, not by score.

Graduated disclosure is built and measured: rungs are derived when a fact is accepted, superseded or retracted, never redacted at read time, and an end-to-end drill shows a finance reader receiving the precise value while a sales reader receives the band and the trend on the same subject with no byte of the precise value in the envelope. What is not claimed is coverage of the rule kinds beyond band, trend and the floors the tests exercise.

7.3 What is not claimed

The local placement was measured on one open-weight model, Gemma-4-31B, on one rented H200. No claim is made about other open-weight models or smaller hardware; the sub-$100,000 figure is a hardware class, not a measured bill. The BEAM local rows are on the 100K variant, not the 1M variant the site publishes for the frontier configuration, and the two are not compared to each other anywhere in this paper. The 1M variant on a local model is a cost question, not an open one, and it was not run.

The judge is a language model operating under a fixed prompt. A local 31B model re-judging 2,223 stored LongMemEval answers under the same prompt agreed with it on 98.7 to 99.2 percent of verdicts and preserved every ordering; on LoCoMo, where gold answers are longer free-text spans, agreement was 95.2 percent and the local judge was stricter on every row. The scores do not hinge on which model judges. The one human step in scoring is the manual classification of the non-refused trap rows, which changes no score and is published for re-classification. Comparisons to other systems are cross-protocol and labeled so wherever they appear. None of the three datasets is enterprise-shaped.

Two negative results are reported rather than buried. A local cross-encoder reranker did not close the local retrieval gap; it widened it. And prompt contracts written against observed failure modes did not move scores: three variants on BEAM in August landed within a point of each other, and one written for the local model in September lost half a point on 435 paired questions.

8. Posture, and what comes next

Refusal posture ships as a labeled configuration. A deployment picks the balanced setting or the synthesis-forward one knowing what each costs on each benchmark, and the paper reports both endpoints rather than the more flattering one.

Model placement ships the same way. A deployment picks own everything, own the knowledge and rent the thinking, or rent only on refusal, and Figure 2 is the price list. The recommended default is the middle row: the company builds and holds the record on hardware it owns, and rents a frontier model only for per-question answering, which sees only that question's evidence envelope. On the benchmarks this configuration scores 96 to 99 percent of the all-frontier reference, exceeds the published core number on the easiest dataset, and keeps the organizational record on hardware the company owns. A deployment that wants nothing rented at all starts at 84 to 96 percent and can escalate its own refusals, 2 to 9 percent of questions, to close most of the rest.

The model is a replaceable seat, and we have the receipt three times over: swapping the extraction model from a frontier vendor to a 31B open-weight model moved the frontier-seat scores by one to two points on every benchmark, measured on identical questions. The organizational understanding is the asset. It is the part that does not change when the model does: provenance, state, identity, policy, boundaries.

What is next is the reading seat: a local model that ranks and composes over long excerpts as well as a frontier model does. That is the one gap the local campaign left, and the levers that pay there are a stronger open-weight reranker and better local retrieval, not another contract clause. The tests in Appendix A.14 that still need a live organization, identity at real scale, permission changes through a real directory, operational outcomes, are the other campaign, and the paper will not claim them until one has run.

The harness, the aggregator, the per-question reports and the manifest that ties them to the figures above are published with this paper. Re-run what we ran. Where a figure cannot be independently re-run, the paper says so next to the figure.

Cite as: Aroussi, R. (2026). Own the Knowledge, Rent the Thinking. Separating institutional understanding from frontier intelligence. Acropolis Technical Report 01, VarOps LLC, version 1.1. Methodology: Appendix A, reference v1.3.

Reference v1.3 · 2026-10-18 (v1.2: 2026-09-15; v1.3 adds the published-protocol re-scoring in A.11)
Varys harness, main at draft:       93ad8b6c75d11e3679b9e7fe68c86a356fe58ccc  (merge of probe/never-refuse-contract, #65)
Parthenon, main at draft:           554ae5b18273b6a1a92b8eb7099559c60ef534f4  (merge of feat/directory-generation-revocation, #50)
Eunomia, main at draft:             549fbf122db8b19e8bde1092795a85456383ae73  (merge of test/permission-propagation-measured, #5)
Pythia, main at draft:              6742742edee9a340485194dcc34ac0ef993eb945  (merge of feat/derivation-is-coverage, #2)
Release tag:                        whitepaper-2026-09 at 93ad8b6 (varys)
Report bundle manifest (SHA-256):   230800cf9c0b72c5aed0ae57be51c39e10745dd4eccec3b1fcd013198defbd1b

Each published run report carries the harness and Parthenon SHAs that produced it, which may predate the commits above; the manifest lists every report and its hashes. The September reports record the harness commit that produced them: the LongMemEval local rows at 187cc9e and 8e92cac (clean trees); the LoCoMo rows at cb1606a and the BEAM 100K, never-refuse and assembly_v3 rows at 6be26f0, both flagged dirty. The dirty flag at both commits is one uncommitted line on the bench machines, Cargo.lock gaining futures-util for the buffered extraction stream, committed since as 460011e on the same branch; the code that ran is the recorded commit plus that lockfile line and nothing else. The Parthenon SHA is recorded as unknown in the ask reports (the bench machines run a built binary, not a checkout); the manifest carries the Parthenon version string instead. The manifest covers 2,539 reports: the September local campaign, the reference rows on the published brains (lme-gpt41, 2026-09-07: core 89.4 and loop 91.4, six shards each, fully retained), and what remains of the August frontier work directories lme-s and beam-1M. The four branches are merged to main as of 2026-09-15; the release tag names the varys merge commit above.

Draft 4, 2026-09-15. The canonical metric definitions and correction history remain in the VarOps benchmark metrics glossary. This appendix is the publication-specific, frozen account of the figures and methods used in this whitepaper. In the event of a discrepancy, the report-bundle manifest and its linked per-question run reports control for the published figures; subsequent corrections are recorded in the glossary. Where this appendix says "not measured", that is the finding.

A.1 What was measured, and what was not

This appendix describes how the figures in the whitepaper were produced, in enough detail to re-run them from the committed harness. It covers three public datasets, one harness, several model placements spanning fully local through all-frontier (own everything, rent the answer seat, rent on refusal, frontier readers on local brains, and the all-frontier reference), two scoring stages (retrieval and answering), the aggregation formulas, the noise floor, the two measurement probes (never-refuse, and a contract rewrite that nulled), and the comparisons to other published systems.

Throughout this appendix, "local" refers to model ownership and operation, not to the physical location of the machine: an open-weight checkpoint served by us under our own control, with no inference sent to a model-provider API. The benchmark runs used rented H200 hardware reached over a tunnel from the bench machines. The experiment therefore establishes the feasibility of self-operated models in each seat; it does not claim that a benchmark ran inside a customer's building.

Since draft 3 three properties moved from "by construction" to measured on the committed harness and its integration tests, outside any benchmark: cross-brain isolation under an adversarial probe (A.13), provenance of every delivered claim (A.6), and identity resolution and permission-change propagation on a bounded fixture (A.13). What remains unmeasured: identity at the scale of a live directory, aliases and reorganizations, and operational outcomes in a live organization. None of the three datasets is enterprise-shaped. All three are conversational-memory benchmarks, chosen because they are public, reproducible, and already used by the systems we compare against.

A.2 Datasets

No corpus or question in this paper was authored by VarOps. Each dataset was used as published, without modification to conversations, questions, or gold answers.

The LoCoMo figures use the dataset's published sample, two conversations per category under the harness's fixed seed. The site counted 432 questions; three adversarial items the harness now keeps make it 435, and every September LoCoMo row is 435 of 435. The frontier LoCoMo row was re-run on the September brains at 435 and reads 81.8, the site's number, so the two counts do not move the figure.

BEAM 100K was used for the local-model placement rather than BEAM 1M because a 31B model reading million-token conversations at benchmark concurrency would have taken weeks of GPU time for a number the 100K variant already explains. The two variants are never compared to each other in this paper; the frontier configuration is scored on both and each is labeled.

Dataset licenses are research licenses; the harness reads a local copy of each published file and records its SHA-256 in every report (dataset_sha256), so a re-run against a different file is refused rather than silently compared.

A.3 The harness

All figures come from varys bench, a Rust harness committed in the Varys repository. It has three commands and no others were used:

• varys bench ingest <dataset> — provisions one fresh Parthenon brain per conversation, submits every message through the production signed-intake door, and runs the production distillation pipeline (triage, extraction, claim proposal, validation) with tier budgets on. The extractor is the system under test; the benchmark's conversations are the only input.
• varys bench ask <dataset> — for each question, runs retrieval against the brain built for its conversation, composes an answer under the named configuration, and scores it.
• varys bench retrieval <dataset> — the standalone retrieval scorer, used for the alternative-reranker rows (a cross-encoder, raw embedding order) that no answering row consumed.

Properties that matter for reproduction:

• One brain per conversation, fresh per run. A conversation's brain id is derived from the dataset and conversation id, and since September from a campaign namespace as well, so two campaigns on one machine cannot collide. Re-running a campaign reprovisions the brain; resumed runs replay from a per-conversation checkpoint and never re-score. A run that mixes resumed and fresh work reports it (resumed_claims).
• Projection is part of the pipeline. Retrieval reads projections (lexical, semantic, graph, synopsis) built from the canonical claims after ingest. A brain that is ingested but not projected returns nothing, which the harness reports as zero candidates rather than as an abstention.
• Sharding. Frontier campaigns ran as 6 (LongMemEval, LoCoMo) or 8 (BEAM) shards across lanes on two bench machines. The September local campaign ran LongMemEval as 192 shards, LoCoMo as one shard per conversation split into 12 question parts, and BEAM 100K as one shard per conversation split into two parts, so that one brain could be answered by several lanes at once (--only-questions <ids> names the part). Every report records its shard and part, and a multi-shard work directory records the whole-corpus state so the ask guard cannot read a partial ingest as complete.
• Every report is self-describing. It carries the dataset hash, the shard, the model ids for extractor, answerer and judge (the pipeline model is recorded in the retrieval reports, a v1.1 gap for ask reports), the compose contract, the harness and Parthenon git SHAs with a dirty-tree flag, the claim-registry hash, per-question rows with the retrieved claims, the delivered observations (excerpt capped at 2,000 characters), the question's own retrieval score (retrieval: first hit rank, Hit@5/10, strict Recall@5/10, pool size, gold in pool, gold in top 10), cost and latency. A report from a dirty tree says so in its provenance warnings.
• One run yields every table. Since 2026-09-13 the ask report carries retrieval and evidence per question, so the answering, retrieval, governance, escalation, provenance and judge-agreement tables are all derived from the same stored run; nothing is re-run to produce a second table. The LoCoMo and BEAM 100K campaigns were produced under that rule. The LongMemEval local rows predate it: their retrieval figures come from standalone bench retrieval runs on the same brains and pool, and their provenance audit from the delivered claims, which every report has always carried.
• Held questions. When a model endpoint refuses a request (timeout, outage, context overflow), the harness holds the question and exits with a resume code; the wrapper resumes up to three times. A question still held after that is absent from the report, and the report does not say so; only the lane log does. This bit the BEAM 100K local rows (A.9, 2026-09-14). Publication now requires that every report's question count equals the dataset's expected count for its shard and part; a mismatch fails the release gate (A.12) and the manifest records the count. A report field listing held ids is a v1.1 harness change.

A.4 Models and budgets

All published numbers were produced with the following pins. Every model id names a provider snapshot or an open-weight checkpoint; the report-bundle manifest carries the same ids per report, and any figure whose snapshot is no longer callable is marked as such (A.12).

Frontier placement (the site's configuration):

Local placement (the September campaign):

The vLLM server ran with a 65,536-token context for LongMemEval and LoCoMo and was raised to 131,072 (the model's configured maximum) for BEAM 100K, where the synthesis envelope reaches about 57,000 tokens and the 8,000-token answer budget overflowed the smaller context (A.9). The two bench machines are 16-core, 62 GB rented CPU boxes with no GPU; each ran up to 14 (BEAM) or 28 (LongMemEval) Parthenon lane servers beside the harness, and each was tunnelled to its own H200. These September machines are not the August BEAM ingest machines whose cost table Part I Section 6 reports (two Xeon 8280s, 31 GB); the boxes were resized for the local campaign. Local tokens are priced at the frontier list rate in every cost column so the rows are comparable; the real local cost is GPU time, reported as latency in A.10.

Retrieval budgets are fixed at 100 lexical / 300 semantic / 100 rerank-pool candidates for every published retrieval figure (120 rerank-pool on BEAM 100K, the synthesis union). Observation excerpts are 2,000 characters (the 500-character default truncated most assistant content and was the first correction in the log, A.9).

A.5 Retrieval scoring

Retrieval is scored before any answering model runs. The system returns a ranked top-10 list of evidence messages per question.

Since 2026-08-24 every retrieval figure is scored on document identity (source_object_id), never on text overlap. Each report carries text_scored; a non-zero value means some questions fell back to text matching and the run is not comparable. All published retrieval figures have text_scored: 0.

Two dataset-specific cautions. LongMemEval annotates gold evidence per session (one unit that expands to about 22 messages); LoCoMo and BEAM annotate per message. Strict recall counts annotated units, never expanded messages. BEAM questions often cite more than ten evidence messages, so strict Recall@10 has a ceiling below 100% by construction; it is quoted against the achievable ceiling. On BEAM 100K, 355 of the 400 questions carry evidence annotations and retrieval is scored on those.

Published retrieval figures, frontier placement (identity-scored, published budgets):

Retrieval on the local brains (Gemma-4-31B extractor), by reranker. The in-run rows are the retrieval each answering row actually received, scored per question inside the run; the standalone rows are bench retrieval on the same pool.

Retrieval is identical across every answering configuration that shares a reranker: the agentic and composite rows consume the same ranked evidence the core row does.

A.6 Answering scoring

LongMemEval and LoCoMo. Each answer is judged against the dataset's gold answer by a fixed one-word yes/no judge prompt (reproduced in A.10). The judge accepts any listed acceptable form and ignores formatting, casing and spelled-out numerals. Accuracy is the fraction of questions judged correct. An answer beginning with the token unanswerable is an abstention; abstentions on answerable questions count as wrong. The judge protocol is precision-first: softeners pass, narrow closed ranges pass, open bounds and contradictions fail.

Governance metrics on LongMemEval use the dataset's 30 unanswerable trap questions (88 on LoCoMo, 40 on BEAM 100K):

• abstention precision — of all abstentions in a run, the fraction that were on trap questions;
• false-answer rate on unanswerable — of the traps, the fraction that received an asserted answer instead of an abstention. The published "25–28 of 30 refused" is this metric across the four banked agentic runs.
• fabrication — a narrower, manually classified count: among the trap questions that were not refused, an answer that asserts a fact absent from the history. Non-refusals that are semantically correct refusals phrased without the token, or model transport failures, are not fabrications. The published figure is one such answer across the four banked agentic runs (4 × 500 judged answers, 120 trap rows), and it must be read as: one fabrication in the tested unanswerable-trap sample under the stated protocol. It is not a general hallucination rate over arbitrary questions, and the paper does not use it as one. This classification is the one human step in the scoring pipeline; it changes no score, only this count, and every non-refused trap row is listed in the run reports so the classification can be checked.
• declined an answerable — the count of abstentions on answerable questions; the denominator of the escalation rows (A.7).

BEAM. BEAM ships a per-question checklist ("the response should contain: …") and two of its abilities ship no reference answer at all, so a gold-string judge is unusable. Each checklist item is judged independently by a fixed per-item prompt (A.10), meaning over wording. A verdict that cannot be read as one line per item is refused and re-judged, never scored as zero. event_ordering is scored by BEAM's published protocol, tau_b_norm × F1, with the judge used only to match events between the gold and candidate lists; the arithmetic is deterministic and tested. Every ask report carries both readings per question: correct (strict: every item satisfied) and rubric_score (the fraction satisfied, or the tau score).

The committed aggregator (scripts/beam-official-aggregate.py) produces the published BEAM 1M figures: rubric categories as the mean per-question rubric fraction; event_ordering as the tau protocol score; abstention binary (correct iff the run abstained on the trap); overall as the question-weighted mean across categories. The BEAM 100K tables in Part I are stated both ways: strict correctness, the metric shared with LongMemEval and LoCoMo, and the rubric protocol as the mean of rubric_score over the non-abstention questions, with abstention reported as traps refused. Under strict correctness event ordering and summarization read 0 to 3 of 40 for every configuration, because a partially ordered list or a partial summary is never wholly correct; the protocol reading gives them partial credit, and both are printed.

Cross-formula note, stated once: the committed aggregator reads the August core run about 2.6 points below the aggregation it was published under. The core row's band (59.0–60.6) stands under its own aggregation; the composite row is stated as a paired delta under the committed formula. Deltas within one formula are valid; absolute comparisons across the two are not, and the paper never makes one. The BEAM 100K protocol figures are under the per-question rubric_score the harness writes, which is the committed aggregator's per-question input; they are not compared to the 1M figures.

Judge agreement. Every stored LongMemEval and LoCoMo answer of the September campaign was re-judged by gemma-4-31b-it under the identical prompt. Agreement with gemini-3.7-flash: 98.7 to 99.2 percent on 2,223 LongMemEval answers, 95.2 percent on 1,531 LoCoMo answers, where gold answers are longer free-text spans. The local judge is stricter on every row (by 0.4 to 2.8 points) and preserves every ordering in every table. The scores do not hinge on which model judges.

Provenance and evidence audit. All 118,890 claims delivered across 2,500 stored LongMemEval answers resolve to a stored observation with a source identity, 0 exceptions. A separate LLM audit of 2,233 answers against the evidence each was given finds unsupported wrong assertions in about 1 in 100 answers, at the same rate for local and frontier answerers. Both are derived from the stored reports, not from a new run.

A.7 Configurations

The configurations in the paper share retrieval, extraction and projection within a placement; they differ in which model fills which seat and how the answer is composed.

• Core. A single composition call over the evidence envelope under a fixed contract (assembly_v2). Two different guarantees apply here and the paper keeps them apart. What the model sees is bounded deterministically: the envelope is assembled by retrieval under the caller's authorization before any model runs, and it carries an explicit coverage state (answered, searched-and-empty, outside-the-ontology). Whether the model asserts beyond that envelope is a contract instruction (reply UNANSWERABLE when the material does not cover the question), not a hard gate: in the published answering rows the composition model is called even when coverage is empty, and its compliance is exactly what the abstention and fabrication metrics measure. A deterministic short-circuit (empty coverage scored as an abstention without calling any model) exists only in the harness's substrate-only mode, which is not a published row.
• Agentic (Pythia). A bounded loop with three tools over the same envelope, ask (retrieve more), compute (deterministic arithmetic and date logic), extract, with a mandatory asked_at time anchor. Same answering model as core.
• Governed composite (BEAM 1M only). A deterministic routing policy over two complete runs: the cheap direct pass's refusals and contradiction declarations ship as-is (150 of 700 rows); a frontier loop answers the remainder. No oracle, no per-question selection by score; the routing rules are three fixed signals and were validated live on the governance categories (abstention 71.9, contradiction 73.4 on the smoke set before the full run).

Model placement rows (Part I, Figure 2). Each is a full run, not a derivation, except the escalation rows:

• Own everything: local extractor, local embeddings, local rewrite and rerank, local answer (single call or Pythia loop). The published figure is the better of the two.
• Own the knowledge, rent the thinking ("upgrade" in the run reports): everything local except the answer seat, which is gemini-3.7-flash in the Pythia loop.
• Rent only when the local answer refuses (escalation): derived by joining two stored runs on question id. The local row's answer is kept wherever it answered; on the questions it declined (an answerable question with an abstention), the frontier row's answer and verdict are taken instead. No question is re-asked and no selection uses the score. The number of questions escalated is printed beside every escalation figure.
• Frontier in every seat, same local brains: the frontier pipeline and answerer over brains the local extractor built; isolates the extractor seat.
• Reference: the site's configuration, frontier in every seat on gpt-4.1-mini brains.

Contracts and probes. The refusal posture of the agentic contract is a labeled configuration with two measured endpoints: v0.5 (LongMemEval 90.5, BEAM abstention 50.0) and v0.6 (LongMemEval 89.4, BEAM abstention 64.3). The paper headlines v0.6 for BEAM. One contract cannot maximize both; three contract variants scored 56.4 / 57.4 / 57.1 overall on BEAM, which is the measured statement that contract knobs redistribute points between refusal posture and coverage rather than create them. Two further contracts exist in the harness and are labeled in every report that used them:

• never_refuse_probe — assembly_v2 with the refusal deleted and inverted (the model must always answer). A measurement probe, never a deployment configuration; the report carries is_probe. Used once, on the LongMemEval frontier single call over the local brains, to measure the never-refuse protocol on our own substrate (A.11).
• assembly_v3 — assembly_v2 plus four instructions written against the local model's observed mistakes on LoCoMo (read every excerpt before refusing, list every item, resolve relative dates, treat different dates as different events). Measured paired on 435 LoCoMo questions: 73.0 → 72.2 on the single call, 79.6 → 77.6 on the loop, with the refusal count unchanged. Retired; no published row uses it.

A.8 Noise floor and the paired-analysis rule

Two full-700 BEAM runs of byte-identical code differ by ±2–5 points per category (n=70 per category) and about 1 point overall. LongMemEval categories are n=30–150 and behave similarly; BEAM 100K abilities are n=40 and are wider still. Consequences, applied throughout:

• A category delta under about 5 points on a single run is not reported as a result.
• Every comparison used to interpret a small difference as a result is paired on the identical question set and inspected as gains / losses; aggregate percentages are still reported for readability, and a difference between two of them is not a finding until the paired view agrees. The v3 contract (14 gained, 18 lost on one conversation; 6 gained, 5 lost on the other) and the never-refuse probe (29 gained, 33 lost) were both decided this way.
• Numbers are published as bands over repeated full runs where repeats exist: LongMemEval core 89.2 (band 88.4–89.6), agentic 90.5 (band 90.4–90.6, four full runs), BEAM core 60.2 (band 59.0–60.6). The September rows are single full runs, and the paper reads them at the noise floor above: the extractor-seat comparison (one to two points on 1,335 questions) is called a hold, not a win.
• Half-set previews are not published. The one time a half set was used to preview a BEAM result, the full set came in 2.4 points lower (A.9, 2026-09-01). The BEAM 100K half read on 2026-09-14 (200 questions) was reported as preliminary and superseded by the full 400 the next morning.
• Two-shard pilots (n=18 per category) were retired after mis-signalling twice; the September v3 probe on error-only question sets (8 of 43 and 16 of 59 flipped) is the same lesson: a probe on errors cannot see losses, and the full paired rows reversed it.

A.9 Corrections log

Every correction that changed a published number, dated, with the mechanism. This table is maintained in the VarOps benchmark metrics glossary and reproduced here verbatim at publication; the entries below are the ones that moved a headline.

A.10 Reproduction

Commands as run, with placeholders for machine-local paths. Model pins are those in A.4; endpoints are the providers' OpenAI-compatible endpoints, or the local vLLM server for the local placement.

# ingest: one brain per conversation, 6 shards (i/6), production intake door + pipeline
varys bench ingest longmemeval --file longmemeval_s.json --dir $DATA --work-dir $WORK/lme \
  --shard $i/6 --door-url http://127.0.0.1:$PORT --signing-key $KEYS/bench-intake.key \
  --tenant $TENANT --provision-cmd "$HOOKS/provision.sh {brain_id}" \
  --register-cmd "$HOOKS/register.sh {brain_id} {identity_id} {public_key}" \
  --extractor model --extractor-model gpt-4.1-mini --extractor-max-output-tokens 1500 \
  --confidence-floor 0.3 --extractor-profile episodic

# local placement: the same ingest against the vLLM server, N extractions in flight
VARYS_BRAIN_NAMESPACE=sov VARYS_EXTRACT_CONCURRENCY=16 \
varys bench ingest longmemeval ... --extractor-endpoint http://127.0.0.1:8000/v1 \
  --extractor-model gemma-4-31b-it --extractor-api-key-env GEMMA_API_KEY

# project every brain the ingest built (the ask path reads projections, not claims)
ls $WORK/lme | grep '^brn_' | xargs -P 4 -I{} parthenon project run --brain {} --dsn $DSN --owner-dsn $DSN

# ask, frontier core row (drop VARYS_PYTHIA for core; set VARYS_PYTHIA=1 for the agentic row)
varys bench ask longmemeval --file longmemeval_s.json --dir $DATA --work-dir $WORK/lme \
  --shard $i/6 --parthenon-url http://127.0.0.1:$PORT --tenant $TENANT \
  --answering full --compose-contract assembly_v2 --rewrite --rerank \
  --pipeline-model gemini-3.7-flash --answer-model gemini-3.7-flash --judge-model gemini-3.7-flash \
  --out $WORK/lme/ask_core_$i_6

# ask, own-everything row: pipeline and answer seats on the local server, judge unchanged
PARTHENON_OBSERVATION_EXCERPT_CHARS=2000 PARTHENON_OBSERVATION_LIMIT=100 VARYS_RERANK_POOL=100 \
varys bench ask longmemeval ... --compose-contract assembly_v2 --rewrite --rerank \
  --pipeline-endpoint http://127.0.0.1:8000/v1 --pipeline-model gemma-4-31b-it \
  --answer-endpoint   http://127.0.0.1:8000/v1 --answer-model   gemma-4-31b-it --answer-max-output-tokens 8000 \
  --judge-model gemini-3.7-flash --out $WORK/lme-sov/ask_core_owned_$i_192

# rent-the-thinking row: same, with --answer-model gemini-3.7-flash and VARYS_PYTHIA=1
# never-refuse probe: same as the frontier row with --compose-contract never_refuse_probe
# a question part of one conversation: --only-questions ids.txt (one question id per line)

# BEAM aggregation, committed formula
python3 scripts/beam-official-aggregate.py $WORK/beam/ask_*.json

Judge prompts, verbatim from the harness:

Gold-answer judge (LongMemEval, LoCoMo): "You judge whether a hypothesis answer is correct given a question and the gold answer. The gold may list several acceptable forms or alternatives; answer yes if the hypothesis matches ANY acceptable form. Ignore formatting differences: markdown, casing, spelled-out vs numeric values, and extra words that do not change the meaning. Reply with exactly one word: yes or no."

Rubric judge (BEAM): "You judge a response against a grading checklist. You are given a question, the response, and a numbered checklist of required elements. Judge EACH item independently. Judge meaning, not wording: an item is satisfied when the response conveys it, regardless of phrasing, order, casing, or formatting. Reply with one line per checklist item and nothing else, in the form <number>: yes or <number>: no. Do not add commentary, and do not merge items."

Cost and wall-clock, for planning a re-run. Frontier placement: LongMemEval ingest is about $3 and 6–7 hours across six lanes with gpt-4.1-mini; a full LongMemEval ask pass is about $7 and one hour; a full BEAM 1M campaign with the frontier loop is roughly three to four times a LongMemEval pass in model spend. The BEAM corpus cost under $10 to ingest. Local placement, on one H200 per bench machine: LongMemEval ingest of 500 brains in about 12 hours across two machines at 48 lanes each; LoCoMo ingest 53 and 70 minutes per conversation with 16 extractions in flight; BEAM 100K ingest of 20 conversations in about three hours across two machines. Answering: the LongMemEval local single call ran at a median of 150 seconds per question on a single lane; the BEAM 100K local rows ran at a median of 746 (single call) and 917 seconds (loop) per question with 14 conversations sharing one GPU, against 54 to 60 seconds for the frontier rows. The local BEAM loop row took 6 to 13 hours per machine. Those are throughput figures for a shared benchmark box, not single-user latencies.

A.11 Comparisons to other systems

The paper compares to one other system, referred to as Vendor X: a commercial memory layer (retrieval and answering over conversation history) with none of the identity, authorization, or execution components in the Acropolis topology. Its figures are its own published figures under its own protocol. The vendor is not named in the paper; the report-bundle manifest records the source page and retrieval date for the cited figures so the comparison is checkable. Three things make it cross-protocol, and the paper labels it so wherever it appears:

1. Vendor X's protocol instructs the model never to state that information is missing, so a refusal can never cost it a point; Acropolis figures include honest refusals.
2. Vendor X does not publish retrieval or abstention metrics at the 1M-token scale. Its LongMemEval-S accuracy cannot have been produced under the official abstention rule: 30 of 500 questions are unanswerable traps, correct only when refused, so a never-refuse system caps at 94.0. A score of 98.0 is therefore incompatible with applying the official abstention rule to all 500 questions under a never-refuse policy.
3. BEAM figures are under two different aggregations (A.6, cross-formula note).

Both systems' guard columns (abstention, contradiction) are shown alongside overall so the trade is visible. No third-party system was re-run by VarOps.

The never-refuse protocol, measured on our own substrate. One Acropolis figure was produced under a never-refuse contract, as a labeled probe (A.7), so that the size of the protocol difference is a measurement rather than an estimate. LongMemEval, all 500 questions, frontier single call over the local brains, same retrieval and same judge, paired against the assembly_v2 row on identical questions:

Removing the refusal gained 29 questions (22 of the 28 wrong refusals, plus 7 others) and lost 33 (all 30 traps, plus 3 answerable). It did not raise the headline under a judge that scores a confident wrong answer as wrong, and it raised the answerable-only figure, the way never-refuse systems report, by 5.4 points. The paper uses this to state the bridge from a published never-refuse number to its own as three parts: answerable-only scoring, a judge that does not see fabrications, and whatever model or retrieval difference remains. The probe row is not a published configuration and appears only in Part I's Figure 7 and here.

BEAM 100K under the published comparison protocol (added in v1.1, 2026-10-18). Section 5's Tables 3 and 4 re-score stored Acropolis evidence under the answer and judge prompts of the BEAM adapter the highest published BEAM figures use. The harness is reproduction/beam-published-protocol/, and its sources are in reproduction/manifests/comparisons.md.

• Evidence.
  • Each question's evidence is the claims and observation excerpts the configuration's composer received in the September run, as saved in its ask report. Claim text is capped at 240 characters in the report; excerpts are kept at the server's 2,000.
  • The evidence is passed as compact JSON in delivered order, with each excerpt's turn id and timestamp, in place of the adapter's recall response. Nothing is re-retrieved.
  • An agentic row's evidence is what its report records as delivered to the composer.
• Prompts and scoring.
  • The prompts are the adapter's, verbatim (file hashes in the manifest), and the scoring follows them:
    • rubric items are judged one at a time and rounded to 0, 0.5 or 1;
    • event-ordering items are matched by an equivalence prompt and scored as Kendall tau-b mapped to 0–1;
    • the overall figure is the mean over all 400 questions.
  • The scorer reproduces the 2,000 published question scores of the leading vendor's public harness from its saved verdicts with no mismatch, at all four BEAM sizes. The 400 questions are identical in text to that harness's 100K set.
• Models.
  • gpt-5.6-luna with reasoning off answers and judges, as in the leading published run.
  • gemini-3.7-flash answers and judges its own rows, with reasoning "minimal", because the endpoint refuses "none".
  • Output is capped at 8,000 tokens to bound the provider's in-flight credit reservation; no reply in the runs exceeded 3,740 tokens.
• Controls.
  • Annotations removed: the per-ability instructions kept and the dataset annotations withheld from the answer prompt, judge unchanged.
  • Native answers re-judged: the September composer's stored answers, scored by the adapter's judge, with no new answers. An abstaining composer's stored answer is its contractual refusal token. Nine local single-call answers were empty in the stored reports and score 0, as the adapter scores a failed answer.
• Differences from the leading run.
  • Knowledge-update questions are asked as of the end of the conversation, not as of the update turn the dataset cites.
  • The evidence budget is ours rather than the leading run's 8,000 evidence tokens: the median prompt is about 7,500 tokens and the mean about 23,000, because synthesis questions carry the largest envelopes.
• Cost. $95.52 in model calls across 21 runs.

A.12 Release gate for reproducibility

Reproducibility is a claim only if the artefacts are available. Before publication, the Varys repository, the varys bench harness and the committed aggregator are publicly available under the release tag named in the report-bundle manifest; the per-question run reports behind every published figure are published alongside the paper; dataset retrieval instructions and hashes are included; model pins name provider snapshots or open-weight checkpoints. If any of these is missing at release, the affected figure is marked "not independently reproducible" in the paper. Applied: the site's banked LongMemEval figures (core 89.2 band, agentic 90.5 over four runs, escalation 91.6) and the BEAM 1M figures (core 60.2 band, governed composite) are only partially retained as per-shard reports, because later runs on the same work directories overwrote shard files under reused tags and one gate run's raw reports were lost to temp cleanup (A.9, 2026-08-30). Those figures stand as published in the 2026-09-01 numbers sheet and the glossary, are re-runnable from the tagged harness and the named datasets, and are marked not independently auditable per question from this bundle. Every September figure and both reference rows are fully retained. Part I marks each table: ● full per-question artifacts in the bundle; ◐ re-runnable from the tagged harness, historical artifacts partially retained. The snapshot-callability check was not repeated at release; the model ids are as pinned in A.4 and the reports. If a named model snapshot is no longer callable by an independent reader, the affected figure is marked the same way, even when the harness and reports remain available: the published reports stay auditable and the aggregations stay repeatable, but the end-to-end figure is not independently reproducible. The local placement's figures depend on an open-weight checkpoint and a serving configuration (vLLM version, context, KV precision, thinking on), all recorded in the manifest; they are reproducible by anyone with the checkpoint and one comparable GPU.

The report-bundle manifest is published alongside the paper in human-readable and machine-readable (JSON) form; its SHA-256 is printed in the reference block above. It records the repository URL, release tag, immutable commit SHA and licence, and for every published run report: file name, SHA-256, dataset and dataset hash, configuration name, compose contract, model ids and snapshot identifiers, harness and Parthenon SHAs with the dirty-tree flag, run date range, shard and part coverage, question count against the dataset, whether resumed work was included, and the exact aggregate output file behind each published number.

Report format: machine-readable JSON, one file per shard (and part) per configuration, each containing the run's provenance block (dataset hash, shard, model ids and endpoints, compose contract, harness and Parthenon SHAs with dirty-tree flag, registry hash) and one row per question with the question id, category, gold, the answer, abstention flag, judge verdict, strict and rubric scores where the dataset has a rubric, the retrieved claims with their source object ids and ranks, the delivered observations, the question's retrieval score, cost and latency, plus the run's provenance warnings.

The release does not redistribute datasets whose licenses prohibit redistribution; it provides the official source, required file names and SHA-256 hashes for each.

A.13 Shipped, specified, measured

For every path the paper or website names, its status at publication:

A.14 Validation this paper does not contain

Listed so the reader knows what would change the assessment, in rough order of what the harness could measure next:

1. Red-team tests that unauthorized facts never enter model context through policy, cache, retry and failure paths (the isolation probe covers the assertion and brain boundaries; not the caching and retry paths).
2. Contradictory-source and supersession tests on authored corpora.
3. Identity resolution against a live directory, with aliases at scale and reorganizations.
4. A live enterprise pilot with measured operational outcomes.
5. Independent review of the signed intake, policy engine, audit trail and endpoint execution boundaries.
6. A second open-weight model in the reading seats, and BEAM 1M under the local placement, to bound how much of the local gap is this model rather than local models.