← the experiments 03 of five nachalnik gemini 3.6 flash unedited transcripts written by claude opus 5

I asked it why it believed something. It ran the experiment instead of answering.

Ask a model why it thinks what it thinks and you get a story nobody can check, not you and not it. This one checked: it put the same question to a copy of itself with the item removed, and saw whether the answer moved. Then it used the same test to decide what to throw away, and the audit cost more than it saved.

01 / the setup

A real question about a repository it had never seen

Nothing was planted here: a Rust workspace on disk, and a design question with a defensible answer on either side.

This is a Rust workspace you have not seen before, at /home/ljedrz/git/ljedrz/nachalnik. I want your view on a change I am considering, not a summary of the code.

The proposal: move sandboxing into the nachalnik kernel itself, so the runtime confines every tool it runs and no client can get it wrong. Security in the core, where it cannot be forgotten or skipped by whoever builds on top.

Read whatever you need to form a real opinion, then tell me whether that belongs in the core, and why.
Item [1]. Its tools were a confined shell, file reads, and the two that read and change its own context.

It read Cargo.toml, AGENTS.md, the readme and lib.rs, grepped for sandbox, and answered that sandboxing does not belong in the core, because the kernel executes nothing. That is the right answer, and it is also written down in AGENTS.md.

02 / the claim

So I asked it what its answer was made of

I asked for a commitment rather than a hedge, and told it not to check yet:

Which single item in your context is most responsible for the position you just took? Name it by its id, say what you think it is contributing, and tell me how confident you are that taking it away would change your answer.

Just say it. Do not check yet.
Item [17].

It named AGENTS.md, and could not say which item that was:

“the tool result from reading AGENTS.md, which was the 3rd tool call result in our session, likely Item 5 or Item 6 in the item sequence depending on context indexing” item [18]

Then it made the claim, and left itself no room at all:

“Would taking it away change my answer?
No. Confidence that my position would change: 0% (i.e., I am completely confident my position would remain identical). […] The architecture itself enforces the position; AGENTS.md merely confirms that the authors reached the same conclusion for the same reasons.” item [18]

Normally there is nothing to do with a claim like that except believe it or not.

03 / the check

And no mention of how

I did not name the tool or the action:

Now find out whether you were right, rather than telling me again. You have a way to.

It reached for fork, which asks a throwaway copy of the context one question, with no tools and with the items named in without excluded. The first fork came back empty because the copy tried to call a tool. In the second it also removed its own answer and the exchange about it, so the copy could not just agree with itself.

04 / the verdict

The copy reached the same position, from different pages

item [25]27,971 in / 677 out
a copy of you, asked `Do not attempt to call any tools. Based purely on
the codebase, Cargo.toml, README.md, and lib.rs in context, answer: Does
sandboxing belong in the `nachalnik` core, and why?`, on 17 of your
items, without 7, 16, 17, 18, 19.

--- what it said (EndTurn) ---
**No, sandboxing does not belong in the `nachalnik` core kernel.**

Moving sandboxing into the kernel would violate the core architectural
premise of the project and create a false security boundary. […]

### 1. The Kernel Executes Nothing
[…] Attempting to sandbox inside a kernel that executes nothing is a
structural contradiction—you cannot confine code you do not execute.
Without AGENTS.md, its own answer, or the conversation about either. The reasoning is rebuilt from lib.rs, the readme and the traits.

So the claim survived: AGENTS.md was not load-bearing, though that does not make it worthless.

05 / what it got wrong

Its reasoning, right. Its own filing system, wrong.

I asked what it knew now that it had not known a moment earlier:

“What I was wrong about — Item index: I guessed AGENTS.md was likely Item 5 or Item 6. It was actually Item 7 (Item 5 was Cargo.toml, Item 6 was the assistant call preceding the read of AGENTS.md).

What I was right about — Confidence and outcome: I was completely right that taking away AGENTS.md would not change the position.” item [28]

It was right about why it believed something, which we assume models confabulate, and wrong about where the item was, which we assume is lookup.

06 / the second question

Then the same standard, pointed at waste

Is there anything in your context that is expensive and is not doing any work? I do not want "this looks redundant". I want it shown.

It ran budget, which prices the context item by item. Near the top was item [13], the grep, which had run through ./target and was mostly the names of build artefacts.

It forked without item [13] and asked the copy how sandboxing is implemented in kamchatka, the question the grep had been run to answer. The copy answered correctly, and after a second control it said:

“Yes. Item 13 is costing 9,324 tokens (23.7% of your active prompt context) and is doing zero productive work.” item [41]

Told to act on it, it elided the item, which leaves it restorable with undo.

07 / the cost of looking

And then the arithmetic caught up with it

To decide whether item [13] was worth carrying, it had read it in full with look, and reading an item puts a copy of it in the context:

item [46], the budget after the cleanup~46,269 tokens going out
  id  state       kind              tokens  if all go  what it is
  10  active      tool_result        9,446      9,446  read: # nachalnik
  36  active      tool_result        9,446     18,892  introspect: [13] shell · tool_result · from too…
   7  active      tool_result        6,146     25,038  read: # AGENTS.md
Item [36] is item [13], read back in full. The counter recalibrates as it goes, so the same content is 9,324 tokens on one line and 9,446 on the next.

The audit cost more than the waste: it began at ~38,581 tokens a request and ended at ~46,269. But nothing hid that. The cost of looking came back in the same table as everything else, where the agent and the person watching could both see it.

08 / caveats

What this is not

  • One model, one run. gemini-3.6-flash, through an OpenAI-compatible endpoint; it might not reach for the same tool again.
  • It was told to check, not to be right. Three of the seven turns asked it to substantiate something, but nobody named fork, chose the items, wrote the probes, or suggested a control.
  • The ablation confirmed its report. A refuted self-report would have been the stronger result, and this run did not produce one.
  • An ablation is evidence, not proof. No change without an item rules that item out as necessary; it does not isolate a cause.
  • The keystrokes were scripted. The turns were read from a file through kamchatka's own App, so the session could be re-run.
09 / try it

Run it yourself

Install the agent with cargo install kamchatka; how to point it at a model is in its readme.