← Documentation index Architecture guide › scratchpad

Metnos

Observation scratchpad

What the module provides and what the runtime actually uses.

runtime/scratchpad.py implements temporary storage that can retain a complete observation while showing the model only a reference or slice. The API exists, but the current engine does not automatically offload large observations to it and does not expose scratchpad_read in its ordinary catalog. This distinction matters: an available module must not be presented as an active feature.

Current behaviour
Available API
Storage and lifetime
Synthetic representation
Sliced reads
What integration requires
Required verification
Security boundaries

1. Current behaviour

In the active engine, step results remain in the turn's execution history. A later step reuses them through from_step or a ${stepN.field} reference; the runtime resolves the actual value without asking the model to copy it. Final synthesis projects and limits the text fields shown to the model, while full data remain in the step result.

At the beginning of a turn, agent_runtime opens the scratchpad database and removes expired rows. On the current path, however, it does not call Scratchpad.put. The historical scratchpad_threshold parameter does not control engine execution either. It is therefore incorrect to say that every observation above 4 KB is automatically offloaded to the database.

2. Available API

Method or value	Role
`Scratchpad.open(path)`	Open or create the SQLite database and schema.
`put(turn_id, step_num, executor_name, observation, ttl_seconds)`	Store an observation and return a synthetic representation.
`get(id)`	Return the complete row for an identifier.
`read(id, mode, n, start, end)`	Return all content, its beginning, its end, or a selected interval.
`list_for_turn(turn_id)`	List metadata and summaries for a turn.
`gc(now)`	Delete rows whose expiry has passed.
`stats()`	Count stored rows and bytes.
`SCRATCHPAD_READ_TOOL`	Describe the possible builtin tool; the constant alone does not make it visible to the current planner.

3. Storage and lifetime

The default location is PATH_USER_DATA/scratchpad.db. Its table stores:

id · turn_id · step_num · executor_name · content_kind
content · size_bytes · summary · created_at · expires_at

put defaults to a one-hour lifetime. Deletion is not an independent background process: it happens only when a caller runs gc. An expired row may therefore remain on disk until the next cleanup.

4. Synthetic representation

For long text, the module stores the complete content and produces a summary from its beginning and end. For binary data, it stores the bytes and reports their size and a SHA-256 prefix. For a structured result, it looks for fields such as entries, matches, or results and reports count and schema without placing the items in the summary.

The synthetic representation can retain useful scalars such as counts, truncation state, size, message, and error. The ref_hint field explains how to reuse a step result or request a content slice.

5. Sliced reads

Mode	Result
`full`	Complete content.
`head`	First `n` characters or bytes; the tool declaration defaults to 2,000.
`tail`	Last `n` characters or bytes.
`range`	Interval from inclusive `start` to exclusive `end`.

Text is returned as UTF-8 with replacement for invalid bytes. Binary content is Base64-encoded. Metadata report full size, returned size, kind, and read mode.

6. What integration requires

A correct engine integration requires at least:

one decision point comparing observation size with a canonical setting;
a call to put before building model-visible context;
exposure of scratchpad_read only when the current user and turn have accessible rows;
an ownership check on every get, read, and list_for_turn call;
limits on full reads so a large row cannot simply re-enter the context it was meant to protect;
cleanup, observability, and resume tests after a suspended dialog.

7. Required verification

Automatic engine-to-database integration is not active. Enabling it requires dedicated tests for text, binary data, structured results, ranges, expiry, concurrency, and turn resume.

The most important test is multi-user isolation: an identifier obtained by one user must never read another user's row, even when passed directly to the builtin API.

8. Security boundaries

The current schema has no owner. It stores turn_id, but no owner_user_id, actor, or channel. get and read accept only an identifier and perform no authorisation.
An opaque identifier is not a credential. Difficulty of guessing it does not replace ownership checks.
The database is shared by the installation. It must not be described as per-user isolated until schema and callers enforce that separation.
The module does not encrypt content. At-rest protection depends on data-directory permissions and the hosting environment.
Expiry is logical. Data are removed only when gc runs.

For these reasons, the module can be studied and tested as a component, but must not be connected to a multi-user surface before ownership, authorisation, and isolation tests are added.