# Save tokens

> Send the last few turns plus StateJar's memory_context as the system message, not the whole history. How to do it in Python and Node, how to count the tokens it saves, and the measured result, which is negative on short chats.

## The short answer

- **The pattern.** Keep your chat as you do now. For each model call, send StateJar's `memory_context` as the system message and only the last N messages, instead of the whole history. StateJar holds the facts, so older turns no longer have to travel with every request.
- **It does not save on a short chat.** The memory block costs 110 to 287 tokens in the demos below, and a short history is cheaper than that. On the 17-turn demo it sent more than the full history at every turn; on the 40-turn demo it saved 3.7% in total with N=1 and lost with N=3 and N=5, though with N=3 it came out ahead from turn 26 on.
- **A history grows with every turn; the memory block grows only with the facts stored, and with a large state it holds just the fields the question needs.** That is where the saving comes from, and why it shows up late. Measure your own chat: the call in the code below returns both numbers.
- **What is counted.** Input tokens, with tiktoken `cl100k_base`. Not counted: your own system prompt, the model's reply, or the few tokens a provider's chat format adds per message. It estimates input tokens; it is not an invoice.

## The code

Python, with the `statejar` package from the [Python SDK page](/docs/python-sdk.html):

```python
import os
import httpx
from statejar import StateJar

sj = StateJar(session_tag="user-42")

# Your app keeps the chat in whatever message format your model provider uses.
history = [
    {"role": "user", "content": "I'm Meera from Pune and I prefer email."},
    {"role": "assistant", "content": "Nice to meet you, Meera. I'll keep to email."},
    {"role": "user", "content": "My budget is ₹3,500."},
    {"role": "assistant", "content": "Noted: a budget of ₹3,500."},
    {"role": "user", "content": "I'm travelling to Goa in December."},
    {"role": "assistant", "content": "Goa in December, got it."},
]
for m in history:
    if m["role"] == "user":
        sj.ingest(m["content"])                 # StateJar keeps the facts, not the transcript

question = "What is my budget?"
history.append({"role": "user", "content": question})

N = 6                                           # how many of the latest messages to still send
memory = sj.recall(question, audit=True)        # only the fields this question needs

# What goes to YOUR model: StateJar's context as the system message + the last N messages,
# instead of the whole history:
messages = [{"role": "system", "content": memory["memory_context"]}, *history[-N:]]
# reply = client.chat.completions.create(model="...", messages=messages)   # your provider here

# Count it: input tokens (tiktoken cl100k_base) of the full history vs what you send now.
base = os.environ.get("STATEJAR_BASE_URL", "https://api.statejar.com/api/v1").rstrip("/")
body = {"messages": history, "memory_context": memory["memory_context"], "window": N}
t = httpx.post(f"{base}/memory/token-compare", json=body, headers={"X-API-Key": sj.api_key}).json()
print(f"full history: {t['full_history_tokens']} tokens")
print(f"last {N} + StateJar: {t['tokens_with_statejar']} tokens ({t['recent_tokens']} recent + {t['context_tokens']} memory)")
print(f"saved: {t['saved_pct']}%   (negative while the chat is short)")
```

Node, with the `statejar` package from the [Node SDK page](/docs/node-sdk.html):

```js
import { StateJarClient } from "statejar";

const baseUrl = process.env.STATEJAR_BASE_URL ?? "https://api.statejar.com/api/v1";
const sj = new StateJarClient({ baseUrl });   // STATEJAR_API_KEY is read automatically in Node
const tag = "user-42";

// Your app keeps the chat in whatever message format your model provider uses.
const history = [
  { role: "user", content: "I'm Meera from Pune and I prefer email." },
  { role: "assistant", content: "Nice to meet you, Meera. I'll keep to email." },
  { role: "user", content: "My budget is ₹3,500." },
  { role: "assistant", content: "Noted: a budget of ₹3,500." },
  { role: "user", content: "I'm travelling to Goa in December." },
  { role: "assistant", content: "Goa in December, got it." },
];
for (const m of history) {
  if (m.role === "user") await sj.ingest({ sessionTag: tag, text: m.content }); // facts, not the transcript
}

const question = "What is my budget?";
history.push({ role: "user", content: question });

const N = 6;                                   // how many of the latest messages to still send
const memory = await sj.recall({ sessionTag: tag, query: question, audit: true });

// What goes to YOUR model: StateJar's context as the system message + the last N messages,
// instead of the whole history:
const messages = [{ role: "system", content: memory.memory_context }, ...history.slice(-N)];
// const reply = await yourModel(messages);    // your provider here

// Count it: input tokens (tiktoken cl100k_base) of the full history vs what you send now.
const res = await fetch(`${baseUrl}/memory/token-compare`, {
  method: "POST",
  headers: { "X-API-Key": process.env.STATEJAR_API_KEY, "Content-Type": "application/json" },
  body: JSON.stringify({ messages: history, memory_context: memory.memory_context, window: N }),
});
const t = await res.json();
console.log(`full history: ${t.full_history_tokens} tokens`);
console.log(`last ${N} + StateJar: ${t.tokens_with_statejar} tokens (${t.recent_tokens} recent + ${t.context_tokens} memory)`);
console.log(`saved: ${t.saved_pct}%   (negative while the chat is short)`);
```

Both run against a real account and print three lines: the full history in tokens, what you now send (recent turns plus memory), and the saving. `POST /memory/token-compare` does the counting on the server with the same tokenizer as the benchmark, reads and writes no memory, calls no model and stores nothing you send it. The [Playground](/playground) shows the same numbers live, turn by turn, as you chat.

Choose N from your own chats. A larger N keeps more of the conversation's wording but spends more tokens.

## Measured on the demos

Run on 31 October 2026 by `backend/benchmarks/window_demo.py` (the same pipeline, demo conversations and tokenizer as `backend/benchmarks/benchmark.py`; the committed output is `window_results.json` and `window_results.md` beside it). No model is called, so the conversations hold the user's turns only: each turn is rendered as the line `user: ` followed by its text, with no assistant replies and no system prompt on either side. "Last N + StateJar" is the memory context for that turn plus the last N user turns, the current one included. Savings are not clamped: a negative figure means StateJar sent more.

Total input tokens over the whole conversation (every turn's request added up):

| Conversation | Full history | Last 1 + StateJar | Last 3 + StateJar | Last 5 + StateJar |
|---|---:|---:|---:|---:|
| 17-turn demo, total input tokens | 2,169 | 3,749 (-72.8%) | 4,165 (-92.0%) | 4,541 (-109.4%) |
| 40-turn demo, total input tokens | 9,874 | 9,506 (3.7%) | 10,401 (-5.3%) | 11,247 (-13.9%) |

The 17-turn demo is a loss for every N, and the 40-turn mixed demo is a small loss for N=3 and N=5 over the whole conversation. Per turn, at N=3, the 17-turn demo:

| Turn | Full history | Memory context | Last 3 + StateJar | Saved |
|---:|---:|---:|---:|---:|
| 1 | 19 | 112 | 131 | -589.5% |
| 5 | 78 | 180 | 223 | -185.9% |
| 9 | 128 | 221 | 253 | -97.7% |
| 13 | 183 | 240 | 286 | -56.3% |
| 17 | 226 | 254 | 287 | -27.0% |

Turn 17: the full history is 226 tokens and the memory context alone is 254, so the history has not yet grown past the block that replaces it. At N=3 the 40-turn demo:

| Turn | Full history | Memory context | Last 3 + StateJar | Saved |
|---:|---:|---:|---:|---:|
| 1 | 19 | 112 | 131 | -589.5% |
| 10 | 138 | 215 | 246 | -78.3% |
| 20 | 237 | 248 | 277 | -16.9% |
| 26 | 308 | 269 | 307 | 0.3% |
| 30 | 345 | 276 | 305 | 11.6% |
| 34 | 390 | 128 | 162 | 58.5% |
| 40 | 467 | 287 | 324 | 30.6% |

At N=3 the per-turn saving is positive from turn 26 of the 40-turn demo to its end (25 at N=1, 29 at N=5), reaching 30.6% at turn 40. The memory context shrinks at turns 34 to 36 because those turns ask about one field, and StateJar sends only the fields a question needs once the state is large.

What this does not show: how the saving behaves with assistant replies, which these runs leave out, or with your own prompts; for that, count your own chat. These conversations are terse, form-like logs (12 to 13 tokens a turn on average), a short-message extreme. A percentage means nothing without its baseline: here it is the full history of the same turns.

## Questions

**Why is the saving negative at first?** The memory block is sent every turn and costs more than the first few messages it replaces. A saving needs a history longer than the block plus your N turns.

**Does this lose context?** The last N turns still go to the model word for word. Older turns reach it only as the facts StateJar stored from them, so something said once and never stored as a fact is gone from the request. Raise N if your chats lean on exact wording.

**Do I still need `audit=True`?** It writes the disclosure to your [audit log](/audit), so you can show later what the model was sent. It does not change what is sent.

**Where do the numbers in the Playground come from?** From the same call, with the chat you see on screen, including the assistant's replies.