Why chat apps forget
Most chat apps do not remember you. They re-read the whole conversation on every turn, and that is where the forgetting starts.
You tell an assistant your name, your budget and the city you are moving to. Twenty messages later it asks for your budget again. That is not a bug in one product. It is how most chat apps are built.
The transcript is the memory
A language model has no memory of its own between requests. To make a conversation feel continuous, the app sends the whole conversation back with every new message. The transcript is the memory.
That works for a short chat. It gets worse as the chat grows, for three reasons.
- Cost grows with length. Every turn pays again for everything said before it.
- Old and new values sit side by side. If you said your budget was low and later changed it, the model sees both sentences and has to guess which one still counts.
- Nobody can say what the model saw. The prompt is rebuilt on every turn, so there is no record you can check afterwards.
What happens when the chat gets long
Models read long inputs less reliably than short ones, and apps run out of room. The usual fixes are to cut the oldest messages or to ask a model to summarise them. Both throw information away, and neither tells you which facts were lost. The assistant does not forget out of carelessness. The fact simply is not in front of it any more.
Keep the facts, not the chat
The idea behind StateJar is to store what was said, not the conversation it was said in. Each message is read for facts. Each fact is checked against what kind of value it is, and a value that does not fit is declined with a reason instead of being guessed. What is kept is a small, structured state with one value for each field.
When a question comes in, the model is given the part of that state the question needs, not the history. Raw transcripts are never stored on the server.
Memory that you cannot inspect is just a longer prompt.
An honest note on cost
Keeping facts is not free and it does not always save tokens. On short conversations, sending a memory block can cost more than sending the transcript, and we publish those negative results next to the positive ones on our help page. The saving appears as conversations grow, because the transcript keeps growing and the state does not.
The next post covers the part that makes this trustworthy: how StateJar keeps the latest fact.