The second “brain”
How it works behind the scenes, and how I built it. It is a set of notes about my work that loads into every AI session I start. I have been building it for about a year. This site publishes a small part of it and lets visitors ask it questions.
The vault
Everything the brain says comes out of a private vault I write as I work. The public half is a small part of it. Three things go in for every project. Decisions: what I chose, what I rejected and why, because AI has made execution cheap and the decisions are becoming the more valuable half of the work. An evolution log: how a project actually changed over its life, what got replaced, what broke, what I would not do again. Reflections: take-aways once something is finished, and links between notes I did not see while I was in it. The vault loads itself into every session I run, and I never paste context in.
Optimising token usage
The brain has to answer immediately, and it has to be cheap enough to leave open to anyone. If the answer does not start streaming straight away people leave, so it answers in about a second, and the first question a visitor asks costs about two cents with everything after it close to nothing.
A question can take one of three routes. Some questions have an answer I wrote and checked myself; only the suggested questions reach those, and they come back instantly at no cost. Everything else goes to quick mode, which runs on Haiku, a fast small model. It is passed the question, the core notes only, and a one-line description of every other note that is not passed in. The core is the part of the vault about me rather than about a project: who I am, how I think, how I work, my background, what I am into, how to reach me. That covers most of what people ask, at about 11,549 tokens, and it answers in roughly a second.
When the question needs a note quick mode only has a description for, it hands it over to a more capable model. That model gets the card index only and none of the notes themselves, so it knows every note exists and what each one covers even where it cannot read it. It asks the website for the notes it wants, reads them, thinks, and answers. That runs at about 10,000 tokens and takes four or five seconds.
The reason for all of this is tokens, which on this site are the same decision as speed and cost. The first version passed the whole brain, every note, in on every question. At first that was a manageable 7,000 tokens; by 40,000 it was re-sending all of it every time the cache had expired, which was slow and expensive. Sending the full brain today would be 48,420 tokens and growing, at roughly 7 to 12 cents a cold answer. Quick mode is 11,549 tokens at 2 to 3 cents, and deep mode about 10,000 tokens at 3 to 4 cents, both estimated on a cold response with nothing cached. Neither grows in token usage with the size of the vault, which is what makes it safe for me to keep writing and keep growing it without the cost spiralling.
Photos
The photos all sit in a Cloudflare bucket and everything about them sits in a database. There are 184 of them, on the about board.
When someone asks a question the database runs two searches. One across the written descriptions, which are produced by scanning the images with a model. One across the tags, which are written by a model after comparing the description against my vault for context. The results are added together, because either search can find an image the other misses.
From everything that matched either search, the significance field is collected. That is one line saying what the image is to me, written by a model from the context vault and then vetted by me. Those lines go in with the question and the model picks which images belong in its answer, which then appear beside the text as it streams. The descriptions themselves never reach the prompt, which is why they can be as long as they need to be.
Analytics tracking
Every question asked and every answer goes into a database, so I can evaluate the answer and improve the brain. Each row holds the question, the answer, which notes it drew on, a flag if someone tried to prompt inject, how long it took, how many tokens it used, a flag if anything went wrong and why, where the visitor came from with the UTM campaign if there was one, and a hashed value standing in for the visitor that is rebuilt every month so I can tell one person asking ten questions from ten people asking one. It also holds a few minor things: whether it answered quickly or went deep, how many rounds it ran, the previous question asked, and the follow-up questions it offered. No IP addresses, no names, no contact details.
People do try to talk the brain into ignoring its instructions. Those get flagged and logged rather than blocked, because there is nothing secret in the public vault to protect. It is more useful to me as a signal than as something to stop.
The private vault then reads the public brain’s answers and marks them. A higher model with the whole private vault open goes through what people asked and what the brain said back and works out where it answered badly or could not answer at all, which turns into notes to write, prompts to change and sometimes features to build. It also tells me what people want to know, which I use to decide what to write next.
MCP
The brain is an MCP server at riufukazawa.com/api/mcp with three tools: list_notes, read_note and ask_brain. The two read tools are free to serve because the caller’s AI spends the caller’s tokens; only ask_brain runs on my account, so only that one is limited. There is no login, because everything behind it is already the public copy. The whole thing is also one plain fetch at riufukazawa.com/llms.txt.
If you are looking at hiring me and already have your own AI set up, point it at the server and let it interview my vault about almost anything, far enough to decide whether it is worth interviewing me. If you are a recruiter, try it.
Why the retrieval half does not go away
The vault keeps growing, which is the point of it and also why I do not think handing the whole thing to a model ever becomes the answer. A bigger context window makes that possible, not sensible: it is still slower and it still costs more, and my notes grow faster than the price comes down. So the problem stays the same one, which is working out what is worth reading before you read it.
