A Self-Hosted Chatbot That Won’t Fake a Tool Result
Runs on a GTX 1060 6GB. Built around one rule the code actually enforces, not just prompts for.
I run a chatbot on an old gaming laptop. GTX 1060, 6GB. It can search the web, do math, read files, edit images, read spreadsheets, and build webpages. The one rule I cared about: if it says a tool told it something, a tool actually has to have told it that. Not implied by the prompt. Checked, in code, after the fact.
Small models lie about this constantly. Ask one for today’s date and it’ll write out a specific-sounding day even if it never checked. Nothing stops it from doing the same with a search result or a calculation.
How it avoids that
The reply is written by a separate step from the one that decides which tools to call. The step that writes the actual sentences you read only ever sees what a tool returned. It can’t call a tool itself. If a number isn’t in front of it, it has nothing to make one up with.
After the reply is written, I check it against the tool log anyway. Does the number it quoted match what the tool actually returned. Did it claim to search for something it never searched for. Small checks, but they catch the cases where the first layer wasn’t enough.
What it can actually do
Search the web, read a page, do exact math, convert units, check the date, read a public GitHub repo, pull fields out of a document, diff two pieces of text, build and edit a webpage, crop and resize images, and read a spreadsheet well enough to filter it or sum a column. Eleven tools right now.
Picking which tool to offer the model each turn needed to be fast and cheap, since this is one GPU with no room to waste on deciding. I tried embeddings first. Plain keyword matching (BM25 with TF-IDF) beat it, and it costs almost nothing to run. So that’s what it uses.
Memory
It keeps the last several messages word for word. Older messages get summarized instead of dropped. Specific details (a name, a number someone mentioned) get pulled out and kept on their own, since a summary tends to blur exactly the thing you’ll ask about again later.
Running it from home
This is reachable from outside my house, on the same network as everyone else’s devices in it. So a few things had to hold up: it checks where a URL actually resolves to before fetching it, so it can’t be pointed at something on my own network. Text pulled from the web gets defused before the model reads it, so a page can’t plant an instruction. Uploads and sessions are tied to whoever created them. Usage is rate-limited per person, so one visitor can’t tie up the only GPU for everyone else.
That’s it. It’s a chatbot that does what it says it did, on a laptop.