2026-10-13

Running a Private AI Chatbot on a $15 Raspberry Pi Zero

Build an offline AI assistant with Wikipedia on a $15 Pi Zero 2W. No internet, no account, no monthly bill. See the real speeds, memory fixes, and honest limits.

In short

A Raspberry Pi Zero 2W with 512 MB of RAM can run a small language model and offline Wikipedia, completely disconnected from the internet.
Memory tuning—disabling zram, freeing GPU memory, and limiting Ollama—matters more than the model size; it doubled inference speed from 2.5 to 5.4-6.5 tokens per second.
The chatbot answers questions by searching Wikipedia offline and passing matching sentences to the model, avoiding hallucinations on factual queries.
Speed is slow (72 to 115 seconds for a full answer) and accuracy is limited, but everything stays private on the board with no account or subscription.

Why You Might Want This

If you've ever wanted an AI that works without Wi-Fi, never sends your questions anywhere, and costs nothing per month, you've probably hit the same wall: every guide online uses a board with 8 gigabytes of memory and an add-on accelerator. A Raspberry Pi Zero 2W has half a gigabyte and costs fifteen dollars. The catch is that tiny memory makes everything slow, and the processor has to swap to the SD card for every word.

The real problem isn't the processor—it's fitting a language model into 512 megabytes and keeping it there while it thinks. Most people solve this by sending the AI work to another computer on the network. But you can run the model right on the board itself if you tune the memory and pick the right tools.

What You're Actually Building

The setup has three parts. A tiny language model that fits in memory, a runtime engine to run it, and an offline copy of Wikipedia so the model can look things up instead of guessing. The models are SmolLM2 135M, SmolLM2 360M, and Qwen2.5 0.5B—all free and open-source. The runtimes are Ollama and llama.cpp. Kiwix serves Wikipedia from a single file with no internet needed.

The trick is that the model never loads all of Wikipedia. A small script searches it, pulls out only the sentences that match your question, and hands those to the model. This way the model stays small and fast, and the answers come from real data instead of the model's guesses.

Memory Tuning: The Real Work

On first boot, Linux only saw 415 megabytes of the 512 available—the rest was already reserved. That's where the fight started. The first fix was to limit Ollama itself: one model loaded at a time, one request at a time, and unload it after a minute. The second fix was a Modelfile for each model with a 1024 token context, a cap on answer length, and a repeat penalty to stop the model from looping.

The real speedbump was hiding in the operating system. Raspberry Pi OS Trixie turns on zram by default—compressed swap that lives in memory. It was holding about 232 megabytes when the kernel killed even the 360 million model. Switching zram off and using a one gigabyte swap file instead freed up space. A small rpi-swap config file makes this permanent, because a routine system reload would switch zram back on in the middle of a run.

The last fix was graphics memory. The board runs with no screen, yet 64 megabytes went to the GPU and 256 more to a graphics reservation. Lowering the GPU to 16 megabytes and the graphics reservation to 32 gave back 48 megabytes for free. After a reboot, Linux saw 463 megabytes instead of 415. That tuning doubled the 360 million model's speed from 2.5 to 5.4 to 6.5 tokens a second.

Real Speeds and Honest Limits

With llama.cpp, the 360 million model reads at 9.3 tokens a second and writes at 6.5 tokens a second, using 282 megabytes of memory. That was before any memory tuning. After tuning, Ollama caught up to the same speed. The Qwen 0.5B model is much slower—0.06 tokens a second—because its file is about 485 megabytes, bigger than the free memory, so it reads the model from the SD card for every word.

A full Wikipedia answer took 72 to 115 seconds. The model alone got facts wrong: it put Canberra in the northwest of Australia instead of the southeast, and it made up a Huawei website and ten million Pi sales. With offline Wikipedia, the same three questions came back right: Canberra in the southeast, Mount Everest at 8,848.86 meters, and over 75 million Raspberry Pis sold. But the model still makes mistakes. Asked 17 times 23, it said 409 instead of 391. Asked about Mount Everest, it dated the measurement to May 2024 instead of 2020. A date from a nearby sentence bled into the answer.

To prove it's offline, you can delete the default internet route and ping out. It says network is unreachable, and the chatbot still answers. Nothing leaves the board. No account, no API key, no subscription.

How to Build It: The Steps

Start with Raspberry Pi Imager, the free official flashing tool. Pick the Raspberry Pi Zero 2 W, then Raspberry Pi OS Lite 64-bit. Lite has no desktop, and on this board every megabyte counts. Give it a hostname, add your Wi-Fi, and switch on SSH so you can log in from your main computer. Write the card, let it verify, insert it into the Pi, and power it up.

Log in over SSH and update everything with apt. Then install Ollama with its official command. On the Pi it warns that there's no GPU, so everything runs on the processor. Pull the three models in the small Q4 format: they're 105, 270, and 397 megabytes. Then apply the memory fixes in order: limit Ollama in a systemd override file, add a Modelfile for each model with a 1024 token context and repeat penalty, switch zram off with rpi-swap, and lower GPU memory in config.txt to 16 megabytes with a 32 megabyte graphics reservation.

Next, install Kiwix with apt and download the mini English Wikipedia—about 14 gigabytes of every article intro, which took just under an hour. Run kiwix-serve as a service on port 8080 so it starts with the board. Finally, create a small script called ask.py that searches Wikipedia, skips the 'may refer to' pages, keeps only matching sentences, and hands those to the model with your question. The video above goes deeper into each step and shows the exact commands.

Adding a Screen and Voice

The board comes without header pins. A solderless 40-pin header presses into the holes straight and even. With the header in place, you can add a PiSugar Whisplay HAT with a small screen, microphones, and a speaker. The Whisplay needs its own driver installed from PiSugar's script, which adds settings to the boot config and asks for a reboot.

After the reboot, the screen test passes and a 240 by 280 frame goes to the screen in under a second. The sound side had a speedbump: the PiSugar battery board in the stack made the sound card disappear. With the battery board off the stack, the sound card shows up and the speaker works. The voice is Piper's Amy, an offline neural voice that loads once in about 25 seconds. A full conversation—speech to text, Wikipedia lookup, model thinking, and voice output—peaked at 235 megabytes of memory and took about 80 seconds.

Other Routes to Consider

A Raspberry Pi 5 with 8 gigabytes runs much bigger models faster. An Orange Pi Zero 3W is the same tiny size with far more memory options—up to 16 gigabytes—but check what's in stock. An old phone in a drawer usually has more memory than this board, and there are offline AI apps for phones. A used mini PC costs more and uses more power, but it runs bigger models and the full Wikipedia without a fight. Cloud AI is smarter and faster than anything here, but it needs the internet, an account, and often a monthly bill.

So yes, a fifteen dollar board can run a real AI chatbot with Wikipedia, completely offline. It's slow, it makes mistakes, and the memory tuning is what makes it work at all. But if you want privacy, no subscription, and a working chatbot in your pocket, this is how.

Watch "I Ran an AI Chatbot + All of Wikipedia on a $15 Pi Zero 2W (Wi-Fi Off)" on YouTube