Offline RAG Knowledge Assistant
A local chatbot over offline Wikipedia and Arch Wiki, answered by a self-hosted Llama 3.2.
- Status:
- Archived
- Stack
- 9
- Architecture notes
- 4
Internal enterprise or private repository
Ask me about this projectOverview
A local chatbot over offline Wikipedia and Arch Wiki snapshots, built with FastAPI, Ollama running Llama 3.2, and Streamlit. The .zim archives are served locally through Kiwix, retrieval runs over them, and generation runs on the same box — so both halves of RAG are self-hosted. What you end up with is a knowledge assistant that keeps working happily with the internet unplugged, with no API key, no per-token bill, and nothing leaving the machine.
Architecture Breakdown
How it works
Core mechanics, failure recovery paths, and system design decisions.
- 01
Archive layer
Kiwix serves the .zim files locally, which turns an offline Wikipedia and Arch Wiki snapshot into something you can search in full.
- 02
Retrieval & API layer
A FastAPI service sits over the archives, turning a question into a retrieval against the local corpus and handing the model only the passages that matter — the same bounded-context discipline that drives repo2graph, applied to prose instead of code.
- 03
Local inference
Ollama runs Llama 3.2 on the same hardware, so generation is local too. No request leaves the box and there is no per-token cost to think about while iterating.
- 04
Interface
A Streamlit front end for conversational querying, with a Windows batch launcher that brings the whole stack up in one go.
Key Takeaways
Engineering Highlights
- Fully offline RAG: retrieval and generation both run locally, with the internet unplugged
- Self-hosted Llama 3.2 through Ollama — no API key and no per-token bill
- FastAPI retrieval layer over offline Wikipedia and Arch Wiki .zim archives
- Streamlit interface for conversational querying
- Runs on ordinary homelab hardware
- One command starts the whole stack
Stack & Tools
- Python
- FastAPI
- Ollama
- Llama 3.2
- Streamlit
- Kiwix
- .zim archives
- Batch
- Self-hosted