itza-rag

I run a local agent on a small model (qwen2.5-coder:7b) on my own machine: no browsing, no paid API, no context window big enough to swallow a whole repo. For it to be useful at all, it had to know the folder it works on. itza-rag is the smallest thing that solves that: a local RAG pipeline in a single Python file — no LangChain, no vector database, no third-party service.

What it does

Two commands, that’s the whole surface:

itza_rag.py index  <folder> [--out DIR] [--exts .py,.ts,.md,...]
itza_rag.py query  "<question>" --index DIR [--k 5] [--model qwen2.5-coder:7b]

index walks the folder, splits files into chunks, embeds them and writes the result to disk. query embeds the question, pulls the k most similar chunks, pastes them as context and hands them to the local model with a deliberately strict instruction: answer using only the context below, cite the file:lines you took it from, and if the context isn’t enough, answer NO_SE: with what’s missing. A small model forced to cite, and explicitly allowed to say “I don’t know”, makes far fewer things up.

It also ships with its own usage rule written into it, which is the part I care about: for “where is X defined?” you use ripgrep, not this. This pipeline is for conceptual questions — “how does X work?”, “why was Y done this way?” — which are exactly the ones a literal search can’t answer. Knowing which tool not to reach for is half the value.

Under the hood

Embeddings via Ollama. It calls /api/embed with nomic-embed-text, in batches of 32, against the Ollama instance already running locally. No key, no quota, and the code never leaves the machine.

Chunking depends on the file. .md files are split by heading (#, ##, …), because in a document the heading is the semantic unit. Code is split into blocks separated by blank lines, then merged up to an 80-line ceiling — which avoids both single-line chunks (useless for cosine) and giant chunks that dilute the signal. Every chunk is stored with its file and line range; that’s where the citations come from.

The “vector store” is two files. An embeddings.npy matrix holding the already-normalized vectors, and a chunks.json with the texts and their origin. No Chroma, no Pinecone, no FAISS.

Search is a dot product. Since the vectors are normalized at index time, cosine similarity is literally matrix @ qvec, and the top-k falls out of an argsort. It’s brute force and I won’t apologize: under ~100k chunks NumPy does it in a blink, and standing up an ANN index here would be paying complexity for a problem I don’t have.

It also skips what should be skipped (.git, node_modules, dist, .venv, __pycache__, .angular, …) and honors a configurable extension list.

The stack

• Python 3 — standard library plus NumPy, nothing else. Zero RAG frameworks.
• Ollama locally: nomic-embed-text to embed, qwen2.5-coder:7b (or whatever you pass to --model) to answer.
• On-disk store: .npy + .json, two files per index.
• An argparse CLI, about 200 lines in a single file.

Worth noting

This project is on the list precisely for what it doesn’t have. A “complete” RAG setup today means five dependencies, a vector database server, and an abstraction layer you have to learn first. The version I actually needed — index a folder, answer conceptual questions about it — fits in one file I can read end to end in five minutes, which is exactly why I can debug it and change the chunking when an answer comes out wrong.

It’s also the local half of a bigger arrangement: the small on-machine agent can answer questions about a repo without shipping the code to any API or spending the large models’ quota. That split — cheap and private locally, expensive and capable in the cloud, with a clear way to decide what goes where — is the same idea behind Agent Fleet Console, the other project on this list.