@mate_mattt: I built a real, runnable RAG project and a Notebook RAG practical course, breaking down RAG pixel by pixel: Markdown chunking → FTS5 / BM25 → Embedding vector search → Hybrid recall RRF → Cross-Encod…

X AI KOLs Timeline 07/12/26, 05:32 AM Tools

rag retrieval-augmented-generation notebook tutorial open-source local-rag

Summary

This is a hands-on project for learning local RAG retrieval core from scratch, including Notebook and real runnable code. It covers the complete workflow: Markdown chunking, BM25, Embedding vector search, hybrid recall RRF, Cross-Encoder re-ranking, and comes with evaluation metrics.

I built a real, runnable RAG project and a Notebook RAG practical course, breaking down RAG pixel by pixel: Markdown chunking → FTS5 / BM25 → Embedding vector search → Hybrid recall RRF → Cross-Encoder re-ranking → Evidence Card. It's not just concepts. Each lesson includes real data, code, running results, and evaluation metrics; from 15 documents chunked into 210 chunks, see how recall and re-ranking change. After reading and running it, you'll truly connect the core skeleton of RAG retrieval, and with AI, you can build your own RAG system. Open source Notebook: https://github.com/mate-matt/rag-memory-lab... If you find it helpful, give it a star. This is the most painstaking step-by-step teaching article I've ever written.

Original Article

View Cached Full Text

Cached at: 07/12/26, 07:00 PM

I built a real, runnable RAG project and a Notebook RAG hands-on course, deconstructing RAG at the pixel level:

Markdown Splitting → FTS5 / BM25 → Embedding Vector Search → Hybrid Recall RRF → Cross-Encoder Reranking → Evidence Card.

It’s not just concepts. Every lesson comes with real data, code, run results, and evaluation metrics — from 15 documents into about 210 chunks, then observe how recall and reranking change.

After reading and running it through, you’ll truly connect the core skeleton of RAG retrieval, and with AI, you’ll be able to build your own RAG system.

Open-source Notebook: https://github.com/mate-matt/rag-memory-lab… If you find it helpful, please give it a star — this is the most labor-intensive hands-on teaching article I’ve ever written.

mate-matt/rag-memory-lab

Source: https://github.com/mate-matt/rag-memory-lab

RAG Memory Lab

A hands-on project to learn the core of local RAG retrieval from scratch. It doesn’t call any external generation services, no API keys needed. You can run each notebook section by section, observing how Markdown is split, indexed, recalled, reranked, and finally turned into traceable evidence cards.

Companion Reading

X long post: Pixel-level RAG deconstruction with Notebook hands-on course (https://x.com/mate_mattt/status/2076175810265018554)
Blog article: Pixel-level RAG deconstruction with Notebook hands-on course (https://matematt.com/blog/notebook-rag/)

Both articles correspond exactly to notebooks 01–07 in this repo: the articles connect the ideas and concepts, while the notebooks run the code, observe real data, and verify results.

text Markdown documents → Chunk splitting → FTS5 / BM25 + local Embedding → RRF hybrid recall → Cross-Encoder reranking → Evidence Card (original text and source)

What You Will Learn

Lesson	Notebook	Topic
01	`01_markdown_parser_and_sentence_splitter.ipynb`	Markdown structure-aware splitting with length fallback
02	`02_sqlite_fts5_and_bm25.ipynb`	SQLite FTS5, inverted index, and BM25
03	`03_embeddings_numpy_milvus_and_fts_comparison.ipynb`	Local embedding, NumPy, Milvus Lite, and retrieval comparison
04	`04_retrieval_evaluation.ipynb`	Recall, Precision, Hit Rate, MRR, and human-annotated evaluation set
05	`05_hybrid_retrieval.ipynb`	RRF: fusing BM25 and vector recall
06	`06_reranking.ipynb`	Cross-Encoder reranking: fine ranking from candidates
07	`07_evidence_cards.ipynb`	Evidence Card: source, title path, and retrieval trail

Environment Requirements

macOS, Linux, or Windows (Python 3.11 recommended)
uv (https://docs.astral.sh/uv/): Python environment and dependency management tool
Internet access on first run: downloads two public local model weights; reuses local cache afterward
At least 8 GB RAM recommended. CPU is fine; lesson 6’s reranking will be slower on CPU.

The original materials are already provided with the repo; no additional downloads needed. Model weights are not committed to the repo, and no keys are required.

Quick Start

In the project root directory, run:

``bash

1. Install locked dependencies and create .venv

make setup

2. Register Jupyter kernel (one-time)

make kernel

3. Start JupyterLab

make lab ``

After the browser opens, select the RAG Memory Lab kernel and run notebooks sequentially from notebooks/01_... to notebooks/07_....

If you don’t have make, simply copy the corresponding uv run ... command from the Makefile and run it in the terminal.

One-Command Run of Real Data Pipeline

Order matters: each step reads local files generated by the previous step.

``bash

1. 15 Markdown files → ~210 chunks

make chunk-real

2. Build SQLite FTS5, Embedding, NumPy files, and Milvus Lite index

make vector-real

3. Evaluate single-path retrieval

make eval-retrieval

4. Hybrid recall with RRF and evaluate

make eval-hybrid

5. Rerank hybrid candidates with local Cross-Encoder and evaluate

make eval-rerank

6. Generate Evidence Cards with original text, source, and ranking trail

make build-evidence ``

Common notebook auto-run commands are also provided. For example:

bash make run-01 make run-03 make run-06

run-02 through run-07 depend on the registered RAG Memory Lab kernel, so run make kernel once first.

Project Structure

text rag-memory-lab/ ├── knowledge-base/openai-cookbook/ # Public Markdown source, upstream version, and license ├── notebooks/ # 01–07: interactive learning lessons ├── src/ # Reusable real pipeline code ├── artifacts/ │ └── evaluation/retrieval-goldens.jsonl # Human-annotated evaluation set (versioned) ├── assets/diagrams/ # Architecture diagrams for lessons and articles ├── Makefile # Common command entry ├── pyproject.toml # Python 3.11 dependency definition └── uv.lock # Reproducible dependency lock file

artifacts/chunks, artifacts/indexes, artifacts/vectors, and most evaluation results are run outputs, ignored by Git by default. After cloning the repo, run the above sequence to regenerate them.

Source Map

File	Purpose
`src/chunking.py`	Uses LlamaIndex’s `MarkdownNodeParser` and `SentenceSplitter` to split Markdown and save full metadata
`src/run_chunking.py`	Real data entry: generates `chunks.jsonl`, document statistics, and sampling review files
`src/fts5_index.py`	Creates a regular SQLite `chunks` table and an FTS5 virtual table, performs BM25 search
`src/vector_search.py`	Local embedding, NumPy cosine similarity, Milvus Lite index and query
`src/run_vector_index.py`	Builds FTS5, vector files, and Milvus Lite index from chunks
`src/evaluation.py` / `src/run_evaluation.py`	Reads goldens, computes Recall@K, Precision@K, Hit@K, MRR
`src/hybrid_search.py` / `src/run_hybrid_search.py`	Fuses BM25 and vector rankings using Reciprocal Rank Fusion (RRF)
`src/reranker.py` / `src/run_reranking.py`	Reranks hybrid candidates using a local Cross-Encoder
`src/evidence.py` / `src/run_evidence.py`	Wraps reranked results into traceable Evidence Cards

Data, Models & License

The original corpus is in knowledge-base/openai-cookbook, a fixed public snapshot of the OpenAI Cookbook. It retains SOURCE.json, upstream revision, and the original MIT License within its directory; please comply when redistributing.
This project’s code, notebooks, and custom diagrams use the MIT License in the root directory.
On first run, it will download two public models (sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 for vectorization and BAAI/bge-reranker-v2-m3 for reranking) to the local cache on demand. They are not included in the repo.

FAQ

make vector-real reports chunks.jsonl not found?

Run make chunk-real first.

Evaluation or reranking reports vectors/FTS5 database not found?

Run make vector-real first.

First run is very slow or requires network?

That’s downloading the public model weights. Once downloaded, it will use the local cache.

I just want to learn, not run the full pipeline?

Run make setup && make kernel && make lab, then start from lesson 01; the notebooks include small samples and real data examples.