Pipeline Components - MS-RAGS(ALL-IN-ONE) Docs

Component Selection Map

Build the pipeline in this order. Do not start with the fanciest option. Start with the simplest working configuration, test real questions, then add stronger components only where the failure is visible.

Model

Choose the LLM that can fit your retrieved context, answer style, latency budget, privacy needs, and provider credentials.

Extractor

Choose how documents become text. Weak extraction creates weak answers even with the best model.

Chunking

Split text so each chunk is small enough to retrieve but complete enough to answer.

Embeddings

Choose vector dimensions before creating the vector DB collection. Changing embeddings means re-ingestion.

Retrieval

Start dense. Add keyword, hybrid, MMR, parent-child, or self-query only when the retrieved chunks show the need.

Quality layers

Add query enhancement, reranking, compression, evaluation, and traces after baseline retrieval works.

FilesExtractorChunksEmbeddingsVector DB

Retrievaldense, hybrid, MMR, parent-child, multi-vector, graph traversal

Qualityquery enhancement, reranking, compression, evaluation

Outputlive answer, traces, pipeline.py, requirements.txt, .env

LLM Providers

Providers are used for generation, query enhancement, LLM-based reranking, summary compression, and some evaluation flows. MS-RAGS(ALL-IN-ONE) keeps credentials in memory during the session and never writes secret values into saved pipeline configs or generated source code.

Provider	Variables	Best use
OpenAI	`OPENAI_API_KEY`, optional `OPENAI_ORG_ID`	Best general hosted default.
Anthropic	`ANTHROPIC_API_KEY`	Careful reasoning and long-form answers.
Cohere	`COHERE_API_KEY`	Embeddings and reranking.
HuggingFace	`HUGGINGFACEHUB_API_TOKEN`	Hosted token-only embeddings or local model ecosystem.
Google Gemini	`GOOGLE_API_KEY`	Google cloud workflows.
Mistral AI	`MISTRAL_API_KEY`	Multilingual and EU-friendly hosted use.
Groq	`GROQ_API_KEY`	Fast inference.
Together AI	`TOGETHER_API_KEY`	Hosted open models.
Replicate	`REPLICATE_API_TOKEN`	Model experimentation.
Azure OpenAI	`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_DEPLOYMENT_NAME`	Azure enterprise deployments.
AWS Bedrock	`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, optional `AWS_REGION`	AWS enterprise deployments.
Ollama	`OLLAMA_MODEL_NAME`, optional `OLLAMA_API_KEY`, optional `OLLAMA_BASE_URL`	Chat models can use local Ollama or Ollama Cloud; embeddings must use local or self-hosted Ollama.

Supported Model Guide

Context-window rule: the final prompt includes system instructions, the user question, retrieved chunks, and any query enhancement or compression output. Pick a model whose context window can hold that full prompt with room for the answer. Context windows change, so check the official provider link before production.

Provider	MS-RAGS(ALL-IN-ONE) default or input	Planning context window	Recommended when	Watch out for
OpenAI	`gpt-4o` by default; users can pass another OpenAI chat model in generated code.	Plan around 128k for `gpt-4o`; newer OpenAI models may differ.	Best first hosted choice, strong general QA, good tool ecosystem.	Hosted API sends prompt/context to OpenAI. Long context increases cost and latency.
Anthropic	`claude-3-5-sonnet-20241022`.	Plan around 200k for Claude Sonnet-family RAG prompts.	Long-form answers, careful reasoning, policy/legal/research style responses.	Extra retrieval, self-checking, or compression calls can multiply cost.
Cohere	`command-a-03-2025` by default; versioned Command R models can also be used.	Plan around long-context retrieval prompts and verify the exact context window on Cohere's official model page before production use.	Enterprise RAG, reranking plus generation workflows, multilingual retrieval apps.	Avoid removed unversioned aliases such as `command-r-plus`; use current/versioned model IDs.
Hugging Face	`meta-llama/Meta-Llama-3-8B-Instruct` by default.	Plan around 8k for Llama 3 8B Instruct unless your endpoint says otherwise.	Token-only hosted open models or custom inference endpoints.	Model availability, rate limits, and context windows vary by endpoint.
Google Gemini	`gemini-1.5-pro`.	Plan around 1M-2M depending on the Gemini 1.5 Pro configuration and current limits.	Very large document context, Google ecosystem, multimodal-adjacent workflows.	Huge context is not a replacement for good retrieval; still chunk and rerank.
Mistral AI	`mistral-large-latest`.	Plan around 128k for Mistral Large-class prompts.	Multilingual or EU-friendly hosted RAG, balanced quality and latency.	Use model-specific limits before selecting chunk count and top_k.
Groq	`llama-3.3-70b-versatile`.	Plan around 128k for Llama 3.3 70B Versatile, then verify the Groq model page.	Low-latency generation where fast answers matter.	Speed does not fix bad retrieval; inspect chunks before tuning model choice.
Together AI	`meta-llama/Meta-Llama-3-8B-Instruct-Turbo`.	Plan around 8k for Llama 3 8B Instruct variants unless the endpoint exposes more.	Hosted open-model experimentation and cost-sensitive prototypes.	Smaller models need cleaner context and stricter prompts.
Replicate	`meta/meta-llama-3-8b-instruct`.	Plan around 8k for this Llama 3 8B Instruct version.	Testing model variants before committing to a provider.	Version pins matter; provider model pages can change.
Azure OpenAI	User-provided `AZURE_OPENAI_DEPLOYMENT_NAME`.	Use the context window of the deployed Azure model, often 128k for GPT-4o deployments.	Azure enterprise, private networking, existing Azure governance.	Deployment name is not always the public model name; document it clearly.
AWS Bedrock	`anthropic.claude-3-5-sonnet-20241022-v2:0`.	Plan around 200k for Claude 3.5 Sonnet on Bedrock where available.	AWS enterprise deployments and Bedrock governance.	Model availability varies by region and account permissions.
Ollama	`OLLAMA_MODEL_NAME`, such as local `llama3` or cloud `gpt-oss:120b`.	Model/runtime dependent. Local defaults may be lower than the model maximum; check `ollama show <model>` and `num_ctx`.	Local/private runs and Ollama Cloud chat.	Ollama Cloud currently supports chat only. Embedding models such as `nomic-embed-text` must run on local or self-hosted Ollama.

Embedding Models

Important: embedding dimensions must match the vector DB collection or index. If the user changes embedding model, use a new collection or re-ingest.

OpenAI

text-embedding-3-small is the recommended hosted default. text-embedding-3-large is better quality but larger and more expensive.

Cohere

Use embed-english-v3.0 for English and embed-multilingual-v3.0 for multilingual retrieval.

HuggingFace Local

Use all-MiniLM-L6-v2 for CPU speed, all-mpnet-base-v2 for stronger general quality, BGE/E5 for advanced local retrieval.

HuggingFace Hosted

Use HuggingFaceEndpointEmbeddings with HUGGINGFACEHUB_API_TOKEN when users want token-only hosted embeddings with no local download.

Google and Mistral

Use when you want the embedding provider to match your LLM/cloud provider.

Ollama

Use local embedding models like nomic-embed-text, or cloud-compatible Ollama settings when credentials are supplied.

Embedding Model Catalogue

Model	Provider	Dimensions	Use when	Where it runs
`text-embedding-3-small`	OpenAI	1536	Best first hosted embedding for cost, speed, and solid retrieval.	Hosted API
`text-embedding-3-large`	OpenAI	3072	Higher retrieval quality matters more than vector size/cost.	Hosted API
`text-embedding-ada-002`	OpenAI	1536	Legacy projects already use it and need compatibility.	Hosted API
`embed-english-v3.0`	Cohere	1024	English-only retrieval with Cohere stack.	Hosted API
`embed-multilingual-v3.0`	Cohere	1024	Multilingual documents or users ask in more than one language.	Hosted API
`all-MiniLM-L6-v2`	Hugging Face	384	Fast local CPU baseline, demos, small corpora.	Local or HF hosted endpoint
`all-mpnet-base-v2`	Hugging Face	768	Better local semantic quality for general documents.	Local or HF hosted endpoint
`multi-qa-mpnet-base-dot-v1`	Hugging Face	768	Question-answer retrieval where queries are natural-language questions.	Local
`BAAI/bge-small-en-v1.5`	Hugging Face	384	Fast English local retrieval with BGE behavior.	Local
`BAAI/bge-base-en-v1.5`	Hugging Face	768	Balanced local quality and resource use.	Local
`BAAI/bge-large-en-v1.5`	Hugging Face	1024	Best English BGE quality when GPU/RAM budget allows.	Local
`BAAI/bge-m3`	Hugging Face	1024	Multilingual, dense+sparse, advanced retrieval experiments.	Local
`intfloat/e5-small-v2`	Hugging Face	384	Resource-constrained local systems.	Local
`intfloat/e5-base-v2`	Hugging Face	768	General-purpose local retrieval.	Local
`intfloat/e5-large-v2`	Hugging Face	1024	Higher-quality E5 retrieval with enough compute.	Local
`intfloat/e5-mistral-7b-instruct`	Hugging Face	4096	High-quality local/GPU instruction embeddings.	Local, GPU recommended
`hkunlp/instructor-base`, `large`, `xl`	Hugging Face	768	Task-instruction-aware retrieval.	Local
`models/text-embedding-004`	Google Gemini	768	Google provider alignment and task-type embeddings.	Hosted API
`mistral-embed`	Mistral AI	1024	Mistral provider alignment and multilingual retrieval.	Hosted API
Ollama user model	Ollama	Model-dependent	Private/local embeddings such as `nomic-embed-text` or `mxbai-embed-large`.	Local or Ollama-compatible endpoint

Embedding Technical Notes

Choice	Technical behavior	Risk to explain to users
Hosted embeddings	Documents are sent to the provider API and vectors are returned.	Requires network and credentials; may have data governance implications.
Local HuggingFace	Model weights are downloaded and inference runs on the user's machine. MS-RAGS disables the fragile Xet transfer path by default and, if a partial cache breaks loading, asks permission before cleaning only that model cache and retrying.	First run may be slow and disk-heavy; CPU models can be slower.
Hosted HuggingFace endpoint	Uses `HuggingFaceEndpointEmbeddings` and `HUGGINGFACEHUB_API_TOKEN`.	Token-only setup is easier, but endpoint availability and rate limits matter.
Ollama local/cloud	No API key defaults to local `http://localhost:11434`; API key defaults to `https://ollama.com` for chat. The framework normalizes `/v1` to the root base URL.	Ollama Cloud is chat-only; embedding models must use local or self-hosted Ollama.
Dimension changes	Vector dimensions are fixed by the embedding model.	Changing models requires a new collection or full re-ingestion.

Extractors And Loaders

Source	Loaders	Recommendation
PDF	PyPDFLoader, UnstructuredPDFLoader, PDFPlumberLoader, CamelotLoader, TabulaLoader, LlamaParseLoader	Start with PyPDFLoader. Use PDFPlumber/Camelot for tables. Use LlamaParse for complex paid parsing.
Word	UnstructuredWordDocumentLoader, Docx2txtLoader, LlamaParseLoader	Use Unstructured for structure, Docx2txt for simple text.
CSV/Excel	CSVLoader, UnstructuredCSVLoader, UnstructuredExcelLoader, PandasDataFrameLoader	Use CSVLoader for row-level retrieval, Pandas for custom preprocessing.
PPTX	UnstructuredPowerPointLoader, LlamaParseLoader	Use Unstructured unless slides are complex.
HTML/URLs	BSHTMLLoader, UnstructuredHTMLLoader, WebBaseLoader, AsyncHtmlLoader, FireCrawlLoader, ApifyWebScraper	Use WebBaseLoader for simple pages, FireCrawl/Apify for rendered sites.
Markdown/JSON/XML	UnstructuredMarkdownLoader, JSONLoader, UnstructuredXMLLoader, TextLoader	Use format-aware loaders when metadata matters.
ePub/RTF	UnstructuredEPubLoader, UnstructuredRTFLoader	Use for books, manuals, rich text exports, and legacy text archives.
YouTube	YoutubeLoader	Use when transcripts are available and accurate.
Images	UnstructuredImageLoader	Verify OCR quality before production ingestion.
Code	GenericLoader, TextLoader	Use GenericLoader with code-aware chunking for repository search.
SQL/MongoDB	SQLDatabaseLoader, MongoDBAtlasLoader	Use when source knowledge lives in databases.

Extractor Decision Guide

Extractor	What it does	When to use	Why choose it	Where to be careful
PyPDFLoader	Reads PDF text quickly using a simple parser.	Clean digital PDFs with normal text flow.	Fast, free, good first PDF default.	Weak on scanned PDFs, complex tables, multi-column layouts.
UnstructuredPDFLoader	Parses PDFs into text elements with layout awareness.	Mixed PDF layouts, scanned/OCR workflows, messy enterprise documents.	Better structure than simple extraction. MS-RAGS checks for Poppler before ingestion when PDF sources are selected.	Needs Poppler for many PDFs and Tesseract for local OCR quality; choose LlamaParse if you prefer cloud parsing.
PDFPlumberLoader	Extracts PDF text with stronger layout/table inspection.	Reports, invoices, PDFs with columns or tables.	Good when table position and text layout matter.	Still needs manual sample review.
CamelotLoader	Specialized PDF table extraction.	PDFs where the answer lives inside tables.	Better table recovery than plain text loaders.	Works best on digital PDFs with clear table lines/structure; Ghostscript is recommended.
TabulaLoader	Java-based PDF table extraction.	Table-heavy PDFs where Tabula performs better than Camelot.	Useful alternate table parser.	Requires Java and table quality varies by PDF structure.
LlamaParseLoader	Cloud parser for complex documents.	High-value PDFs, DOCX, PPTX, scanned or layout-heavy documents.	Best when extraction quality matters more than cost.	Requires credentials and sends documents to a cloud parser.
UnstructuredWordDocumentLoader	Parses Word documents with structure.	Policies, manuals, reports with headings.	Keeps document hierarchy more useful for chunking.	Inspect lists, tables, and images.
Docx2txtLoader	Simple DOCX-to-text extraction.	Clean Word docs where plain text is enough.	Lightweight and easy.	Less structure and metadata.
CSVLoader	Turns CSV rows into documents.	FAQ tables, product catalogs, structured row-level data.	Good when each row is a retrievable fact.	Bad if rows require joins or complex aggregation.
PandasDataFrameLoader	Loads tabular data through pandas.	You need preprocessing, filtering, computed columns, or custom row text.	Maximum control over table-to-document conversion.	Requires careful schema decisions.
UnstructuredExcelLoader	Extracts Excel sheets into text/documents.	Simple spreadsheet knowledge bases.	Handles workbook/sheet structure.	Formula values and merged cells should be reviewed.
UnstructuredPowerPointLoader	Extracts slide text.	Training decks, sales decks, course slides.	Preserves slide-level source metadata.	Speaker notes, diagrams, and images may need extra extraction.
BSHTMLLoader	Extracts text from HTML with BeautifulSoup.	Local static HTML pages.	Simple and predictable.	Does not execute JavaScript.
WebBaseLoader	Fetches web pages with requests and parses text.	Simple public pages and documentation sites.	Good first URL loader.	Rendered apps, auth, and dynamic content may fail.
AsyncHtmlLoader	Fetches many HTML pages concurrently.	Multiple URLs where speed matters.	Faster batch ingestion.	Respect rate limits and robots policies.
FireCrawlLoader	Cloud web extraction with cleaned markdown.	JavaScript-rendered websites or multi-page crawling.	Cleaner RAG-ready web text.	Requires credentials and sends URLs/content to a service.
ApifyWebScraper	Actor-based cloud scraping.	Complex sites, repeatable crawl jobs, scalable scraping.	Operational web extraction platform.	Credential, cost, and site-permission considerations.
YoutubeLoader	Loads YouTube transcripts.	Video knowledge where transcripts are available.	Good for courses, interviews, talks.	Transcript accuracy controls answer quality.
UnstructuredImageLoader	OCR text from images.	Scanned images, screenshots, photographed pages.	Adds image text to RAG corpus.	OCR noise must be sampled before production.
UnstructuredEPubLoader	Extracts text from ePub books and manuals.	Books, long guides, educational content, and packaged documentation.	Keeps book-style content available for chunking.	Review chapters, navigation text, and boilerplate before ingestion.
UnstructuredRTFLoader	Extracts text from RTF rich text documents.	Legacy reports, exported notes, and older office documents.	Useful when the source is not DOCX or plain text.	Formatting artifacts can appear in old RTF exports.
GenericLoader	Language-aware source code loading.	Repository search, API docs, code understanding.	Pairs well with code-aware chunking.	Generated/vendor folders should be excluded.
SQLDatabaseLoader	Turns SQL rows into documents.	Knowledge lives in relational tables.	Useful for internal business data.	Do not expose sensitive columns accidentally.
MongoDBAtlasLoader	Loads MongoDB collection documents.	Product content or records already live in MongoDB.	Preserves document-native source shape.	Choose fields carefully and avoid private data.

Chunk Splitters

Recursive Character

Recommended default. Keeps paragraphs and sentences where possible.

Fixed Size

Consistent size, weaker semantic boundaries.

Semantic

Uses embeddings to detect topic changes.

Sentence

Preserves full sentence boundaries for precise domains.

Paragraph

Good for articles, books, and reports.

Token Based

Best when token budgets must match model windows.

Markdown/HTML Aware

Preserves document or web structure as metadata.

Code Aware

Splits source code by language-specific boundaries.

Agentic

LLM-driven chunk boundaries for high-value documents.

Document Aware

Uses headings and hierarchy for structured manuals.

Chunk Splitter Decision Guide

Splitter	What	When	Why	Where it can fail
Recursive Character	Splits by paragraph, sentence, word, then character fallback.	Default for most text/PDF/Markdown docs.	Stable, simple, strong baseline.	May cut tables or code awkwardly.
Fixed Size	Splits every N characters/tokens.	Need predictable chunk size for experiments.	Easy to compare and debug.	Can break semantic boundaries.
Semantic	Uses embeddings to detect topic changes.	Long prose where topics shift naturally.	Better semantic chunk boundaries.	Costs more because it needs embeddings during splitting.
Sentence	Keeps full sentences together.	Legal, medical, policy, or precise domains.	Reduces partial-sentence retrieval.	Can create tiny or uneven chunks.
Paragraph	Keeps paragraphs as primary units.	Reports, articles, books, manuals.	Natural author-defined context.	Long paragraphs may exceed retrieval budget.
Token Based	Splits by model tokenizer budget.	Strict context-window management.	Prevents prompt overflow.	Tokenizer mismatch can cause surprises.
Markdown/HTML Aware	Uses headings and document structure.	Docs sites, READMEs, manuals, HTML pages.	Preserves section metadata.	Bad markup creates bad hierarchy.
Code Aware	Splits by functions/classes/language syntax.	Repository search and code assistants.	Keeps code units meaningful.	Generated/minified files should be filtered out.
Agentic	Uses an LLM to decide chunk boundaries.	Small, high-value corpora where quality beats speed.	Can produce human-like sections.	Slow and costly for large ingestion.
Document Aware	Uses headings, pages, sections, and hierarchy.	Policies, PDFs, manuals, and structured docs.	Best for preserving source organization.	Requires reliable extracted structure.

Chunking Parameters

Parameter	Meaning	Recommendation
`chunk_size`	Maximum chunk length in characters or tokens depending on splitter.	Start around 800-1200 characters for general docs or 400-800 tokens for token-based splitting.
`chunk_overlap`	Repeated text between adjacent chunks.	Use 10-20 percent of chunk size. Too much overlap increases cost and duplicate retrieval.
`separators`	Priority boundaries for recursive splitting.	Keep paragraph and sentence boundaries before falling back to words or characters.
`tokenizer`	Tokenizer used to count tokens for token-based splitting.	Match the tokenizer to the model family where possible.
`language`	Programming language for code-aware splitting.	Choose the actual source language so functions/classes are preserved.

Retrieval, Enhancement, Reranking, Compression

Retrieval

Dense vector is the simple default. BM25/TF-IDF help exact terms. Hybrid combines semantic and keyword. MMR adds diversity. Self-query uses metadata filters.

Query Enhancement

Use rewriting for clarity, expansion for recall, HyDE for language mismatch, multi-query/fusion for ambiguous questions, and decomposition for complex questions.

Reranking

Use Cohere for hosted reranking, BGE/Cross-Encoder for local quality, FlashRank for CPU-friendly deployments, LLM scoring for flexible but expensive judgment.

Compression

Use embeddings filters for cheap pruning, redundancy removal for duplicates, LLM extraction/summary for noisy long chunks, and contextual compression for production retrieval wrappers.

Retrieval Strategy Reference

Strategy	How it works	When to use
Dense Vector	Embeds the query and ranks chunks by vector similarity.	Default semantic search.
BM25	Ranks by term frequency and document frequency.	Error codes, exact names, product IDs, legal phrases.
TF-IDF	Lightweight keyword scoring based on term importance.	Small local corpora and interpretable keyword baselines.
Hybrid	Combines vector and keyword scores using alpha.	Production docs where exact terms and meaning both matter.
MMR	Balances relevance with diversity.	When retrieved chunks are too repetitive.
Ensemble	Combines multiple retrievers with weights that sum to 1.0.	Advanced users with measured complementary retrievers.
Parent-Child	Retrieves small child chunks, maps them to parent IDs, and returns larger parent context.	Long PDFs, reports, policies, legal docs, and research docs where surrounding context matters.
Multi-Vector	Builds a local FAISS representation index from compact chunk representations and returns the original chunks.	Documents where short representations, titles, or first lines retrieve better than noisy full text.
Self-Query	LLM generates structured metadata filters.	Corpora with reliable metadata fields.
Time-Weighted	Combines dense similarity with `ms_rag_ingested_at` recency metadata.	Frequently updated knowledge bases, support tickets, changelogs, and policy updates.

Parent-Child state

MS-RAGS(ALL-IN-ONE) creates parent document IDs and child chunk IDs during ingestion. Saved sessions rebuild this state from original sources before runtime starts.

Multi-Vector model

Uses the selected embedding model to build a separate local FAISS representation index. Synthetic representation vectors are not written into Pinecone, Qdrant, Chroma, or other production stores.

Time metadata

Every chunk gets ms_rag_ingested_at. If timestamps are missing, MS-RAGS(ALL-IN-ONE) raises a setup error instead of silently pretending time-weighted retrieval is active.

Keyword store	Why it exists	Recommended use
SQLite	Persists raw chunk text locally for BM25/TF-IDF/hybrid.	Single-server production and development.
PostgreSQL	Stores chunk text and metadata beside production app data.	Managed Postgres deployments.
Elasticsearch	Provides production full-text search for keyword retrieval.	Enterprise search and large corpora.
OpenSearch	OpenSearch/AWS-compatible full-text keyword storage.	AWS/OpenSearch deployments.

Query Enhancement Reference

Technique	What it does	When to use	Why it helps	Risk
Query rewriting	Rephrases the user query into a clearer retrieval query.	User questions are vague, conversational, or messy.	Improves match between user wording and document wording.	Can remove important user intent if the rewrite is too aggressive.
Query expansion	Adds related terms, synonyms, acronyms, and domain phrases.	Documents use many names for the same concept.	Improves recall.	Can retrieve too much broad/noisy context.
HyDE	Generates a hypothetical answer/document and retrieves against it.	Questions are abstract and direct search misses context.	Bridges wording mismatch.	Hypothetical text can drift from the corpus.
Multi-query	Creates several query variants and retrieves for each.	One phrasing misses useful evidence.	Improves recall for ambiguous questions.	More retrieval calls and deduplication work.
Step-back	Asks a broader conceptual question first.	Questions need principles plus facts.	Adds background context before final answer.	Unnecessary for exact lookup questions.
Decomposition	Breaks a complex question into smaller sub-questions.	Multi-hop or comparison questions.	Finds evidence for each part separately.	More LLM/retrieval calls and harder trace review.
RAG-Fusion	Merges rankings from several generated queries.	Recall and ranking quality matter more than speed.	Balances multiple retrieval perspectives.	Higher latency and tuning complexity.

Reranking Reference

Reranker	What	When	Why	Tradeoff
Cohere Rerank	Hosted reranking model scores retrieved chunks.	Production systems need better final chunk ordering.	Strong quality without local model hosting.	Requires Cohere credentials and sends chunk text to Cohere.
Cross-Encoder	Local model scores query+chunk pairs together.	You want local/private reranking.	Better relevance than pure vector similarity.	Slower than embeddings; CPU can be expensive.
BGE Reranker	Local or hosted BGE-style reranking.	Open-source reranking quality is needed.	Good balance for support/docs RAG.	Model setup and GPU/CPU sizing matter.
ColBERT	Late-interaction ranking approach.	Advanced retrieval experiments and high recall/ranking needs.	Strong matching between query terms and document tokens.	More complex storage and serving.
FlashRank	Lightweight CPU-friendly reranking.	You need cheap reranking without a large service.	Easy operational footprint.	May not beat stronger hosted/local rerankers.
LLM scoring	Asks the LLM to score or filter chunks.	Domain-specific relevance rules are hard to encode.	Flexible and explainable if prompted well.	Slowest and most expensive option.

Context Compression Reference

Compressor	What it does	When to use	Why it helps	Risk
Embeddings filter	Drops chunks below similarity threshold.	Retrieved context includes weakly related chunks.	Cheap pruning before LLM calls.	Threshold too high can remove useful evidence.
Redundancy filter	Removes near-duplicate chunks.	Overlap or repeated boilerplate appears in results.	Saves context budget.	Can remove repeated but important evidence if tuned poorly.
LLM extraction	Extracts only relevant spans from chunks.	Chunks are long but contain a small relevant section.	Improves answer focus.	Extra LLM cost and possible missed facts.
LLM summary	Summarizes retrieved chunks before final answer.	Context is too long for the answer model.	Fits more evidence into the model window.	Summaries can lose citations or details.
Contextual compression retriever	Wraps retrieval with filtering/extraction.	You want compression as a reusable retrieval layer.	Keeps the main RAG chain cleaner.	Debug traces must show both original and compressed context.
Lost-in-the-middle reorder	Reorders chunks to improve attention to important context.	Many chunks are passed to a long-context model.	Places key evidence near prompt positions models attend to better.	Does not fix irrelevant retrieval.

Evaluation And Monitoring Reference

Tool or method	What it measures	When to use	Production reason
RAGAS	Faithfulness, answer relevancy, and no-reference context precision using an evaluator LLM. Step 15 asks for `OPENAI_API_KEY` and an evaluator model such as `gpt-4o-mini`; visible lexical fallback metrics are used if the evaluator API/package fails.	You need automated RAG quality metrics.	Detects retrieval and hallucination regressions without hiding evaluator credential or package problems.
DeepEval	LLM evaluation metrics using the configured evaluator model and OpenAI key.	You want CI-style evals around known questions.	Blocks low-quality generated pipelines before release and reports clean fallback metrics if the evaluator cannot run.
TruLens	Modern TruLens package validation when compatible, plus TruLens-prefixed groundedness scores with visible fallback warning when its LangChain adapter is incompatible.	You want TruLens-compatible local scoring and a path to deeper feedback workflows.	Helps inspect answer/context grounding without silently pretending an optional adapter loaded.
LangSmith	Trace debugging, datasets, evaluation runs.	LangChain-heavy teams need trace-level debugging.	Makes complex chains and agents observable.
Langfuse	Open-source LLM observability and prompt traces.	You want self-hostable or vendor-flexible observability.	Tracks prompts, latency, tokens, and errors.
Arize Phoenix	OpenInference/Phoenix trace export when the collector endpoint is configured, plus Phoenix-prefixed scores.	You need rich inspection of retrieval and model behavior.	Useful for production debugging and review.
ARES	ARES-compatible retrieval/generation scores. The external `ares-ai` package is optional and intentionally separate from production installs because current releases pin older OpenAI/Anthropic SDKs.	You want ARES-style RAG quality signals for a live query.	Keeps package-backed status separate from local fallback metrics and avoids Docker dependency conflicts.
RAGBench	Hugging Face datasets tooling check plus RAGBench-compatible single-query scores.	You want benchmark-style metrics while testing local generated pipelines.	Works without pretending to run the full external benchmark suite.
OpenTelemetry	Vendor-neutral traces and spans.	You already run observability infrastructure.	Connects MS-RAGS(ALL-IN-ONE) phases to dashboards and alerts.

Failure Modes And Fixes

Good answer exists but is not retrieved

Improve chunking, use query rewriting, enable hybrid retrieval, increase top_k, or add RAG-Fusion.

Retrieved chunks are relevant but noisy

Add reranking, MMR, contextual compression, embeddings filters, or smaller chunks.

Answers hallucinate

Use stricter system prompts, Self-RAG/Corrective RAG, faithfulness evaluation, and source-citation UX in generated code.

Costs are too high

Reduce top_k, use cheaper embeddings, remove unnecessary multi-query calls, and use embeddings filters before LLM compression.

Evaluation

Use RAGAS for faithfulness and context metrics with an explicit evaluator LLM, DeepEval for package-backed answer checks, TruLens-compatible groundedness scores, LangSmith, Langfuse, or Arize Phoenix for tracing, ARES/RAGBench benchmark-compatible signals, CI/CD gates for release protection, and monitoring export for production dashboards. Optional evaluator import incompatibilities or invalid evaluator credentials produce visible warnings and lexical fallback metrics.

Next: Vector databases