MS-RAGS(ALL-IN-ONE) Docs

Production

Production Readiness

MS-RAGS(ALL-IN-ONE) gives you production-shaped wiring, but real production readiness also needs live smoke tests, monitoring, evaluation, and operational discipline.

Readiness Checklist

  1. Run the full test suite: python -m pytest tests/ -q.
  2. Run opt-in smoke tests against the exact vector DBs you will support.
  3. Confirm embedding dimension and vector collection compatibility.
  4. Enable structured logs and optional OpenTelemetry tracing.
  5. Configure at least one evaluation framework or CI/CD quality gate.
  6. Test generated pipeline.py in a clean environment.
  7. Rotate any API keys shared during development before launch.

Production Gate Checklist

Use this as the final review before giving MS-RAGS(ALL-IN-ONE)-generated code to production users. A pipeline is ready only when the code works, the data path is understood, the selected third-party services are verified, and failures are observable.

Gate What to verify Common failure it catches
Documents Sample every source type after extraction and inspect the chunks users will retrieve. Blank pages, OCR noise, broken tables, bad web extraction, duplicated boilerplate.
Embeddings Record model name, vector dimension, provider, and whether inference is hosted or local. Changing from one model to another without rebuilding the index.
Vector DB Run a live add/query smoke test against the exact production backend and collection. Wrong credentials, missing index, dimension mismatch, network or region errors.
Retrieval quality Test real user questions and inspect retrieved chunks before judging final answers. Good LLM response hiding poor retrieval.
Observability Confirm JSON logs and OpenTelemetry traces include ingestion, retrieval, generation, and failure events. No way to debug latency, empty context, provider errors, or poor answer quality.
Generated code Install generated requirements.txt in a clean environment and run ingest plus query. Hidden dependency on the MS-RAGS(ALL-IN-ONE) development checkout.

Third-Party Service Verification

When a production setup depends on a hosted service, open the official site and confirm credentials, limits, regions, model names, data policy, and billing before launch.

LLM and embeddings

Check OpenAI, Anthropic, Cohere, Hugging Face, Google Gemini API, Mistral AI, Groq, AWS Bedrock, Azure OpenAI, or Ollama depending on the provider selected.

Vector databases

Check Pinecone, Qdrant, ChromaDB, FAISS, Weaviate, Milvus, Redis, PGVector, OpenSearch, Azure AI Search, or MongoDB Atlas Vector Search before running live ingestion.

Parsing and extraction

For difficult files or web pages, verify official guidance for Unstructured, LlamaParse, Firecrawl, Apify, pypdf, and pdfplumber.

Evaluation and tracing

Use official docs for RAGAS, DeepEval, TruLens, LangSmith, Langfuse, Arize Phoenix, and OpenTelemetry.

What "Production Ready" Means Here

MS-RAGS(ALL-IN-ONE) can generate production-shaped code, but a RAG system is only production-ready after it has been tested against the exact providers, vector databases, documents, prompts, credentials, and deployment environment your users will rely on.

Area Required before launch Why
Retrieval quality Evaluation dataset, RAGAS/DeepEval metrics, manual review of hard queries. Passing unit tests does not prove answer quality.
Vector DB Live smoke test with the exact database and credentials. Connection settings, dimensions, and indexes fail differently per backend.
Observability Structured logs, visible runtime notices, OpenTelemetry traces, query success/failure events. Users need to debug ingestion, latency, retrieval, fallback behavior, and provider failures.
Security Secrets in environment variables or secret managers, never in generated code. Generated apps are meant to be committed and deployed safely.
Operations Backups, index naming, re-ingestion plan, model-change plan. Embedding model changes require deliberate index lifecycle management.

Logs And OpenTelemetry

At startup, the CLI asks whether to enable OpenTelemetry. Users can decline and continue with structured logging only.

MS_RAG_OTEL_ENABLED=1
OTEL_SERVICE_NAME=ms-rags-all-in-one
OTEL_ENVIRONMENT=production
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer your-token

Smoke Test Matrix

Test What it proves What it does not prove
Unit/property tests Model serialization, validation, prompt flow, code generation, factory wiring. Live cloud credentials or production DB availability.
Integration tests End-to-end local runtime wiring and rebuild behavior. Every hosted backend's operational behavior.
Vector DB smoke tests Real add/query cycle for selected vector backends. Full answer quality across your corpus.
Generated pipeline test Exported code can ingest and query outside the MS-RAGS(ALL-IN-ONE) package. Long-term monitoring or scaling behavior.

Docker Production Checklist

MS-RAGS(ALL-IN-ONE) includes a Dockerfile for teams that want a repeatable CLI runtime, CI smoke-test container, or production-style build environment.

  1. Use the public image with docker pull saad469/ms-rags-all-in-one, or build locally only when you need a custom dependency set.
  2. Pass secrets at runtime through --env-file, Docker secrets, or the deployment platform secret manager.
  3. Mount document input and output folders instead of copying private data into the image.
  4. Mount persistence folders for local Chroma, FAISS, graph stores, keyword SQLite, generated code, and agent memory when needed.
  5. If using host Ollama from Docker Desktop, set OLLAMA_BASE_URL=http://host.docker.internal:11434.
  6. For Redis vectors, build a Redis-focused image separately because the upstream Redis/HuggingFace dependency families currently conflict.
  7. For package-backed ARES, use a separate ARES-focused environment because current ares-ai releases pin older OpenAI/Anthropic SDKs that conflict with the production provider stack.
docker run --rm -it \
  --env-file .env \
  -v "${PWD}/SmokeDocs:/app/SmokeDocs:ro" \
  -v "${PWD}/output:/app/output" \
  saad469/ms-rags-all-in-one

Generated Code Workflow

# Ingest
python pipeline.py --ingest --sources ./docs/ https://example.com

# Query once
python pipeline.py --query "What does the policy say?"

# Interactive loop
python pipeline.py

Generated code reads secrets from environment variables and should be smoke-tested separately from the MS-RAGS(ALL-IN-ONE) CLI.

Security Notes

Credential handling

Saved pipeline configs do not include secret values. Generated code uses os.getenv so deployments can use environment variables or secret managers.

Token rotation

Rotate any API token pasted into development tools, logs, or chat during testing before production release.

Data governance

Hosted LLMs and hosted embeddings send text to external APIs. Use local models or approved enterprise providers for sensitive documents.

Access boundaries

Database credentials should be scoped to the needed collection/index where possible, especially for production vector DBs.

Troubleshooting

Embedding mismatch

Create a new collection or re-ingest when embedding dimensions change.

No chunks stored

Check paths, loader compatibility, OCR quality, and document permissions.

Weak answers

Improve extraction and chunking first, then add hybrid retrieval, reranking, query rewriting, or compression.

Provider API failures

Check API keys, rate limits, model names, network access, and whether the selected provider supports the requested chat or embedding operation.

Back to overview