From RAG to production: my LLMOps learning roadmap

When I started studying for ITA in 2018–2019, I picked up a cycle I still use: learn just enough to start, practice, and go back to the theory when something gets stuck. It works because the theory stops being abstract. It shows up exactly when you need it. I want to apply that same process to building LLM applications.

This roadmap starts with retrieval-augmented generation, or RAG, and ends with a service running on Kubernetes. The short version: instead of trusting only what the model memorized, you hand it the right documents along with the question. From there it is one application evolving: retrieve information, measure answer quality, build an API, chase down failures, automate delivery. LLM fundamentals and training from scratch stay out of scope.

Roadmap revised on October 20, 2026. The projects and progression criteria below are my own synthesis for guiding study, not official course requirements.

How to follow the path

Pick a corpus you already know: public technical documentation, your study notes, or a support knowledge base without sensitive data. Keep the same corpus across all three capstones, so you compare how the results evolve without also changing the ground under your feet.

To start, Python, Git, HTTP/JSON, basic SQL, and Docker are enough. Kubernetes shows up halfway through. A hosted model API covers the early stages, so you do not need a GPU on your desk this soon.

The path has four phases:

1. Build: ingestion → retrieval → answers with sources.
2. Measure: evaluation → retrieval improvements → API → tracing.
3. Operate: data updates → Kubernetes → inference service.
4. Deliver: monitoring → scaling → GitOps → failure recovery.

The phases overlap. Do not be surprised to walk back to an earlier one when a load test exposes a retrieval problem.

@mnsgrosa inverted-classroom 2 stars

My study loop became an open skill, and the card above leads to the repository where it lives. It is the inverted classroom with a habit of mine: I cannot study without trying first anymore. I read just enough to understand what the exercise is asking, one idea or one example, and go straight to the code. Getting it wrong there is the point. The error is what tells you which page of theory is worth opening, not the other way around.

That is why every section below puts the practice before the links. Reading everything first gives you a pleasant feeling of understanding. And it disappears the moment you run the first index build. Trying first is what shows you what was missing.

1. Build a reproducible RAG baseline

Start with an explicit pipeline, no magic tricks: load documents, clean the text, split it into chunks, generate embeddings, index, and retrieve context to answer. Store document identifiers, provenance, and versions from day one. PostgreSQL with pgvector works as your main store; FAISS is worth one exercise for comparison.

Your practice: write commands to index a small corpus and ask questions. Record the retrieved chunks and return answers with citations you can open and check. When the answer is not in the corpus, the system has to say so instead of making something up.

Move on when: you can rebuild the index from a clean installation, explain five answers end to end, and show questions the corpus cannot answer.

Resources: Maxime Labonne’s LLM Engineer track and RAG and vector search in LLM Zoomcamp.

2. Evaluate before making retrieval more complex

The temptation here is strong: jump straight to hybrid search and reranking because it sounds more advanced. Hold off for one step. Without an evaluation set, you cannot tell whether you improved the system or just swapped it.

Write questions whose relevant documents you already know. Keep the development set separate from the final evaluation, or you end up tuning the system for the test. Include unanswerable questions, ambiguous ones, and attempts to read content that should not be available.

Measure retrieval and generation separately, because the bug can live on either side and you need to know which one is at fault. Hit Rate and MRR judge search; correctness, grounding, and citation accuracy judge answers. An LLM judge is only useful with a rubric and a comparison against human review.

Then compare the baseline with hybrid search and reranking. Change chunk size and the context budget one variable at a time. Shift two things at once and a better score tells you nothing about which one did the work. Query rewriting, HyDE, and SQL or graph retrieval can wait for the problem that justifies them.

Move on when: you have a comparable report across configurations, with per-question failures, quality, latency, and cost. Write down why you chose the final setup.

Resources: Zoomcamp’s evaluation module, Ragas, and production RAG with LlamaIndex.

3. Build an observable application

Expose the pipeline through an API. Define request and response schemas, authentication, limits, timeouts, bounded retries, and cancellation. Keep ingestion separate from answering, because the two tempos are very different: one is a batch job, the other has to answer in seconds.

Instrument the whole path: API → retrieval → reranking → generation → response. A request identifier should tie logs, spans, sources, versions, and feedback together. Track latency, errors, tokens, and cost per successful answer, and decide carefully what data goes into telemetry.

Capstone 1, observable knowledge assistant: deliver an API or interface, a Docker Compose environment, reproducible evaluation, and a monitoring dashboard. Then trigger a retrieval failure and a timeout, and show how you found the cause in the collected signals.

Progression gate: set quality targets and latency/cost budgets before the final test. As a starting reference, try Hit Rate@5 of 0.85 on answerable questions and at least 90% valid citations. Report counts, human review, and limitations, including what did not work.

Resources: Zoomcamp’s capstone, FastAPI, Langfuse, and OpenTelemetry signals.

4. Operate the data and learn Kubernetes

After the first capstone, two tracks can move in parallel.

• Data: automate ingestion, reprocessing, and index publication. Test updates, deletion, access revocation, partial failures, and backup restoration. Running the job twice must not duplicate content.
• Platform: learn Pods, Deployments, Services, Jobs, Secrets, PVCs, probes, and resource limits. Use KIND to deploy the application and diagnose configuration, networking, and availability problems.

Move on when: you can replace a Pod without losing required data, keep an unavailable instance from receiving traffic, and restore a previous index version.

Resources: Zoomcamp orchestration, Kestra, Kubernetes Basics, and School of DevOps Lab 00.

5. Serve models, observe capacity, and scale

Connect the application to a model served with vLLM while keeping retrieval independent of the inference backend. Learn to version weights, validate artifacts, and wait for the model to load before accepting traffic. Compare OCI distribution with object-storage downloads through an initContainer.

Serving the same model three ways is worth the effort, even just to feel the difference:

• Plain vLLM Deployment: a starting point with few moving parts.
• vLLM Router: distribution across replicas and investigation of session/cache affinity.
• KServe: declarative model resources and reusable serving runtimes.

Use Prometheus and Grafana to link queues, cache pressure, and throughput to time to first token and the latency users actually feel. Compare fixed replicas with autoscaling. But watch out: a model takes time to start and capacity has a ceiling, so HPA and KEDA need to react to signals that fit the workload.

Capstone 2, reproducible serving platform: rebuild the environment, run load tests, deliberately lose a Pod, and demonstrate diagnosis and recovery. Pick the architecture from the measurements, not from what sounds most modern.

Resources: packaging, serving, observability, and scaling labs, vLLM metrics, and KServe. CPU exercises teach the operating mechanics; their performance numbers do not represent a GPU deployment.

6. Automate delivery and practice incidents

Version a complete release: application image, model, prompt, index, and evaluation report. Use CI to validate changes, Argo Workflows to run pipelines, and Argo CD to reconcile the version declared in Git.

The flow I want to implement is change → evaluation → immutable artifacts → promotion → observation → rollback when needed. School of DevOps Lab 12 teaches orchestration, but openly states that there is no quality gate and includes demonstration steps that need real scripts. In this roadmap, a failed evaluation blocks promotion. No exceptions.

Capstone 3, operating through an incident: simulate a corpus update, model unavailability, a traffic burst, index loss, and an access-boundary violation. Record impact, detection, diagnosis, recovery, and the regression test you added afterward.

Finish when: you can promote and roll back a real version, restore data, and present the architecture, quality/capacity/cost results, service objectives, and operating procedures.

Resources: GitOps with Argo CD and the Argo Workflows pipeline.

Optional deeper study

Fine-tuning enters when failure analysis justifies adapting model behavior. If retrieval solves the problem, do not complicate it. When it does make sense, use reviewed data, an independent evaluation, and a comparison against the original model, and send the result through the same delivery process. The LoRA lab is a starting point.

GPU scheduling, GPU Operator, and inference optimization are another possible track, and they make more sense after you measure the service’s limits. The AI infrastructure roadmap helps organize that deeper study.

My criterion for continuing is simple: show what I built, explain the result, reproduce the evidence.