Role: Main developer — owned the platform end-to-end: UI and frontend, the NestJS + FastAPI backend, the LangGraph AI service, data, and CI/CD.
What it is. Full-stack AI meal-planning platform built as a pnpm monorepo and deployed on AWS. NestJS owns business logic while a separate Python FastAPI ai-service runs the AI brain.
AI generation pipeline. The AI brain is a single tool-using agent orchestrated as a LangGraph StateGraph with a ReAct inner loop that calls a USDA lookup tool but never writes nutrition numbers itself; the server computes every macro from USDA ground truth, a hard anti-hallucination guarantee. Two complementary RAG systems serve different jobs — a lexical USDA RAG (Postgres full-text + Haiku reranker) grounds every macro, while a separate semantic recipe RAG (HyDE → text-embedding-3-small → pgvector HNSW + full-text fused with Reciprocal Rank Fusion → Haiku reranker, seeded from RecipeNLG via a LlamaIndex ingestion pipeline) feeds curated dish ideas into a retrieve_recipes node before generation.
Reliability & quality. Quality is enforced by a 3-layer validate-and-retry cascade, and typed state is checkpointed to Postgres for crash-resume. Validated meals flow back into the recipe corpus through a guarded data flywheel, an offline eval harness with a CI gate catches regressions, and every generation is traced in self-hosted Langfuse.
Deployment. Server-Sent Events stream meals live from Python → NestJS → Next.js, all running on AWS — ECS Fargate plus Lambda/OpenNext over RDS, ElastiCache, and CloudFront.
AI Generation Pipeline
One tool-using LangGraph agent, traced end-to-end — the request flows top to bottom through nine layers, wrapped by two cross-cutting rails.
Client / API Gateway
Receives the request, enqueues a job, and POSTs to the Python service.
Interface / Serving
SSE endpoint with a 90s hang guard that streams generation events back.
Orchestration / Control-flow
A LangGraph StateGraph drives the day loop and the validate-retry loop.
Reasoning / Model
The ReAct decision loop — reason, act, observe, repeat.
Tooling / Action
Forced tool use to fetch ground-truth nutrition before committing a day.
The LLM never writes nutrition numbers — the server computes them from USDA ground truth.
A hard architectural constraint: the submit schema has no nutrition fields, so the model cannot fabricate macros — stronger than any prompt instruction.
Knowledge / Retrieval (RAG)
Two complementary RAG systems — exact ingredient macros vs open-ended dish ideas.
Exact ingredient → macro grounding.
Open-ended dish ideas before generation.
two RAGs, two query shapes — exact ingredient macros vs open-ended dish ideas
Validated meals are re-ingested into the recipe corpus via a LlamaIndex pipeline — future generations retrieve from real past outputs, not just the seed corpus.
Drift guards prevent mode collapse: provenance weighting (×0.7), a ≤1-of-3 generated cap, semantic dedup (cosine >0.95), and a weekly variety audit with clean rollback.
Memory / State / Persistence
Typed graph state checkpointed after every node for crash-resume.
Guardrails / Validation
Schema enforcement plus dietary-restriction checks on every output.
Evaluation / Quality
A 3-layer online validate-and-retry cascade, backed by an offline eval harness.
9. Observability
cross-cutting · spans every layerTraces every LLM call across the whole stack.
10. Data / Infrastructure
cross-cutting · spans every layerDeployed on AWS.
Key Features
- Single tool-using agent on a LangGraph StateGraph with a ReAct inner loop (generate_day ⇄ usda_tools → validate_day)
- Server-computed USDA nutrition — the LLM submit schema has no macro fields, so calorie/macro numbers can't be hallucinated
- 3-layer validate-and-retry: programmatic checks → LLM-as-judge → blocking USDA ground-truth fact-check, with an oscillation detector
- Crash-resume via typed MealPlanState checkpointed to Postgres after every node (AsyncPostgresSaver)
- Two complementary RAGs — lexical USDA grounding (Postgres FTS + Haiku reranker) and a semantic recipe vector RAG (HyDE + pgvector HNSW + RRF) feeding dish ideas before generation
- Real-time SSE streaming Python → NestJS → Next.js, self-hosted Langfuse tracing, deployed on AWS (Fargate + Lambda/OpenNext + RDS + ElastiCache)
Tech Stack
Backend
AI Orchestration
RAG & Retrieval
Eval & Observability
Frontend
Infrastructure
Challenges & Solutions
Reliable AI Meal Generation
An LLM is non-deterministic, yet a multi-day meal plan needs a reliable, repeatable pipeline with tool use, retries, and streaming — a hand-rolled while-loop gets brittle fast.
Modeled generation as a LangGraph StateGraph: a deterministic day loop (prepare_context → generate_day ⇄ usda_tools → validate_day → emit_day → update_history) wraps a single ReAct inner loop where the LLM decides tool calls, and forced tool_choice cleanly terminates each day.
Preventing Hallucinated Nutrition Numbers
LLMs confidently fabricate plausible-but-wrong calorie and macro values, and a meal planner that lies about nutrition is worse than useless.
The LLM's submit schema (DayPlanLLMSubmit) has no nutrition fields, so the model physically can't write a macro number — it must call a USDA lookup tool for ground truth and the server computes every calorie and macro. A hard architectural constraint, not a prompt instruction.
Enforcing Quality on Non-Deterministic Output
A generated day can still miss calorie/macro targets or violate dietary restrictions (keto, vegan, allergen-free), and a single LLM pass can't be trusted.
A 3-layer validate-and-retry cascade — programmatic checks → optional LLM-as-judge (Haiku) → a blocking USDA ground-truth fact-check that retries at >50% calorie or >60% macro deviation — re-prompts with structured feedback injected into the next attempt, with an oscillation detector to bail on stuck days.
Crash-Resumable Long-Running Generation
Generating a full multi-day plan is long-running, and a crash or timeout mid-plan would otherwise discard every day already produced.
Typed MealPlanState (TypedDict + reducers) is checkpointed to Postgres after every node via AsyncPostgresSaver, keyed by thread_id, so a half-finished plan resumes from the last completed node instead of restarting.
Two RAGs for Two Problems: Nutrition vs Dish Ideas
Exact ingredient→macro lookup and open-ended dish inspiration are different retrieval problems. Short ingredient names ('chicken breast, raw') are won by lexical search, while conceptual dish queries ('Moroccan chickpea stew' ≈ 'North African lentil soup') are lexically distant but semantically equivalent, so full-text search misses them — one retrieval mechanism can't serve both.
Two complementary RAG systems that don't compete. The factual layer is a lexical USDA RAG — Postgres full-text (tsvector/ts_rank) + a Haiku reranker — grounding every macro. The creative layer is a semantic recipe RAG in a retrieve_recipes node before generate_day: build a query from (cuisine, dietary_tags, calorie_target) → HyDE (one Haiku call writes a hypothetical recipe to close the query↔document embedding gap) → embed with text-embedding-3-small (1536-dim) → hybrid search (pgvector HNSW cosine, metadata-filtered, + FTS) fused with RRF (1/(60+rank)) → top-20 → Haiku reranker → top-3 injected as inspiration only. USDA still grounds every number; the recipe RAG only shapes dish selection.
Preventing Mode Collapse in the Data Flywheel
Re-ingesting the model's own validated meals into the recipe corpus closes a data flywheel, but conditioning future generation on past generations risks an ever-narrowing dish distribution — mode collapse, where the system retrieves and re-generates the same handful of dishes in a tightening loop.
A guarded self-ingestion ETL: only meals that pass all 3 cascade layers with <5% deviation, no oscillation, and no force-commit qualify. Drift guards prevent collapse — provenance weighting (generated recipes ×0.7 at retrieval), a fraction cap (≤1 of 3 injected recipes may be generated), semantic dedup (skip if cosine >0.95), and a weekly variety audit with a provenance audit trail for clean rollback. Feature-flagged until 30-day variety metrics prove stable.
Real-Time Streaming Across Services
Meal generation takes 10-30s through the AI pipeline. Users need immediate feedback, but the response crosses 3 service boundaries (Python → NestJS → Next.js).
SSE streaming pipeline where the FastAPI ai-service emits Server-Sent Events to NestJS, which proxies them to the Next.js frontend behind a 90s hang guard. Users see meals being generated in real-time.
Right-Sizing Cloud Compute per Workload on AWS
A mostly-cached SEO site, an always-warm authenticated app, and continuous BullMQ/LangGraph workers have opposite cost and latency profiles — no single runtime gives both zero-idle-cost and no cold start.
Split by workload: OpenNext → Lambda + S3 + CloudFront for the scale-to-zero cached SEO site, and ECS Fargate (always-warm) for the authenticated web-agent, NestJS API/workers, and FastAPI ai-service, backed by RDS PostgreSQL and ElastiCache Redis.