Role: Main developer — owned the platform end-to-end: UI and frontend, the NestJS + FastAPI backend, the LangGraph AI service, data, and CI/CD.
The agent. AI meal-planning pipeline built as a single tool-using agent on a LangGraph StateGraph with a ReAct inner loop: Claude Haiku 3.5 calls a USDA lookup tool, but the submit schema has no nutrition fields, so the server computes every macro from ground truth — hallucinated calories are architecturally impossible.
Reliability & quality. A 3-layer validate-and-retry cascade (programmatic checks → LLM-as-judge → blocking USDA fact-check) enforces macro targets and dietary restrictions, and typed MealPlanState is checkpointed to Postgres for crash-resume.
Dual RAG. Two complementary RAGs split the work — a lexical USDA RAG (Postgres FTS + Haiku reranker) grounds every macro, while a separate semantic recipe RAG (HyDE → text-embedding-3-small → pgvector HNSW + RRF → Haiku reranker, seeded from RecipeNLG via LlamaIndex) feeds dish ideas into a retrieve_recipes node before generation.
Flywheel & eval. Validated meals re-enter the corpus through a guarded data flywheel, an offline eval harness (RAGAS/DeepEval/promptfoo → MLflow, Recall@40, agent-trajectory metrics, CI gate) catches regressions, and every run is traced in self-hosted Langfuse.
AI Generation Pipeline
One tool-using LangGraph agent, traced end-to-end — the request flows top to bottom through nine layers, wrapped by two cross-cutting rails.
Client / API Gateway
Receives the request, enqueues a job, and POSTs to the Python service.
Interface / Serving
SSE endpoint with a 90s hang guard that streams generation events back.
Orchestration / Control-flow
A LangGraph StateGraph drives the day loop and the validate-retry loop.
Reasoning / Model
The ReAct decision loop — reason, act, observe, repeat.
Tooling / Action
Forced tool use to fetch ground-truth nutrition before committing a day.
The LLM never writes nutrition numbers — the server computes them from USDA ground truth.
A hard architectural constraint: the submit schema has no nutrition fields, so the model cannot fabricate macros — stronger than any prompt instruction.
Knowledge / Retrieval (RAG)
Two complementary RAG systems — exact ingredient macros vs open-ended dish ideas.
Exact ingredient → macro grounding.
Open-ended dish ideas before generation.
two RAGs, two query shapes — exact ingredient macros vs open-ended dish ideas
Validated meals are re-ingested into the recipe corpus via a LlamaIndex pipeline — future generations retrieve from real past outputs, not just the seed corpus.
Drift guards prevent mode collapse: provenance weighting (×0.7), a ≤1-of-3 generated cap, semantic dedup (cosine >0.95), and a weekly variety audit with clean rollback.
Memory / State / Persistence
Typed graph state checkpointed after every node for crash-resume.
Guardrails / Validation
Schema enforcement plus dietary-restriction checks on every output.
Evaluation / Quality
A 3-layer online validate-and-retry cascade, backed by an offline eval harness.
9. Observability
cross-cutting · spans every layerTraces every LLM call across the whole stack.
10. Data / Infrastructure
cross-cutting · spans every layerDeployed on AWS.
Key Features
- Single tool-using agent on a LangGraph StateGraph with a ReAct inner loop and forced tool_choice to terminate the loop
- ChatAnthropic Claude Haiku 3.5 generation (+ Haiku 3.0 judge/reranker, Haiku 4.5 rubric) bound to lookup_usda_batch via bind_tools
- Server-computed USDA nutrition — the submit schema has no macro fields, so the model can't fabricate calories or macros
- 3-layer validate-and-retry: programmatic checks → LLM-as-judge → blocking USDA fact-check (>50% cal / >60% macro deviation), with an oscillation detector
- Two complementary RAGs — lexical USDA grounding (Postgres FTS + ts_rank + Haiku reranker) for exact macros, plus a semantic recipe vector RAG for open-ended dish ideas
- Recipe RAG: a retrieve_recipes node runs HyDE → text-embedding-3-small (1536-dim) → pgvector HNSW + FTS fused with RRF (1/(60+rank)) → Haiku reranker → top-3 dish ideas injected before generate_day, seeded from RecipeNLG (~200k) via a LlamaIndex ingestion pipeline
- Closed-loop data flywheel — validated meals re-ingested with provenance weighting (×0.7), a ≤1/3 generated cap, semantic dedup (cosine >0.95), and a weekly variety audit to prevent mode collapse
- Offline eval harness — golden set, USDA Recall@40, RAGAS/DeepEval/promptfoo + agent-trajectory metrics (force-commit, no-op, redundant-call) logged to MLflow, with a CI regression gate
- Supervisor-workers scaling path via LangGraph Send() map-reduce for 4+ participant households (post-aggregation diversity reconciliation); self-hosted Langfuse traces, prompt caching, Instructor, OpenTelemetry GenAI
Tech Stack
AI Orchestration
RAG & Retrieval
Eval & Observability
Backend
Frontend
Infrastructure
Challenges & Solutions
Two RAGs for Two Problems: Nutrition vs Dish Ideas
Exact ingredient→macro lookup and open-ended dish inspiration are different retrieval problems. Short ingredient names ('chicken breast, raw') are won by lexical search, while conceptual dish queries ('Moroccan chickpea stew' ≈ 'North African lentil soup') are lexically distant but semantically equivalent, so full-text search misses them — one retrieval mechanism can't serve both.
Two complementary RAG systems that don't compete. The factual layer is a lexical USDA RAG — Postgres full-text (tsvector/ts_rank) + a Haiku reranker — grounding every macro. The creative layer is a semantic recipe RAG in a retrieve_recipes node before generate_day: build a query from (cuisine, dietary_tags, calorie_target) → HyDE (one Haiku call writes a hypothetical recipe to close the query↔document embedding gap) → embed with text-embedding-3-small (1536-dim) → hybrid search (pgvector HNSW cosine, metadata-filtered, + FTS) fused with RRF (1/(60+rank)) → top-20 → Haiku reranker → top-3 injected as inspiration only. USDA still grounds every number; the recipe RAG only shapes dish selection.
Preventing Mode Collapse in the Data Flywheel
Re-ingesting the model's own validated meals into the recipe corpus closes a data flywheel, but conditioning future generation on past generations risks an ever-narrowing dish distribution — mode collapse, where the system retrieves and re-generates the same handful of dishes in a tightening loop.
A guarded self-ingestion ETL: only meals that pass all 3 cascade layers with <5% deviation, no oscillation, and no force-commit qualify. Drift guards prevent collapse — provenance weighting (generated recipes ×0.7 at retrieval), a fraction cap (≤1 of 3 injected recipes may be generated), semantic dedup (skip if cosine >0.95), and a weekly variety audit with a provenance audit trail for clean rollback. Feature-flagged until 30-day variety metrics prove stable.
Preventing Hallucinated Nutrition Numbers
LLMs confidently fabricate plausible-but-wrong calorie and macro values, and a meal planner that lies about nutrition is worse than useless.
The LLM's submit schema (DayPlanLLMSubmit) has no nutrition fields, so the model physically can't write a macro number — it must call a USDA lookup tool for ground truth and the server computes every calorie and macro. A hard architectural constraint, not a prompt instruction.
Enforcing Quality on Non-Deterministic Output
A generated day can still miss calorie/macro targets or violate dietary restrictions (keto, vegan, allergen-free), and a single LLM pass can't be trusted.
A 3-layer validate-and-retry cascade — programmatic checks → optional LLM-as-judge (Haiku) → a blocking USDA ground-truth fact-check that retries at >50% calorie or >60% macro deviation — re-prompts with structured feedback injected into the next attempt, with an oscillation detector to bail on stuck days.
Reliable AI Meal Generation
An LLM is non-deterministic, yet a multi-day meal plan needs a reliable, repeatable pipeline with tool use, retries, and streaming — a hand-rolled while-loop gets brittle fast.
Modeled generation as a LangGraph StateGraph: a deterministic day loop (prepare_context → generate_day ⇄ usda_tools → validate_day → emit_day → update_history) wraps a single ReAct inner loop where the LLM decides tool calls, and forced tool_choice cleanly terminates each day.
Crash-Resumable Long-Running Generation
Generating a full multi-day plan is long-running, and a crash or timeout mid-plan would otherwise discard every day already produced.
Typed MealPlanState (TypedDict + reducers) is checkpointed to Postgres after every node via AsyncPostgresSaver, keyed by thread_id, so a half-finished plan resumes from the last completed node instead of restarting.
Real-Time Streaming Across Services
Meal generation takes 10-30s through the AI pipeline. Users need immediate feedback, but the response crosses 3 service boundaries (Python → NestJS → Next.js).
SSE streaming pipeline where the FastAPI ai-service emits Server-Sent Events to NestJS, which proxies them to the Next.js frontend behind a 90s hang guard. Users see meals being generated in real-time.
Right-Sizing Cloud Compute per Workload on AWS
A mostly-cached SEO site, an always-warm authenticated app, and continuous BullMQ/LangGraph workers have opposite cost and latency profiles — no single runtime gives both zero-idle-cost and no cold start.
Split by workload: OpenNext → Lambda + S3 + CloudFront for the scale-to-zero cached SEO site, and ECS Fargate (always-warm) for the authenticated web-agent, NestJS API/workers, and FastAPI ai-service, backed by RDS PostgreSQL and ElastiCache Redis.