Role: Main developer — owned the platform end-to-end: UI and frontend, the NestJS + FastAPI backend, the LangGraph AI service, data, and CI/CD.
NestJS + Python FastAPI dual-service backend with PostgreSQL, a BullMQ + Redis job queue, and Passport.js JWT auth. NestJS POSTs generation jobs to the FastAPI ai-service, which streams Server-Sent Events back across 3 service boundaries (Python → NestJS → Next.js) behind a 90s hang guard. Stripe handles subscription billing, and the services run on AWS ECS Fargate over RDS PostgreSQL and ElastiCache Redis.
AI Generation Pipeline
One tool-using LangGraph agent, traced end-to-end — the request flows top to bottom through nine layers, wrapped by two cross-cutting rails.
Client / API Gateway
Receives the request, enqueues a job, and POSTs to the Python service.
Interface / Serving
SSE endpoint with a 90s hang guard that streams generation events back.
Orchestration / Control-flow
A LangGraph StateGraph drives the day loop and the validate-retry loop.
Reasoning / Model
The ReAct decision loop — reason, act, observe, repeat.
Tooling / Action
Forced tool use to fetch ground-truth nutrition before committing a day.
The LLM never writes nutrition numbers — the server computes them from USDA ground truth.
A hard architectural constraint: the submit schema has no nutrition fields, so the model cannot fabricate macros — stronger than any prompt instruction.
Knowledge / Retrieval (RAG)
Two complementary RAG systems — exact ingredient macros vs open-ended dish ideas.
Exact ingredient → macro grounding.
Open-ended dish ideas before generation.
two RAGs, two query shapes — exact ingredient macros vs open-ended dish ideas
Validated meals are re-ingested into the recipe corpus via a LlamaIndex pipeline — future generations retrieve from real past outputs, not just the seed corpus.
Drift guards prevent mode collapse: provenance weighting (×0.7), a ≤1-of-3 generated cap, semantic dedup (cosine >0.95), and a weekly variety audit with clean rollback.
Memory / State / Persistence
Typed graph state checkpointed after every node for crash-resume.
Guardrails / Validation
Schema enforcement plus dietary-restriction checks on every output.
Evaluation / Quality
A 3-layer online validate-and-retry cascade, backed by an offline eval harness.
9. Observability
cross-cutting · spans every layerTraces every LLM call across the whole stack.
10. Data / Infrastructure
cross-cutting · spans every layerDeployed on AWS.
Key Features
- NestJS + Python FastAPI dual-service architecture in a pnpm monorepo
- PostgreSQL with a BullMQ + Redis job queue for async generation jobs
- SSE streaming pipeline proxying real-time AI responses across 3 service boundaries (90s hang guard)
- Passport.js JWT authentication with refresh tokens
- Stripe subscription billing with webhook integration
- Deployed on AWS — ECS Fargate (NestJS API + workers, FastAPI ai-service) over RDS PostgreSQL and ElastiCache Redis
Tech Stack
Backend
Infrastructure
AI Orchestration
RAG & Retrieval
Eval & Observability
Frontend
Challenges & Solutions
Real-Time Streaming Across Services
Meal generation takes 10-30s through the AI pipeline. Users need immediate feedback, but the response crosses 3 service boundaries (Python → NestJS → Next.js).
SSE streaming pipeline where the FastAPI ai-service emits Server-Sent Events to NestJS, which proxies them to the Next.js frontend behind a 90s hang guard. Users see meals being generated in real-time.
Crash-Resumable Long-Running Generation
Generating a full multi-day plan is long-running, and a crash or timeout mid-plan would otherwise discard every day already produced.
Typed MealPlanState (TypedDict + reducers) is checkpointed to Postgres after every node via AsyncPostgresSaver, keyed by thread_id, so a half-finished plan resumes from the last completed node instead of restarting.
Right-Sizing Cloud Compute per Workload on AWS
A mostly-cached SEO site, an always-warm authenticated app, and continuous BullMQ/LangGraph workers have opposite cost and latency profiles — no single runtime gives both zero-idle-cost and no cold start.
Split by workload: OpenNext → Lambda + S3 + CloudFront for the scale-to-zero cached SEO site, and ECS Fargate (always-warm) for the authenticated web-agent, NestJS API/workers, and FastAPI ai-service, backed by RDS PostgreSQL and ElastiCache Redis.
Reliable AI Meal Generation
An LLM is non-deterministic, yet a multi-day meal plan needs a reliable, repeatable pipeline with tool use, retries, and streaming — a hand-rolled while-loop gets brittle fast.
Modeled generation as a LangGraph StateGraph: a deterministic day loop (prepare_context → generate_day ⇄ usda_tools → validate_day → emit_day → update_history) wraps a single ReAct inner loop where the LLM decides tool calls, and forced tool_choice cleanly terminates each day.
Preventing Hallucinated Nutrition Numbers
LLMs confidently fabricate plausible-but-wrong calorie and macro values, and a meal planner that lies about nutrition is worse than useless.
The LLM's submit schema (DayPlanLLMSubmit) has no nutrition fields, so the model physically can't write a macro number — it must call a USDA lookup tool for ground truth and the server computes every calorie and macro. A hard architectural constraint, not a prompt instruction.
Enforcing Quality on Non-Deterministic Output
A generated day can still miss calorie/macro targets or violate dietary restrictions (keto, vegan, allergen-free), and a single LLM pass can't be trusted.
A 3-layer validate-and-retry cascade — programmatic checks → optional LLM-as-judge (Haiku) → a blocking USDA ground-truth fact-check that retries at >50% calorie or >60% macro deviation — re-prompts with structured feedback injected into the next attempt, with an oscillation detector to bail on stuck days.
Two RAGs for Two Problems: Nutrition vs Dish Ideas
Exact ingredient→macro lookup and open-ended dish inspiration are different retrieval problems. Short ingredient names ('chicken breast, raw') are won by lexical search, while conceptual dish queries ('Moroccan chickpea stew' ≈ 'North African lentil soup') are lexically distant but semantically equivalent, so full-text search misses them — one retrieval mechanism can't serve both.
Two complementary RAG systems that don't compete. The factual layer is a lexical USDA RAG — Postgres full-text (tsvector/ts_rank) + a Haiku reranker — grounding every macro. The creative layer is a semantic recipe RAG in a retrieve_recipes node before generate_day: build a query from (cuisine, dietary_tags, calorie_target) → HyDE (one Haiku call writes a hypothetical recipe to close the query↔document embedding gap) → embed with text-embedding-3-small (1536-dim) → hybrid search (pgvector HNSW cosine, metadata-filtered, + FTS) fused with RRF (1/(60+rank)) → top-20 → Haiku reranker → top-3 injected as inspiration only. USDA still grounds every number; the recipe RAG only shapes dish selection.
Preventing Mode Collapse in the Data Flywheel
Re-ingesting the model's own validated meals into the recipe corpus closes a data flywheel, but conditioning future generation on past generations risks an ever-narrowing dish distribution — mode collapse, where the system retrieves and re-generates the same handful of dishes in a tightening loop.
A guarded self-ingestion ETL: only meals that pass all 3 cascade layers with <5% deviation, no oscillation, and no force-commit qualify. Drift guards prevent collapse — provenance weighting (generated recipes ×0.7 at retrieval), a fraction cap (≤1 of 3 injected recipes may be generated), semantic dedup (skip if cosine >0.95), and a weekly variety audit with a provenance audit trail for clean rollback. Feature-flagged until 30-day variety metrics prove stable.