Role: Main developer in a team of four senior engineers — designed and built the core platform end to end: the SmartSync ETL and its move to Temporal, the CDC projection into the ClickHouse read model and the PostgreSQL mirror, the Express + Sequelize API, the Next.js frontend on Cloudflare Workers, the Python AI content service, and CI/CD.
Data platform first. SmartSync, the SAM.gov + USAspending.gov ETL, runs on Temporal in production since 2026-09-25 — 32 schedules on 6 task queues, extract → transform → load → verify per window, a 13-key SAM.gov pool with a usage ledger. Completeness is checked against SAM.gov's own files: 0 of 80,922 active notices missing.
CDC and CQRS. PostgreSQL is the write model; a Temporal coordinator per table projects new rows into the ClickHouse read model by watermark, with daily checksum parity over ~94.4M award transactions. A logical-replication PostgreSQL mirror runs as Temporal Workflows with hourly health and daily hash reconciliation.
Serving it fast. Express 4 + Sequelize (TypeScript) API with ~242 endpoints and 78 filters: a headline query 13.6 s → ~1 s, ClickHouse queries 2.2 s → 0.27 s, a single-flight stampede guard on the in-memory response cache, cache TTLs aligned at 10 min, and a ClickHouse → Postgres fallback chain later simplified (32 Postgres MVs dropped, ClickHouse MVs 31 → 14). Running on OVHcloud with Docker and Komodo.
Engineering: Data Platform, AI & Frontend
Main developer on GovChime: the data platform, the AI features, and the frontend performance work, one view at a time.
A Temporal-orchestrated ETL from SAM.gov into PostgreSQL, projected by CDC into a ClickHouse read model and mirrored to a second PostgreSQL — every hop is checked against a second copy of the data.
Orchestrate — Temporal
Temporal is the control plane: Schedules start Workflows, rows never pass through it. Production cut over on 2026-09-25 and the legacy scheduler was deleted.
Extract
Every SAM.gov call runs on one task queue with a queue-wide rate limit and leased API keys.
Transform + Load
One canonical transform per source, then change-aware idempotent upserts into PostgreSQL — a failed window just reruns.
Verify
A green run proves execution. Completeness is proven against a second copy of the source.
A green run proves execution, not completeness — every dataset is checked against a second copy of the source.
0 of 80,922 active notices and 0 of 733,627 FY2025–26 archive notices missing against SAM.gov's own files; PostgreSQL and ClickHouse checksums match on 94.4M award transactions.
Enrich
Follow-up pipelines fill what the source APIs leave out, started by each source run and backed by their own Schedules.
~7.06M contracts · ~1.48M entities · every pipeline writes here
~94.4M award transactions · fed by CDC · analytics reads here
CQRS read model — ClickHouse
PostgreSQL stays the write model; one Temporal coordinator per table projects new rows into ClickHouse, and a daily reconciliation proves parity.
PostgreSQL mirror
A logical-replication mirror of production into a second PostgreSQL, run as Temporal Workflows on its own worker.
Serve
Express + Sequelize API reading the ClickHouse read model, with a ClickHouse → Postgres fallback chain that was later simplified.
Present
Next.js 16 frontend on Cloudflare Workers via OpenNext, with cache freshness matched to the sync cadence.
9. Guards & alerts
cross-cutting · spans every layerScheduled guards watch the pipelines themselves; alerts are stored and shown in the admin UI.
10. Delivery
cross-cutting · spans every layerTests gate every deploy; each push to main deploys SmartSync, then the app.
11. Infrastructure
cross-cutting · spans every layerSelf-hosted compute on OVHcloud, fronted by Cloudflare.
Key Features
- Temporal orchestration: 32 schedules, 6 task queues, 2 production workers; replay tests of production histories in CI
- Per-window extract → staging → transform → change-aware idempotent upsert → verify gates; enrichment pipelines for descriptions, attachments and SBA data
- Completeness vs SAM.gov's own files (0 of 80,922 active notices missing); a 13-key API pool with a usage ledger
- Watermark-based CDC into the ClickHouse read model with daily checksum parity; logical-replication PostgreSQL mirror with hash reconciliation
- Headline query 13.6 s → ~1 s via an indexed slug column; ClickHouse 2.2 s → 0.27 s; single-flight guard on the response cache
- Express 4 + Sequelize (TypeScript) API: ~242 endpoints, 78 filters, iron-session + JWT, Stripe; OVHcloud + Docker + Komodo
Tech Stack
Data Platform / ETL
Database & OLAP
Backend
Infrastructure
Frontend & Edge
AI & Dev Tools
Challenges & Solutions
An ETL That Must Prove It Is Complete
The SAM.gov pipelines ran on a hand-rolled scheduler, and a green run only proved that code executed. The opportunities sync once reported success while holding only 17.5% of notices — API-key quota starvation, plus a completeness check that passed when nothing was proven.
Rebuilt SmartSync on Temporal: 32 schedules on 6 task queues, per-window extract → transform → load → verify, a 13-key SAM.gov pool with a usage ledger, and replay tests of production histories in CI. Completeness is checked against SAM.gov's own files — 0 of 80,922 active notices and 0 of 733,627 FY2025–26 archive notices missing. Production cut over on 2026-09-25 and the legacy scheduler was deleted.
Keeping ClickHouse and a Mirror in Exact Sync
Analytics read from ClickHouse, and a second PostgreSQL needs a live copy of production. Both drift silently if nothing proves they match.
Rewrote the teammate's first PostgreSQL → ClickHouse copy scripts with checkpoints and a parity check and moved them onto Temporal. PostgreSQL is the CQRS write model; a coordinator per table projects new rows into ClickHouse by watermark (query-based CDC), and a daily reconciliation compares counts, distinct ids and checksums — contracts, entities and 94.4M award transactions match. The PostgreSQL mirror uses logical replication (pgoutput; snapshot, then streaming) as Temporal Workflows, with hourly slot-lag checks and a daily count-and-hash reconciliation.
Slow Queries and Stampedes on ~94M Rows
A headline Postgres query took 13.6 s because it applied a slug function to every row of an 89M-row table, which forces a full scan. Separately, concurrent requests for the same heavy ClickHouse count could pile up into several multi-GiB scans at once.
Filtered on a trigger-maintained indexed slug column with a 2-year window instead (13.6 s → ~1 s) and on an agency-code lookup in ClickHouse (2.2 s → 0.27 s). Added a single-flight guard to the API's in-memory response cache, so concurrent callers share one query and a failed refresh serves the last good value. Built the ClickHouse → Postgres fallback chain on the team's ClickHouse read model, then simplified it — 32 Postgres MVs dropped, ClickHouse MVs 31 → 14.
Silent Data Loss in Ingestion
For 10 days the SAM.gov sync reported success while silently dropping ~95–99% of records: SAM.gov changed `totalRecords` from a number to a string, the pipeline read it as null, `null ?? 0` coerced it to 0, and the completeness check `0 >= floor(0 × 0.95)` always passed — no exception thrown, no alert.
A `parseTotalRecords()` helper that accepts both string and number shapes and treats any NaN / 0 / missing value as an API error (triggering key rotation) rather than coercing to 0. The deeper fix was completeness verification comparing expected vs actual counts — `no exception thrown` does not mean the data is correct.
Next.js Caching on Cloudflare Workers
The Next.js 16 frontend runs on Cloudflare Workers via OpenNext with an R2 page cache. With a long-lived regional cache and no tag cache, a published page kept serving the stale copy and on-demand revalidation did nothing; in the worst case feed pages lagged the database by ~3–5 hours.
Switched the regional cache to short-lived mode and added a D1 tag cache, so revalidatePath and revalidateTag take effect after a publish; aligned the React Query, ISR and API cache TTLs at 10 minutes (worst-case feed lag ~10 min by design); and edge-cached the sitemaps with the Cloudflare Cache API.
AI Content Without Invented Figures
AI-generated blog, social and SEO copy about federal contracts must never state numbers the data doesn't support.
A Python AI content service generates the copy from contract data, a guard strips any AI-stated figure not present in the source data, and every piece goes through a human review queue. It moved from the Anthropic API to Cloudflare Workers AI (Llama 4 Scout) in September 2026.