About the role
Responsibilities:
- Architecture execution. The core positions are settled: models reason over summaries, samples, and clustered exceptions and emit specifications; deterministic code executes those specifications over the full dataset. A durable workflow engine owns the long-lived process; agent orchestration stays inside a single step and holds no state that must survive a restart. You will hold those boundaries as the system grows and argue with us where you think we are wrong.
- Polyglot workers. TypeScript workflow definitions alongside the NestJS service layer, Python data, and agent activities on a separate task queue. Keeping workflow code deterministic and bulk data out of it is your line to defend.
- The data engine. Polars and DuckDB over Parquet, streaming. No Spark, no Databricks; this fits in one container and scales by queue depth and horizontal replicas. Making that hold at millions of rows is a real engineering problem.
- Evaluation. The system's output is not obviously wrong when it is wrong, and there is no human downstream to catch it. A regression harness over historical input/output pairs is what keeps us honest, and it is load-bearing infrastructure rather than a testing nicety. You will build it early.
- The team. Hiring alongside you, code review, setting the bar for what gets merged, and growing a junior engineer into someone senior.
- Production. Development in our own environment, production inside customer cloud tenants. Containerized, portable, and yours to keep running.
Requirements:
- 5-7 years building backend or data-intensive systems, with at least one system you took from design to production and then lived with afterwards.
- Deep TypeScript or deep Python, and real working competence in the other. This is a genuinely polyglot codebase.
- You have designed a distributed system with long-running state and understand why that is hard: idempotency, retries, and exactly-once as a fiction, and what happens when a process dies mid-flight on day four.
- Serious data engineering instincts. You have hit the wall where the obvious approach stops working at volume, and you know what you did next.
- Experience leading engineers: reviewing their work, unblocking them, and telling them when something is not good enough without making it personal.
- Judgment about correctness. You are the kind of engineer who asks "how would we know if this were silently wrong? " before shipping.
Strongly Relevant:
- Temporal, or any durable-execution or workflow-orchestration engine in production.
- LLM systems where accuracy was measurable and retrieval, structured extraction, and agentic pipelines with evaluation were behind them.
- Columnar and out-of-core processing: Polars, DuckDB, Arrow, Parquet.
- Azure or cloud work inside an enterprise tenant with real security and data-residency constraints.