Staff Engineer - Core Knowledge Platform

Responsibilities:

• Lead architecture and implementation of the knowledge_base stackInfoNode models, persistent graph representations, semantic index, Cypher DSL, and storage adapters using immutable data models, declarative transformations, and interpreter-style flows.
• Steward the metaclass/pydantic infrastructure: keep the InfoNode registry coherent, codify field annotations (semantic, edge, key roles), and make the schema-definition DSL approachable to other engineers.
• Design and harden ingestion pipelines (schema, stats, query history, BI model inference, column indexing) as deterministic interpreters over warehouse state; ensure they are composable, observable, and easy to extend.
• Advance the semantic index by defining embedding schemas, normalisation strategies, and retrieval DSLs that keep concept search consistent as we evolve vector backends and storage layers.
• Build meta-programming utilities, code generation, InfoNode introspection, and query builders that let other teams extend the graph and DSL safely without touching low-level plumbing.
• Drive reliability/observability for ingestion runs, graph migrations, and index updates; implement diagnostics, rollback strategies, and property-based tests for functional pipelines.
• Partner with interpretation/conversation teams to expose new graph/search capabilities via formal DSL contracts and query APIs, and mentor engineers on FP thinking, DSL design, and testing strategies.
• Improve developer experience: local fixtures, REPL-like tooling for graph queries, schema compilers for InfoNodes, and CI coverage emphasising deterministic behaviour.

Requirements:

• 8+ years building large-scale data or knowledge-graph platforms; expert-level Python or equivalent FP language experience that translates into Pythonic FP practices.
• Demonstrated mastery of functional programming, persistent data structures, and metaprogramming (immutable models, algebraic data types, interpreter pipelines, DSL compilers).
• Experience designing metaclass- or macro-driven schema systems (e. g., custom Pydantic bases, static analysers, IR builders) and making them usable across a codebase.
• Hands-on experience with graph databases/Cypher-like DSLs and ingestion/ELT systems spanning warehouses (Snowflake, Redshift, BigQuery, etc. ) and semantic/ML consumers.
• Practical knowledge of vector search systems (Milvus, Lance, FAISS, pgvector, etc. ) and embedding lifecycle management.
• Product-focused infrastructure mindset: stable APIs, thoughtful abstractions, strong debugging instincts, and a balance of rapid iteration with architectural rigour.

Nice to Have:

• Background in BI tooling, semantic layers, or OLAP modelling.
• Experience connecting LLM/agent systems to deterministic metadata stacks.
• Familiarity with Typer/Rich CLIs, workflow orchestration (Airflow/Prefect), or metadata platforms like DataHub or OpenMetadata.
• Prior work on persistent data-structure libraries, compilers/interpreters, or type-registry tooling with metaclass-heavy foundations.

What Success Looks Like:

• You deliver a resilient, composable knowledge platform where ingestion steps behave like FP interpreters, the InfoNode metamodel stays coherent across thousands of classes, the semantic index feels like an immutable structure regardless of the backing engine, and downstream teams can evolve the DSL confidently while keeping semantic fidelity and freshness high.