Data Engineer with Apache Flink

Senior Data Platform Engineer — Nexus Data Platform
About the team
What you'll do
Own pipeline slices end-to-end: data ingest into a distributed streaming platform, stream-processing transform jobs, and projection of clean, contract-compliant records into downstream stores.
Onboard new data domains and production entities — legacy schema mapping, streaming topic contracts, and stream-processing job development.
Build and run stream-processing jobs in the cloud — transforms along the change-data-capture → streaming → processing path and batch ingestion via managed big-data services — including checkpointing, failure recovery, and job-level runbooks.
Deploy and troubleshoot streaming and processing workloads on a container orchestration platform — job lifecycle and day-to-day application-level troubleshooting, in partnership with SRE on the underlying cluster.
Partner with downstream teams on data contracts: schema ownership, where consumers read from, and clear boundaries between the ETL pipeline and the teams that serve on top of it.
What we’re looking for
Required
5+ years experience building production grade streaming pipeline systems.
Hands-on experience with a distributed streaming/messaging platform (Kafka + Flink/Spark or similar): topic and partition design, consumer group semantics, compaction, and schema-aware serialization.
Production experience with a distributed stream-processing engine: SQL or DataStream-style jobs, state management, checkpointing, and operational debugging of long-running jobs.
CDC ingestion familiarity — WAL or log-based change capture into a streaming platform.
Demonstrated experience with SQL and schema design across relational sources.
Demonstrated experience deploying and troubleshooting streaming workloads on a container orchestration platform — basic troubleshooting of deployments and running jobs.
Track record shipping observable, testable pipeline components — health checks, smoke gates, benchmark-driven validation.
Ability to work independently on assigned roadmap items while coordinating with cross-functional partners on contract boundaries.
Preferred
Direct, hands-on experience with Apache Kafka and Apache Flink
Familiarity with Debezium CDC capture.
Experience projecting streaming data into an OLAP or analytical serving engine (StarRocks, ClickHouse, Druid, Pinot, or similar).
Awareness of Gov Cloud deployment constraints and data residency considerations.
Experience with multi-tenant SaaS data workloads (tenant partitioning, hot-partition skew).
How we work
AI-assisted development — we use coding agents and automation to move fast; you should be comfortable reviewing and directing AI-generated changes, not writing every line by hand.
Local-first iteration — every engineer has a local stack to develop and test pipeline changes before promoting to shared environments; schema and job work should be fast to try without waiting on central infrastructure.
Prove it before you ship — automated smoke tests and performance benchmarks are part of how we validate changes, not a handoff to someone else at the end.
Cross-team by design — data contracts between the pipeline and downstream teams are explicit; we coordinate early with external teams so downstream consumers stay aligned without slowing delivery.
What success looks like in the first 6 months
At least one production data domain onboarded through Kafka and Flink end-to-end, with automated validation passing in local and shared environments.
A partner team consuming pipeline output under a documented contract — clear access patterns and agreed SLA baselines.
Deploy and integration workflow dependable enough for routine use, not a manual heroics exercise each time.
Contributed runbooks, automation, or reusable patterns that make onboarding the next domain meaningfully faster.

Work arrangement
Yes