Senior Data Engineer (Pipelines & Architecture)

Remote · Full-time · EU time zone preferred · Fluent English
Who are we?
We are an infrastructure company powering data-driven businesses at scale. We operate a global proxy network, build managed APIs for large-scale web data collection, and develop AI-powered automation tools for collecting and processing data. Our Monetization SDK lets app developers offer users an alternative way to pay: instead of paying money or watching ads, users opt in to share small amounts of unused computing resources. The SDK is used by millions of people across desktop, mobile and smart TV platforms.
What do we do?
At the center of this role is our data platform. The events our products generate become the numbers the business runs on: customer-facing usage views, invoices, dashboards and reports. We are building a new streaming pipeline that will operate at roughly 5 TB of events per day.
Why do we do it?
When invoices and customer usage views are built from the pipeline, a data error is a customer-facing problem. Our goal is a platform that is fast, correct under failure, cost-efficient, and trusted by the people who use its numbers without having to ask the engineer who built it.
How do we do it?
We treat data engineering as production engineering. That means designing for replays, duplicates, out-of-order and late-arriving data. It means tracking schema changes in code, running backfills and reconciliation as repeatable jobs, and monitoring freshness, lag, errors and cost. Our pipeline services are written in Go, and our core storage is ClickHouse and PostgreSQL. We actively use agentic coding tools in everyday engineering work.
Why is there a vacancy?
We are taking a new streaming pipeline from design to production and moving usage aggregation off our legacy database onto it, while invoices, dashboards and reports keep working throughout the migration. We need a senior engineer to own the pipeline as a system: its architecture, storage, aggregation layer and day-to-day operation.
Who are we looking for?
An experienced Senior Data Engineer who has owned business-critical pipelines in production, is comfortable making architectural decisions, and still writes and debugs production code. You’ve been the person accountable for a pipeline that was still correct the morning after it broke.
What professional skills are important to us?
7+ years building and operating production data pipelines
Hands-on ClickHouse experience: engine selection (ReplacingMergeTree and friends), partitioning, materialized views, and query tuning across billions of rows
Strong PostgreSQL: partitioning, indexing, bulk-insert performance, and a clear view of where it stops being the right tool
Deep SQL and analytical data modeling: turning raw events into aggregates other people can trust without asking you first
Active use of agentic coding tools
Strong written communication: you write clearly, escalate early, and make your reasoning visible in an async team
Self-directed and comfortable in a fast-paced startup where the scope is yours to define
Nice to have
Kafka or a comparable streaming platform run in production: partitioning, consumer groups, delivery guarantees
Go: our pipeline services are written in it, and you’d be writing and debugging them
High-volume telemetry or observability data (multi-TB/day ingest)
Vector or comparable collectors (Fluent Bit, OpenTelemetry Collector, Logstash)
Usage-based metering or billing pipelines, including reconciliation and late-data handling
Workflow orchestration for backfills and scheduled jobs (Temporal, Airflow, Dagster)
BI and transformation tooling: Metabase, Grafana, dbt or equivalent
AWS (RDS, Lambda, SQS, S3/Parquet, EC2) and Terraform
Self-hosting and operating ClickHouse or Kafka clusters rather than consuming them as a service
What will you do in the project?
Take the streaming pipeline from design to production, then operate it at roughly 5 TB/day
Move usage aggregation off the legacy database onto the new streaming path, while everything downstream keeps working through the migration
Design the storage layer that decides what the system costs: how events are laid out, partitioned, retained and expired
Make ingestion correct under failure: replays, duplicates, out-of-order arrivals, and late data that shows up after a window has already closed
Own the aggregation layer behind customer-facing usage views and internal reporting: the models, metric definitions and freshness guarantees
Keep performance predictable at both ends of the system, the ingest firehose and the analyst queries, as traffic grows
Automate what is currently manual: schema changes tracked in code, backfills and reconciliation as repeatable jobs, no ad-hoc surgery on production data
Instrument the platform for freshness, lag, ingest errors and cost, with alerts that fire before anyone notices a wrong number on a dashboard
Take part in on-call and incident response for the data platform, and drive the follow-ups that keep the same failure from recurring
Work with backend, proxy infrastructure and product teams so the events we emit answer the questions the business actually asks
What conditions do we offer?
Full-time, fully remote
EU time zone preferred
Unlimited vacation
Health insurance support (different plans for US and international team members)
$500 home-office stipend
$50/month health and wellness budget
$50/month learning budget
Access to the latest AI and technology tools
Compensation: our recruiter will share the range in messages once you apply
What is our hiring process?
Apply via the link https://rpls.cc/96r7Kh
Our recruiter gets back to you in messages with the compensation range, no screening call needed first
Intro call with Recrucial
Interviews with the team

Work arrangement
Yes