Senior Platform Engineer

Responsibilities:
Own accelerator's 6-n stage processing pipeline end-to-end: Ingestion to output.
Build the connectors, PII/PHI safety layer, compliance audit trail, and multi-tenant isolation.
Pure platform engineering, async Python, event-driven architecture, and enterprise-scale reliability.
Design the core pipeline orchestration framework with async Python, supporting parallel/sequential stage execution with retry and dead-letter queues.
Build the connector framework for 50+ enterprise data sources (APIs, databases, file systems, SaaS platforms).
Implement safety validation, PII/PHI detection, data integrity checks, format compliance, and input sanitization.
Build the reconstruction module transforming extracted data into Markdown/vector representations and context maps.
Design event-driven architecture with message queuing, backpressure handling, and graceful degradation.
Implement immutable audit trail logging across all 6 stages for every decision, action, and transformation.
Build multi-tenant data isolation, ensuring strict customer separation at every pipeline stage.
Requirements:
Experience: 4-8 years | 2+ years building data-intensive pipelines.
Must-Have Technical Skills:
Distributed Python systems: asyncio, aiohttp, and Celery; not scripting.
Message queues/streaming Kafka, SQS, RabbitMQ, or Redis Streams.
High-volume data pipelines: 100K+ documents/records per batch.
API design: REST/GraphQL, versioning, rate limiting, and auth.
AWS services: Lambda, SQS/SNS, MSK, EKS, Step Functions, ECS/EKS, S3 DynamoDB, and RDS.
Multi-tenant SaaS architecture and customer data isolation patterns.
Production habits: comprehensive testing, CI/CD, monitoring, and incident response.
Core Tech Stack: Python, FastAPI, Celery, asyncio, Kafka/SQS/Redis Streams, PostgreSQL, DynamoDB, Redis, Elasticsearch, Terraform/CDK, Prometheus/Grafana/OpenTelemetry, GitHub Actions, Docker, and AWS (Lambda, ECS, Step Functions, S3).

Experience
6-9 yrs