Data Engineer

We are looking for a Data Engineer to help build and scale PixieDust's data platform and enrichment pipelines. This role focuses on building reliable data pipelines, integrating external data sources, and supporting AI-driven decisioning systems. You will work closely with AI engineers, product teams, and analytics teams to ensure high-quality data flows that power PixieDust's lending workflows. This role is ideal for someone who enjoys working with large datasets, APIs, and real-world financial data systems.
The core responsibilities for the job include the following:
Data Pipeline Development:
Build and maintain scalable data pipelines for ingestion, transformation, and enrichment.
Develop ETL/ELT processes to process large volumes of business and financial data.
Ensure data pipelines are reliable, efficient, and production-ready.
Data Integration:
Integrate with external data providers such as financial data platforms, credit bureaus, banking APIs, and enrichment services.
Build connectors for APIs and third-party data platforms.
Ensure consistent and normalized data across multiple sources.
Data Architecture:
Design and manage data models and data storage systems.
Maintain data warehouses and operational data stores.
Optimize database performance and query efficiency.
AI and Analytics Data Support:
Prepare and structure data for AI models and decisioning systems.
Support retrieval pipelines used by AI workflows and conversational agents.
Work closely with AI engineers to enable data-driven product features.
Data Quality and Monitoring:
Implement data validation, monitoring, and error handling.
Ensure high data integrity and consistency across systems.
Build monitoring tools and dashboards for data pipelines.
Requirements:
3-5 years of experience as a Data Engineer or Backend/Data Platform Engineer.
Strong experience with Python.
Experience building ETL/ELT pipelines.
Experience working with SQL databases (PostgreSQL, MySQL, etc. ).
Experience with data processing frameworks or workflow orchestration tools.
Experience integrating external APIs and data services.
Familiarity with cloud environments (AWS, GCP, Azure).
Preferred Qualifications:
Experience working with large-scale data pipelines.
Experience with data enrichment or external data providers.
Familiarity with vector databases or AI data pipelines.
Experience working with financial, credit, or lending data.
Experience with stream processing or real-time data systems.
Exposure to data warehousing platforms.
Technology Stack:
Engineers at PixieDust work with modern data infrastructure, including:
Languages: Python, SQL.
Data Processing: ETL pipelines, workflow orchestration tools.
Databases: PostgreSQL, Redis, and data warehouses.
Infrastructure: AWS / GCP, Docker.
Integrations: Financial data providers, banking APIs, enrichment platforms.
AI Data Systems: Vector databases and retrieval pipelines.

Experience
2-6 yrs