Senior AI Data Engineer

Global Growth – Global BD Agent Suite:
Build the permission-aware knowledge layer under the Global BD agents. Ingest and keep fresh the Global BD Hub, SharePoint, Salesforce and licensed sources; extract, chunk and PII-scrub content; carry access-control metadata to the search index; and make sure agents only ever retrieve what the requesting user is entitled to see.
Mandatory focus:
Microsoft Graph delta queries and change webhooks; Salesforce CDC / SOQL / Bulk API; Azure Data Factory / Logic Apps and Azure Functions; ADLS Gen2; Azure AI Search indexing with ACL metadata; document extraction and PII scrubbing.
Key Responsibilities:
Build ingestion from SharePoint Online / Global BD Hub using Graph delta queries and change webhooks, and from Salesforce using CDC and SOQL watermark pulls.
Land data in an immutable, source-first, date-partitioned ADLS Gen2 raw zone and maintain lineage.
Extract and chunk PDF, Word, Excel and PowerPoint content, preserving tables, headings and metadata for retrieval.
Propagate permissions by carrying SharePoint / Salesforce ACLs into chunk metadata so Azure AI Search can security-trim results per user.
Scrub PII in pitch records, client bios and contact data in line with GDPR and Purview classification.
Onboard licensed sources such as COMvergence and other third-party data, validating API availability, licensing and usage constraints with source owners.
Monitor freshness quality and failures, and flag stale, duplicate or contradictory content back to the Hub team.
Required Skills & Expected Capability:
Core Data Engineering: Strong Python and SQL; pipeline orchestration with Azure Data Factory/Logic Apps; Azure Functions
M365 / SharePoint Data: Microsoft Graph API, delta queries, change notifications / webhooks, SharePoint lists, libraries and permissions model
Salesforce Data: Salesforce CDC, SOQL, Bulk API, Platform Events
Search Indexing: Azure AI Search indexes, indexers and skillsets; embeddings pipelines; hybrid (vector + metadata) retrieval support
Governance: ACL propagation, PII detection and scrubbing, Purview, data lineage, GDPR awareness
You will fit right in if you have:
5–8 years in data engineering, with 1+ year preparing unstructured content for RAG / LLM use
Has built ingestion from SharePoint / M365 and / or Salesforce in an enterprise tenant
Understands that a wrong permission on a chunk is a data-exposure incident, not a bug

Work arrangement
No