AI Quality Analyst（Spanish）

Role Summary
Microsoft's AI products serve hundreds of millions of users across languages and markets. We need people who can define what a great AI response looks like in German, not just label whether one is correct.
As a Language Evaluation Specialist, you will be the quality authority for Spanish. You'll start by evaluating AI outputs hands-on, then grow into shaping the evaluation standards themselves — working directly with product and engineering teams to close the gap between what our AI produces and what local users actually need.
Key Responsibilities
Define quality for your language market. Evaluate AI-generated content against real user needs — not just surface-level correctness, but whether the response truly serves someone working in that language and culture.
Turn observations into standards. Identify patterns across evaluations, extract reusable principles, and collaborate with PMs to build evaluation guidelines that scale beyond your own judgment.
Evaluate cases with local user needs at the center. Score, review, and quality-check AI-generated cases in your target language — grounded in the linguistic nuances, cultural context, and real user expectations of your market.
Collaborate with LLMs to drive quality at scale. Work alongside AI-assisted evaluation pipelines — reviewing model outputs, identifying where automated judgments fall short, and helping build feedback loops that continuously improve both human and model evaluation quality.
Required Qualifications
Native-level fluency in Spanish(reading and writing), with deep understanding of local culture, communication norms, and user expectations.
Independent judgment with strong logical and analytical thinking: able to form your own view of what makes an AI response good or bad, articulate why, and abstract from concrete cases to generalizable evaluation criteria.
Self-motivated with a quality-first mindset. Able to work independently, follow established guidelines with discipline, and stay engaged with detail-heavy work. Passion for AI products and how they can be improved is a strong plus.
Professional working proficiency in English (written and verbal), able to collaborate effectively in a global, cross-timezone team environment.
Preferred Qualifications
Experience in linguistic QA, localization QA, search relevance, or human evaluation programs — especially roles involving evaluation criteria design or refinement, not just execution.
Hands-on familiarity with AI/LLM products (ChatGPT, Copilot, Gemini, etc.) as a regular user, with a sense of where they perform well and where they fall short.
Understanding of how professionals in the target market work — their tools, workflows, and communication habits — so you can evaluate AI outputs from the perspective of the people who actually use them.
Experience working in cross-cultural, cross-timezone team environments.

Work arrangement
Yes