QA Tester- Functional and Non-Functional Testing

QA Tester — Functional & Non-Functional Testing
Experience: 7–10 Years
Kynaa AI | Remote (India)
Engagement: Contract / Contract-to-Hire
Joining: Immediate joiner preferred- Maximum Notice Period: 15 Days
About Kynaa AI
Kynaa AI is a voice-driven talent platform. An AI interviewer holds live spoken conversations with candidates and produces AI-scored assessments. The platform is a web application, Python/FastAPI backend services, a live voice pipeline, and an AI interview and scoring layer, with a candidate-facing product and two admin consoles (university and corporate).
Scope — functional testing (does it do what it should)
Smoke testing. A fixed checklist run after every deploy, completed within 30 minutes: candidate login, interview start, voice connect, question flow, session end, score generation, report visible in the admin console.
Sanity testing. Targeted re-check of the changed area after each fix, before full regression.
Regression testing. Full pass over documented user journeys across the candidate product and both admin consoles before every release. Automated in Playwright where the UI is stable, manual where it is in flux. Suite maintained with pass/fail history per release.
End-to-end testing. Complete flows: candidate invited → registers → completes voice interview → scored → report visible to university/corporate admin → exported.
Integration testing. Web app ↔ backend APIs ↔ voice pipeline ↔ LLM and scoring layer ↔ database; email and notification delivery; any payment or invoicing flow.
API testing. Every FastAPI endpoint: request/response contract, status codes, authentication, input validation, error handling, pagination, idempotency. Written in pytest with httpx or equivalent.
Data integrity testing. Scores, transcripts, recordings and candidate records persist correctly in Postgres, verified with SQL. Zero cross-tenant leakage between universities or corporates.
Role and permission testing. Candidate, university admin, corporate admin, Kynaa internal — every screen and endpoint tested for both allowed and denied access.
Voice pipeline testing. Microphone permission, connect and reconnect, interruption (barge-in), silence handling, background noise, accent variation, session end, transcript accuracy spot checks.
AI behaviour testing. Run the Python evaluation harnesses before any prompt, model or scoring change ships. Review a fixed weekly sample of real transcripts for drift: off-script questions, invented facts, inconsistent scoring, tone. Re-run the same transcript through scoring to check consistency. Convert every escaped defect into a new eval fixture.
Exploratory testing. Time-boxed sessions with a written charter and logged findings.
Scope — non-functional testing (does it hold up)
Performance. Voice turn latency (candidate stops speaking → AI starts speaking), page load and API response times, reported as p50/p95 per release against targets agreed with engineering [e.g. p95 turn latency under 1.5 s].
Load and stress. Concurrent live interviews at [50 / 200 / 500]; identify degradation points and failure modes. k6 or Locust.
Soak. Sustained sessions over several hours; memory growth, connection drops, cost drift.
Reliability and recovery. Network drop mid-interview, tab refresh, browser close, backend restart — what the candidate sees, what is saved, whether the session resumes.
Security. OWASP Top 10 basics: session handling, IDOR across tenants, injection, file upload validation, rate limiting, secrets exposed in client code. Coordinate any external penetration test.
Data privacy. PII handling, consent capture, deletion requests, retention, access control on recordings and transcripts.
Compatibility. Chrome, Edge, Safari, Firefox (latest two versions); Windows, macOS; headset and built-in mic; throttled bandwidth.
Accessibility. WCAG 2.1 AA on core flows: keyboard navigation, screen reader, contrast.
Usability. Daily live voice sessions with the product. Judge pacing, interruptions and naturalness, reported with specific timestamps and examples.
Cost observability. AI inference cost per interview tracked per release; anomalies such as runaway token usage or retry loops flagged.
Responsibilities
Design and build the browser automation suite from scratch (Playwright, TypeScript or Python) and wire it into the release pipeline so it runs on every pull request.
Own the test plan, test cases and a traceability matrix from requirement to test to result.
Write bug reports a developer can act on without calling you: title, environment, build, steps, expected vs actual, evidence (video, HAR, logs), severity and priority.
Verify changes before merge, run post-deploy verification and give release sign-off.
Take live voice sessions with the product every day.
Decide what gets tested, how, and to what depth.
Definitions used in this role
Severity. S1 blocker: interview cannot start or complete, data loss, cross-tenant leak. S2 major: a core journey is broken with a workaround. S3 minor: non-core defect. S4 cosmetic.
Release sign-off. No open S1 or S2, regression suite at 100% pass or each failure explicitly accepted, eval harness thresholds met, smoke checklist passed on production.
Qualifications
7–10 years in QA, SDET or test automation roles, covering both functional and non-functional testing.
Automation code you wrote yourself, in a suite that ran in production.
Test code or tooling written in Python; TypeScript a plus.
Browser automation with Playwright, Selenium or Cypress.
API testing against FastAPI or similar; working SQL.
Performance or load testing with k6, Locust or JMeter.
Experience testing LLM-based products, prompt evaluation or non-deterministic systems.
Experience as the first or only QA hire on a team.
Interested candidates can share their resume at recruitment@namasys.ai

Work arrangement
No