Senior Quality Automation Engineer / SDET

As we accelerate the development of autonomous AI agents and expand across Web, Mobile, API/Backend, and AI/LLM surfaces, product complexity and release frequency are scaling rapidly. To maintain high quality standards without slowing development velocity, we are moving beyond traditional QA toward a modern, engineering-driven quality architecture.

Addressing non-deterministic AI/LLM outputs and microservice consistency requires embedding quality early in the development lifecycle (“Shift-Left”) and building automated testing pipelines. We are seeking a Senior QA Engineer (SET) to drive our test automation architecture, establish quality gates, and foster a culture of quality ownership across our global engineering organization.

Technical Environment

- ENS Development Common Stack

- Programming Languages: C#, Python, TypeScript

- Backend Frameworks: ASP.NET Core, FastAPI

- Frontend Framework: Angular

- Infrastructure: Microsoft Azure

- Databases: SQL Server, Cosmos DB

- Communication: Slack, Google Meet, Jira

- Documentation: Azure Wiki, Confluence

- AI Tools: Claude Code, Devin, Gemini

- QA Specific Tools

- Web E2E Automation: Playwright, Cypress

- API Testing: Postman, REST-assured, or code-based API testing tools

- CI/CD: Azure Pipelines

- Primary Development Languages: TypeScript, C#

Highlights of the Role

- Global Cross-Cultural Team: Work primarily in English alongside cross-cultural engineering teams based in Tokyo and EMEA.

- Architectural Impact: Take full ownership of shaping our test automation foundation, CI/CD integration, and quality governance rather than simply executing test cases.

- Cutting-Edge AI/LLM Testing: Gain rare hands-on experience defining evaluation frameworks and automated regression testing for non-deterministic generative AI features.

Responsibilities

- Lead our quality engineering strategy across Web, Mobile, API/Backend, and AI/LLM features—from architecture design of test automation frameworks to CI/CD pipeline integration.

- Cross-Platform Test Strategy: Define test plans across Web (Frontend), Mobile (iOS/Android), API/Backend microservices, and AI/ML features while leading “Shift-Left” practices from the design phase to identify defects early.

- Test Automation Architecture: Architect, build, and maintain scalable test automation frameworks (Playwright, Cypress, Appium, REST, etc.), optimizing execution speed and eliminating test flakiness.

- CI/CD Quality Gates: Integrate automated test suites into CI/CD pipelines (Azure Pipelines, GitHub Actions, CircleCI, etc.) to automate release verification and support high-frequency deployments.

- AI/LLM Quality Evaluation: Establish methodologies and automated metrics for evaluating LLM outputs, including regression testing for prompt updates, edge-case validation, and safety assessments.

- Exploratory Testing & Triage: Perform advanced exploratory testing to uncover subtle edge cases, drive root-cause analysis, and streamline defect prioritization with engineering teams.

- Technical Leadership: Track and visualize quality metrics (coverage, defect rates, escaped bugs) and partner with engineering leadership to embed quality standards across teams.

Requirements

- 5+ years of software testing experience, including 3+ years architecting test automation frameworks.

- Strong hands-on experience with Web E2E automation using Playwright or Cypress.

- Experience with API testing (Postman, REST, or code-based API testing tools).

- Proven track record of running automated test suites within CI/CD pipelines (GitHub Actions, CircleCI, Jenkins, Azure Pipelines, etc.).

- Business-level English proficiency (primary language for daily technical operations).

- Conversational Japanese proficiency (JLPT N3 equivalent or above).

Nice to haves

While not specifically required, tell us if you have any of the following.

- Experience testing AI/ML or LLM-powered features (including non-deterministic output evaluation).

- Experience with performance, load, or security testing.

- Background in B2B SaaS, multi-sided platforms, or marketplace products.

- Software testing certifications (e.g., ISTQB).

- Practical experience with mobile test automation using Appium (iOS and Android).

- Strong programming proficiency in TypeScript or C#.

- Experience collaborating or negotiating with external stakeholders or vendors.

- Team leadership, management, or vendor management experience.