Quality Assurance - Assistant Vice President

Key Responsibilities

• AI / Multi-Agent System Testing Design and implement testing strategies for: Multi-agent architectures (cooperative and autonomous agents) Agent workflows involving planning, reasoning
• tool usage Validate: Agent decision-making logic Orchestration across multiple agents and APIs Memory handling (context retention, long-term memory consistency) Test AI agent behavior across: Edge cases, failure scenarios
• fallback mechanisms Multi-step workflows and distributed execution 2.
• LLM & GenAI Testing Validate LLM-powered applications, including: Retrieval-Augmented Generation (RAG) systems Prompt engineering workflows and tuning strategies Perform: Prompt testing and validation Hallucination detection and response quality checks Output accuracy benchmarking aligned with business expectations (aligned with enterprise requirement to define and test output quality for GenAI systems [2026.03.26...s-Overview | PDF] ) Build automated test frameworks for: Response validation Context relevance Semantic correctness 3.
• Test Automation & Framework Development Develop AI-aware automation frameworks: Extend traditional automation (Selenium, API testing) with AI validation layers Automate: Test case generation using AI Regression suites with dynamic test data Integrate with tools such as: qTest, JIRA, CI/CD pipelines Enable intelligent test case reuse and generation (aligned with internal QA AI initiatives for automated test case generation and reuse [GL-Help_AI..._Mar30Demo | PowerPoint] ) 4.
• Data & Pipeline Validation Validate: Data ingestion pipelines Embedding processes and vector databases Retrieval quality from knowledge sources Ensure correctness of: Search results Context injection into prompts Data lineage and integrity 5.
• Enterprise AI Quality & Governance Ensure AI systems meet enterprise requirements: Accuracy, explainability, consistency Auditability and traceability Support: Responsible AI (RAI) compliance Model risk validation and governance reviews Implement monitoring for: Prompt execution traces Agent actions and outcomes Failure patterns and anomalies 6.
• Performance & Reliability Testing Conduct: Load and scalability testing for AI services Latency validation for real-time decision systems Validate: Cost efficiency (LLM token usage optimization) System resilience and fallback mechanisms 7.
• Collaboration & Leadership Work closely with: AI engineers, architects, product teams
• business stakeholders Mentor QA engineers in: AI testing practices Automation frameworks Contribute to: Test strategy definition for enterprise AI programs Architecture reviews and quality governance Required Qualifications: Education Bachelor’s or Master’s in Computer Science, Engineering
• related field Experience 10+ years of QA / SDET experience 2–4 years of experience in AI/ML or GenAI testing Experience in enterprise platforms (preferably banking / financial systems) Technical Skills: AI / GenAI Testing Understanding of: LLMs, RAG architectures, agentic AI Prompt engineering and evaluation Experience validating: Multi-agent workflows AI reasoning systems Test Automation Strong expertise in: Selenium / Playwright / API automation Test frameworks (BDD, Data-Driven) Programming: Java / Python Data & Validation Experience with: SQL and data validation API testing and microservices validation Exposure to: Vector databases and embeddings (preferred) DevOps & Tools CI/CD: GitHub Actions, Jenkins, Harness Containers: Docker, Kubernetes Tools: qTest, JIRA, Postman Preferred Qualifications: Experience testing: AI copilots / enterprise assistants Agent-based automation platforms Familiarity with: Regulatory compliance (RBI, Basel, data governance) Observability tools (Grafana, Prometheus) Key Competencies: Analytical Thinking: Strong ability to identify AI failure patterns Quality Mindset: Focus on reliability, accuracy
• robustness Innovation: Adoption of AI-driven testing approaches Collaboration: Strong stakeholder communication Success Metrics: High-quality AI releases with minimal production issues Improved: Test coverage for AI workflows Accuracy and reliability of AI outputs Reduction in: Manual testing effort via AI-driven automation Faster validation cycles for AI features About State Street Across the globe, institutional investors rely on us to help them manage risk, respond to challenges
• drive performance and profitability.
• We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.
• We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential.
• As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days
• vibrant employee networks that keep you connected to what matters most.
• Join us in shaping the future.
• As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status
• other characteristics protected by applicable law.
• Discover more information on jobs at StateStreet.com/careers Read our CEO Statement