Senior Technical Program Manager, Customer Experience and Business Trends
Amazon.com Services LLC•4h ago
United StatesOnsiteFull-timeSenior Level5+ yrs exp
H-1B verified · 2310 LCAs
Top focus
Program ManagerTechnical Program ManagerCustomer Success Manager
- Customer Experience and Business Trends (CXBT) is seeking a Senior Technical Program Manager to join our exciting team building advanced model and agent evaluation capabilities that measure the performance of GenAI (LLMs, multi-modal models, LLM-based agents) and end-customer experience, providing performance benchmarks versus alternatives. This product provides an opportunity to influence how Amazon teams improve customer experiences through comprehensive model assessment and optimization capabilities. Our objective is to create a self-guided application that empowers teams across Amazon to independently evaluate AI/ML models and LLM-powered agents
- measure application-level outcome metrics. This allows teams to make data-driven decisions about model selection and optimization with speed and agility. The ideal candidate will be comfortable in a fast-paced environment and have proven experience planning, executing
- delivering projects across cross-functional teams. The role will work with engineering, product managers, scientists
- legal teams. This position requires balancing competing priorities, negotiating resources, managing technical trade-offs
- delivering measurable business impact. The successful candidate must demonstrate deep technical expertise in data-driven decision making, experience with model evaluation frameworks and methodologies, hands-on experience in LLM-based agent development or LLM/Agent evaluation
- the ability to influence without authority across multiple teams. Key job responsibilities - Work with product managers and software development managers to scope and define roadmap priorities. - Develop and execute program roadmaps across product, engineering
- science teams. - Define success metrics and KPIs, tracking progress towards program goals. - Create cost awareness of both headcount investment and OPEX
- establish tracking and forecasting mechanism for OP1/OP2 input. - Leverage your technical experience of Agents/LLMs to highlight projects risks, look around corners and help the team deliver efficiently. - Anticipate bottlenecks, provide escalation management
- make strategic tradeoffs to optimize time-to-launch. - Interface with senior leadership to provide program updates and strategic recommendations. A day in the life In this role, you'll collaborate daily with engineering teams, applied scientists
- product managers to advance self-service Agent/LLM/Application evaluation capabilities. You'll partner with internal customers across organizations and CXBT teams to understand their study needs and translate them into technical requirements. Your day involves balancing multiple priorities-from coordinating onboarding requests for new models like reasoning LLMs to managing cross-functional integrations. You'll navigate technical trade-offs with engineering/science, negotiate timelines with stakeholders
- ensure evaluation frameworks meet the diverse needs of both technical and non-technical users seeking to optimize their AI solutions. About the team Customer Experience and Business Trends (CXBT) is an organization made up of a diverse suite of functions dedicated to deeply understanding and improving customer experience, globally. We are a team of builders that develop products, services, ideas
- various ways of leveraging data to influence product and service offerings – for almost every business at Amazon – for every customer (e.g., consumers, developers, sellers/brands, employees, investors, streamers, gamers). Our approach is based on determining the customer need, along with problem solving
- we work backwards from there. We use technical and non-technical approaches and stay aware of industry and business trends. We are a global team, made up of a diverse set of profiles, skills
- backgrounds – including: Product Managers, Software Developers, Computer Vision experts, Solutions Architects, Data Scientists, Business Intelligence Engineers, Business Analysts, Risk Managers
- 5+ years of technical product or program management experience - 7+ years of working directly with engineering teams experience - 3+ years of software development experience - 5+ years of technical program management working directly with software engineering teams experience - Experience managing programs across cross functional teams, building processes and coordinating release schedules
- 5+ years of project management disciplines including scope, schedule, budget, quality, along with risk and critical path management experience - Experience managing projects across cross functional teams, building sustainable processes and coordinating release schedules - Experience defining KPI's/SLA's used to drive multi-million dollar businesses and reporting to senior leadership - Experience in developing and deploying LLMs in production on GPUs, Neuron, TPU or other AI acceleration hardware Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability
- other legally protected status. Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner. The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications
- location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off
- parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits . USA, WA, Seattle - 148,700.00 - 201,200.00 USD annually
Required skills
program managementproject managementAIMLLLMsoftware developmentKPISLA