Sr. Manager, Experimentation

Job Description

Sr. Manager, Experimentation (Technical)

Job Description
Data is part of the DNA at NBCUniversal, and the Experimentation team plays a large role in driving our data driven culture. We own the technology that enables experimentation and act as subject matter experts to help teams measure the performance of the features and campaigns that customers experience every day. Our work is high visibility as it directly informs product, advertisement, marketing, and merchandising decisions.
The Sr. Manager, Experimentation (Technical) will own the statistical frameworks and processes that power our analytics and operational solutions to support self-service insights and decisioning within the organization. The team's output enables automated dashboards and exploratory analysis that will be used by Product, Marketing, Engineering and Merchandising stakeholders to monitor performance and generate insights from experiments. This role will manage a team and work closely with other Data and Decision Science partners to define best practices and continuously improve statistical methods of how we analyze and share experiment results.
Responsibilities include, but are not limited to:
• Develop statistical frameworks and processes designed to analyze experiments to enable fast, reliable and accurate decision-making for product, marketing, ads and marketing teams
• Enhance experiment analysis methodologies, including Bayesian and frequentist approaches, variance reduction techniques, and sequential testing
• Collaborate with engineering to integrate statistical algorithms and new metrics into our experimentation platform
• Develop automated solutions for experiment diagnostics, anomaly detection, and guardrail monitoring
• Stay ahead of industry advancements, improving our statistical engine with the latest methodologies and best practices
• Communicate statistical concepts to stakeholders, including product managers, analysts, and engineering teams
Qualifications
• 6+ years of applied experience in experimentation at an ecommerce, social network, direct to consumer, media entertainment or similar tech company
• 3+ years of experience leading a technical team and developing talent
• BA/BS in technical or quantitative disciple or significant relevant work experience
• Direct experience working with 1st and/or 3rd party experimentation platforms (i.e. in-house, Optimizely, Eppo, Stat-sig, or Similar)
• Expert knowledge of experimental design, statistics and other analytical concepts (i.e., RCTs, Frequentist and Bayesian analysis)
• Advanced coding skills to implement and automate computational methods and algorithms, related but not limited to causal inference and experimentation (Python, R, SQL, etc.)

Desired Characteristics
• Interest in inferring experimentation led changes in human behavior and business impact of these changes with the help of big data
• A great storyteller and communicator that can use data to help influence product decisions and change opinions
• Combination of technical skill sets and business savvy to interface with all levels of the organization
• Past relevant experience in computational programming in either industry or academia
• Ability to work independently, multitask, and drive your own projects
• A passion for TV and movies and defining the future of streaming entertainment
Additional Requirements:
Hybrid: This position has been designated as hybrid, generally contributing from the office a minimum of three days per week.
This position is eligible for company sponsored benefits, including medical, dental and vision insurance, 401(k), paid leave, tuition reimbursement, and a variety of other discounts and perks. Learn more about the benefits offered by NBCUniversal by visiting the Benefits page of the Careers website.
Salary range: $130,000 - $180,000 (bonus and long-term incentive eligible)

Jobcode: Reference SBJ-wp4pkp-18-220-207-231-42 in your application.