The Manager, Experimentation leads NBCU’s incrementality testing program for Performance and Device media in partnership with the Sr. Manager of Media Experimentation. The ideal candidate is the methods & QA owner: you will shape the learning agenda, enforce power standards, ensure study designs are decision-grade, and translate vendor results into clear budget guidance—reconciled with Marketing Mix Modeling (MMM) and Multi-Touch Attribution (MTA) so leaders can act with confidence and speed. This person will be a key cross-functional leader with the Performance and Device Media teams, helping to inform crucial company decisions on marketing investment and allocation. 

Key responsibilities could include, but are not limited to:

• Co-own the experimentation roadmap with the Sr. Manager of Media Experimentation; define learning agenda, decision questions, hypotheses, success metrics, guardrails, and power thresholds; review/approve vendor designs and reject under-powered or invalid proposals.  
• Lead day-to-day relationship with external measurement vendor— partner with Sr. Manager to prioritize the study backlog, deliver accurate data, align on methods, track SLAs and change logs, escalate blockers 
• Collaborate with Sr. Manager of Media Experimentation on ensuring clear schedules, accurate data inputs, agency and/or vendor alignment, and structured documentation of results 
• Partner with the vendor and a Media Analytics analyst to produce clear, defensible readouts that inform decisions (lift, uncertainty, risks); write concise decision memos and hand off to Optimization and & Insights team for execution and pacing/optimization.  
• Drive calibration with MMM/MTA; propose tiebreakers and thresholds for decisioning; contribute to cross-channel measurement efforts by triangulating learnings.  
• Ensure high quality of analysis with speed; Monitor design defects/leakage; run targeted validations (SQL/R/Python spot checks) as needed; improve run-rate without sacrificing validity. 
• Mentor a Media Analytics analyst; uplevel partner teams (brief quality, power, common pitfalls); collaborate with data science and other analytics & media teams; contribute to a standardized measurement approach across the NBCU Media Group.