Open assignment

About the team

The Recommendations team builds and operates the systems that decide what hundreds of millions of users see, surfaced in real time and tuned continuously. We run production retrieval and ranking pipelines, contextual bandit-driven decisioning, and large-scale experimentation. A recommendation is only as good as our ability to prove it moved the right metric - and that's where we need help.

We're require a data analyst to strengthen the measurement and analysis layer around our recommendation systems, working alongside engineering product.

Scope of the engagement

• Own the measurement of recommendation experiments - design A/B tests, define success metrics, and deliver readouts that explain not just whether something worked but why .
• Investigate model and system behavior: ranking quality, retrieval coverage, exposure and feedback effects, and the gaps between offline metrics and online outcomes.
• Build the dashboards, data models, and self-serve tooling the team relies on to monitor recommendation health, and leave them in a state the team can maintain after the engagement.
• Partner with engineers and PMs to scope analytical questions sharply - clarifying the objective and sizing the opportunity before they turn into long projects.
• Surface insight proactively: the segment behaving oddly, the metric drifting, the counterfactual worth running.

We care about the deliverables and the handover. Work should be reproducible and documented well enough that it outlives the contract.

What you bring

• 5+ years in an analytics or data science role, with a track record of operating independently and delivering with minimal ramp-up.
• Direct experience with recommendation, search, ranking, ads, or another large-scale ML-driven product.
• Strong SQL and comfort with large datasets (we work in BigQuery); fluency in Python for analysis beyond SQL.
• A real grasp of experimentation - A/B testing, statistical significance, and the common traps (peeking, interference, novelty effects).
• The instinct to clarify the objective before reaching for a solution, and the discipline to separate what the data shows from what you'd like it to show.
• Clear communication: you can take a messy analysis and land a single, defensible takeaway for any audience, from an engineer to a director.
• A very good understanding and application of advanced statistics

Nice to have

• Familiarity with recommender systems internals - retrieval, ranking, candidate generation, offline vs. online evaluation.
• Exposure to bandits, off-policy evaluation, or causal inference.
• Experience building data models and dashboards others depend on (dbt, Looker, or similar).

This assignment is managed by FILL that has the exclusive end client relationship. Consultants presented to FILL through Commended will be priced at their indicated rate +4%.