Design a Personalized News Feed Ranking System

Problem and Scope

Design the machine-learning and serving system that ranks a personalized social news feed. For each request, the system must retrieve eligible posts, score and order them using user, content, and context signals, enforce freshness and diversity constraints, and return the feed within a strict latency budget. Explain how labels and objectives are defined, how features and training data are produced, how models are evaluated offline and online, how feedback is incorporated safely, and how the serving path behaves during partial failures or model regressions.

Start by confirming the core user journey, exclusions, success criteria, and the constraints that materially affect the architecture.

Requirements to Clarify

A strong answer should establish scope before choosing components.

Functional requirements:

• Generate a personalized ranked feed from eligible posts for each active user.
• Incorporate explicit and implicit feedback while controlling freshness, diversity, and repeated content.
• Support new users, new posts, and rapidly changing interests with sensible fallback behavior.
• Train, evaluate, deploy, and roll back ranking models without interrupting feed serving.

Non-functional requirements:

• Serve the first page within a few hundred milliseconds at global social-network scale.
• Keep online features sufficiently fresh while preventing training-serving skew and leakage.
• Degrade safely when feature stores, candidate sources, or model serving are unavailable.
• Protect user privacy and make harmful feedback loops observable and controllable.

Scale and Capacity

Use the workload to justify storage, partitioning, caching, and reliability decisions. Clarify or challenge these assumptions rather than treating them as unquestionable facts:

• Support hundreds of millions of daily active users and billions of candidate posts per day.
• Each request may begin with thousands of candidates but can afford expensive scoring for only a much smaller set.
• Behavioral signals arrive continuously and range from strong explicit feedback to noisy implicit engagement.

Architecture Discussion

Walk through the important read and write paths, identify ownership boundaries, and explain how the design behaves when dependencies fail. Cover these areas explicitly:

• Objective definition, labels, guardrail metrics, and bias in engagement signals
• Candidate generation, filtering, multi-stage ranking, and diversity constraints
• Offline feature pipelines, online feature freshness, and training-serving consistency
• Model training, validation, deployment, experimentation, and rollback
• Cold start, exploration, feedback loops, drift, and abuse resistance
• Latency budgets, caching, observability, and graceful degradation

Follow-up Questions

Expect the interviewer to test the consequences of your choices. Practise answering these questions with a concrete decision, its benefit, and its cost:

• Which user actions become labels, and how do you prevent clickbait from winning the objective?
  • Focus: Test objective design, delayed outcomes, negative signals, and guardrail metrics.
• How do you reduce millions of eligible posts to a few hundred candidates before expensive ranking?
  • Focus: Test retrieval architecture, source blending, precomputation, and recall trade-offs.
• What is your offline evaluation plan before exposing a new model to users?
  • Focus: Test temporal splits, leakage prevention, ranking metrics, calibration, and slice analysis.
• How do you detect and respond when online engagement rises but user satisfaction falls?
  • Focus: Test experimentation, guardrails, causal interpretation, and rollback discipline.
• The online feature store is stale for one region. What should feed serving do?
  • Focus: Test freshness contracts, fallback models, feature defaults, and safe degradation.
• How would you serve a useful feed to a brand-new user?
  • Focus: Test cold-start strategies, exploration, contextual priors, and rapid adaptation.

Evaluation Rubric

MockMe evaluates the answer across the following dimensions. A complete answer should connect claims to requirements and explain consequential trade-offs.

• Requirements and scope (15%): Clarifies and prioritizes the required behavior for Generate a personalized ranked feed from eligible posts for each active user; Incorporate explicit and implicit feedback while controlling freshness, diversity, and repeated content; Support new users, new posts, and rapidly changing interests with sensible fallback behavior. Establishes the constraints that materially affect Design a Personalized News Feed Ranking System, including Serve the first page within a few hundred milliseconds at global social-network scale; Keep online features sufficiently fresh while preventing training-serving skew and leakage. Strong evidence includes Separates the critical path from secondary features and resolves ambiguous requirements before choosing components.
• Architecture and interfaces (20%): Presents coherent ownership boundaries and end-to-end flows covering Objective definition, labels, guardrail metrics, and bias in engagement signals; Candidate generation, filtering, multi-stage ranking, and diversity constraints; Offline feature pipelines, online feature freshness, and training-serving consistency; Model training, validation, deployment. Strong evidence includes Defines interfaces and traces important success, retry, and failure paths across the proposed components.
• Data and scaling (25%): Uses workload assumptions such as Support hundreds of millions of daily active users and billions of candidate posts per day; Each request may begin with thousands of candidates but can afford expensive scoring for only a much smaller set to justify capacity and partitioning decisions. Explains the data, state, or model strategy for Candidate generation, filtering, multi-stage ranking, and diversity constraints; Offline feature pipelines, online feature freshness, and training-serving consistency; Model training, validation, deployment, experimentation, and rollback. Strong evidence includes Quantifies a dominant workload, identifies the first bottleneck, and explains how the design evolves as that workload grows.
• Reliability, correctness, and safety (20%): Explains concrete failure behavior, recovery, and operational safeguards for Offline feature pipelines, online feature freshness, and training-serving consistency; Cold start, exploration, feedback loops, drift, and abuse resistance; Degrade safely when feature stores, candidate sources, or model serving are unavailable. Strong evidence includes States the required correctness or consistency boundary and covers retries, partial failure, observability, and safe degradation.
• Communication and trade-offs (20%): Drives a structured discussion and compares consequential alternatives for Model training, validation, deployment, experimentation, and rollback; Cold start, exploration, feedback loops, drift, and abuse resistance; Latency budgets, caching, observability, and graceful degradation. Strong evidence includes Makes assumptions explicit, answers the question asked, and explains both the benefit and cost of major decisions.