Document Processing Pipeline Design

Design a scalable pipeline to ingest, extract, classify, and store structured data from documents using OCR/NLP.

Design a scalable pipeline to ingest, extract, classify, and store structured data from documents using OCR/NLP. Use this guide to structure the discussion, test the design under pressure, and practise explaining trade-offs clearly.

Problem and Scope

Design a document processing pipeline that ingests documents (PDFs, images, Office files), extracts text and structured fields using OCR and NLP, classifies documents, and saves structured data for search and downstream use. The pipeline should handle high volume, support accuracy monitoring and human-in-the-loop correction, and be scalable and secure.

Start by confirming the core user journey, exclusions, success criteria, and the constraints that materially affect the architecture.

Requirements to Clarify

A strong answer should establish scope before choosing components.

Functional requirements:

Ingest documents from multiple sources (upload, email, SFTP, connectors).
Detect document type and language.
Extract raw text (OCR for scans, native text extraction for digital files).
Classify documents and extract structured fields/entities.
Provide storage for raw docs, extracted text, structured records, and annotations.
Provide APIs for ingestion, retrieval, search, and human review.
Support human corrections and feedback loop to improve models.

Non-functional requirements:

Scalable to tens or hundreds of thousands of documents/day.
Reasonable latency options (real-time for small docs, batch for large volumes).
High availability, secure storage, and auditability.
Monitoring, alerting, and model performance tracking.

Scale and Capacity

Use the workload to justify storage, partitioning, caching, and reliability decisions. Clarify or challenge these assumptions rather than treating them as unquestionable facts:

Baseline: 100,000 documents/day (~1.2 docs/sec average).
Peak: 5,000 documents/minute (~83 docs/sec) for short bursts.
Average doc size: 0.5–2 MB; storage ingest ~50–200 GB/day.
Extracted text size typically 1–10% of binary size.
Metadata and structured records: ~1–5 KB/document => ~0.5 GB/day.

Architecture Discussion

Walk through the important read and write paths, identify ownership boundaries, and explain how the design behaves when dependencies fail. Cover these areas explicitly:

Idempotent ingestion, document identity, malware screening, and backpressure
Format detection, native extraction, OCR, classification, and field extraction stages
Workflow orchestration, retries, dead-letter handling, and historical reprocessing
Raw document, derived artifact, metadata, lineage, and search storage models
Accuracy measurement, confidence thresholds, human review, and correction feedback
Tenant isolation, PII handling, encryption, auditability, and retention

Follow-up Questions

Expect the interviewer to test the consequences of your choices. Practise answering these questions with a concrete decision, its benefit, and its cost:

What types of documents? (scanned images, native PDFs, Word, email)
- Focus: Assess the candidate's answer to "What types of documents? (scanned images, native PDFs, Word, email)" for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Which languages and scripts must be supported?
- Focus: Assess the candidate's answer to "Which languages and scripts must be supported" for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Expected volume: documents/day and peak ingestion rate?
- Focus: Assess the candidate's answer to "Expected volume: documents/day and peak ingestion rate" for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Latency requirements: real-time (<5s), near real-time (<minutes), or batch?
- Focus: Assess the candidate's answer to "Latency requirements: real-time (<5s), near real-time (<minutes), or batch" for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Accuracy targets for OCR and extraction, and SLA for human review?
- Focus: Assess the candidate's answer to "Accuracy targets for OCR and extraction, and SLA for human review" for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Do we need PII redaction, encryption at rest/in transit, audit logs?
- Focus: Assess the candidate's answer to "Do we need PII redaction, encryption at rest/in transit, audit logs" for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
What downstream uses: search, analytics, indexing, RPA, databases?
- Focus: Assess the candidate's answer to "What downstream uses: search, analytics, indexing, RPA, databases" for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Are there existing storage or tooling constraints (S3, Azure Blob, Databricks)?
- Focus: Assess the candidate's answer to "Are there existing storage or tooling constraints (S3, Azure Blob, Databricks)" for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Support handwriting recognition and multi-language OCR.
- Focus: Assess the candidate's answer to "Support handwriting recognition and multi-language OCR." for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Add entity linking and canonicalization to external knowledge bases.
- Focus: Assess the candidate's answer to "Add entity linking and canonicalization to external knowledge bases." for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Implement active learning: surface uncertain examples to human reviewers and incorporate into training pipeline.
- Focus: Assess the candidate's answer to "Implement active learning: surface uncertain examples to human reviewers and incorporate into training pipeline." for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Add streaming analytics (near real-time dashboards) on extracted entities.
- Focus: Assess the candidate's answer to "Add streaming analytics (near real-time dashboards) on extracted entities." for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Privacy features: automated PII detection and redaction, differential privacy for analytics.
- Focus: Assess the candidate's answer to "Privacy features: automated PII detection and redaction, differential privacy for analytics." for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.
Offline/batch reprocessing: ability to re-run improved models over historical data (use Databricks jobs + Delta Lake snapshots).
- Focus: Assess the candidate's answer to "Offline/batch reprocessing: ability to re-run improved models over historical data (use Databricks jobs + Delta Lake snapshots)." for explicit assumptions, a workable mechanism, failure behavior, and consequential trade-offs.

Evaluation Rubric

MockMe evaluates the answer across the following dimensions. A complete answer should connect claims to requirements and explain consequential trade-offs.

Requirements and scope (15%): Clarifies and prioritizes the required behavior for Ingest documents from multiple sources (upload, email, SFTP, connectors); Detect document type and language; Extract raw text (OCR for scans, native text extraction for digital files). Establishes the constraints that materially affect Document Processing Pipeline Design, including Scalable to tens or hundreds of thousands of documents/day; Reasonable latency options (real-time for small docs, batch for large volumes). Strong evidence includes Separates the critical path from secondary features and resolves ambiguous requirements before choosing components.
Architecture and interfaces (20%): Presents coherent ownership boundaries and end-to-end flows covering Idempotent ingestion, document identity, malware screening, and backpressure; Format detection, native extraction, OCR, classification, and field extraction stages; Workflow orchestration, retries, dead-letter handling, and historical reprocessing; Raw document, derived artifact, metadata. Strong evidence includes Defines interfaces and traces important success, retry, and failure paths across the proposed components.
Data and scaling (25%): Uses workload assumptions such as Baseline: 100,000 documents/day (~1.2 docs/sec average); Peak: 5,000 documents/minute (~83 docs/sec) for short bursts to justify capacity and partitioning decisions. Explains the data, state, or model strategy for Raw document, derived artifact, metadata, lineage, and search storage models. Strong evidence includes Quantifies a dominant workload, identifies the first bottleneck, and explains how the design evolves as that workload grows.
Reliability, correctness, and safety (20%): Explains concrete failure behavior, recovery, and operational safeguards for Idempotent ingestion, document identity, malware screening, and backpressure; Accuracy measurement, confidence thresholds, human review, and correction feedback; Tenant isolation, PII handling, encryption, auditability, and retention. Strong evidence includes States the required correctness or consistency boundary and covers retries, partial failure, observability, and safe degradation.
Communication and trade-offs (20%): Drives a structured discussion and compares consequential alternatives for Raw document, derived artifact, metadata, lineage, and search storage models; Accuracy measurement, confidence thresholds, human review, and correction feedback; Tenant isolation, PII handling, encryption, auditability, and retention. Strong evidence includes Makes assumptions explicit, answers the question asked, and explains both the benefit and cost of major decisions.