Document Processing
Architectureaievent-drivenserverlessdata
Step Functions doc pipeline: Textract, Comprehend, human review, search
Open & fork this on Zstem →Architecture
- Ingest: Uploads become pipeline executions
- Extract & Review: Extraction with a human check on low confidence
- Store & Serve: Durable results, search and notifications
- Upload Bucket: Documents land here; ObjectCreated events start the pipeline
- Event Trigger: Validates the object and starts a pipeline execution
- Doc Pipeline: Orchestrates extraction, enrichment and review with retries
- Textract: OCR + form/table extraction; async for multi-page documents
- Comprehend: Entities, key phrases and PII detection over extracted text
- Human Review: Low-confidence extractions routed to a reviewer before storage
- Results DB: Structured extraction results keyed by document
- Processed Archive: Original + normalized artifacts retained for audit
- Search Index: Full-text + entity search over processed documents
- Notify: Completion events for downstream subscribers
Event flow
- Upload Bucket: emits ObjectCreated on every document
- Textract: emits completion for async OCR jobs
- Object Created Events
- Textract Completion
- Review Tasks
- Document Processed
- Doc Pipeline: Step Functions execution per document
- Resume Handler: feeds OCR results back into the pipeline
- Human Review: low-confidence extractions
- Search Indexer
- Downstream Subscribers