ML Pipeline
Architectureaidata
Gated SageMaker pipeline: train, evaluate, register, deploy, watch for drift
Open & fork this on Zstem →Architecture
- Training: Orchestrated build of a candidate model
- Serving: Promoted model behind an autoscaled endpoint
- Monitoring: Live-traffic feedback loop back into training
- Training Data: Versioned datasets; snapshots taken per pipeline run
- Pipeline Orchestrator: Runs prep → train → eval → register; triggered nightly or by drift
- Data Prep: Feature engineering and train/validation splits
- Training Job: Trains the candidate model; spot instances with checkpointing
- Evaluation Gate: Compares candidate vs baseline; blocks regressions from registering
- Model Registry: Versioned, eval-gated model artifacts approved for deployment
- Inference Endpoint: Real-time endpoint serving the promoted model version
- Auto Scaling: Scales endpoint instances on invocation load
- Data Capture: Sampled request/response payloads from the live endpoint
- Drift Detector: Compares live traffic stats to the training baseline; triggers retraining
- Metrics & Alarms: Latency, error and drift alarms
Workflow
- Nightly trigger or drift alarm
- Snapshot training data
- Preprocess + feature engineering
- Train candidate model
- Evaluate vs baseline
- Beats baseline + quality gates?
- Archive run
- Register model version
- Manual approval
- Canary deploy (10%)
- Canary healthy?
- Rollback to previous version
- Promote to 100%
- Model live, monitoring armed
UI/UX Studio
- Flexbox
- Frame 2