ML Platform Architecture

What Is an ML Platform?

An ML platform is the shared infrastructure that enables data scientists and ML engineers to build, deploy, and operate ML systems reliably and efficiently. It abstracts away infrastructure complexity so that teams can focus on models and business value.

Without a platform:

• Every team reinvents data pipelines, serving infrastructure, and monitoring
• Models are deployed inconsistently, with no standard evaluation or rollback process
• ML systems are brittle — dependencies are implicit, not declared

A mature platform provides five integrated layers:

Layer	Purpose	Key Tools
Data & Feature	Ingest, transform, and serve features	Feature store, data warehouse, streaming
Training	Managed, reproducible model training	Compute orchestration, experiment tracking
Model Registry	Version, evaluate, and promote models	MLflow, W&B, SageMaker Model Registry
Serving	Scalable, low-latency model inference	Triton, TorchServe, KServe, FastAPI
Monitoring	Detect drift, degradation, and failures	Evidently, Arize, Prometheus, Grafana

Layer 1: Data & Feature Platform

The foundation of any ML system is data. The feature platform handles:

Feature Engineering Pipeline

• Batch: Spark/dbt jobs compute aggregated features from the data warehouse
• Streaming: Kafka + Flink/Spark Streaming compute real-time features
• On-demand: compute simple features from the request payload at inference time

Storage

• Offline store: data warehouse (BigQuery, Snowflake, Redshift) — terabyte-scale historical data
• Online store: Redis or DynamoDB — millisecond reads of current feature values

Data Catalog & Lineage

Track where data comes from and how it flows through the system. Tools: DataHub, Apache Atlas, dbt lineage.

Layer 2: Training Infrastructure

Compute

• Cloud managed: Vertex AI Training, SageMaker Training, Azure ML — handles provisioning, scaling, and spot instance fallback
• Self-managed: Kubernetes with GPU nodes, job queues (Kueue, Volcano)

Orchestration

• Pipeline orchestration: Kubeflow Pipelines, Vertex AI Pipelines, Metaflow — define multi-step ML workflows as code
• Data orchestration: Airflow, Prefect — schedule and trigger training pipelines

Experiment Tracking

MLflow Tracking / W&B / Neptune log every run with parameters, metrics, and artifacts — linked back to the data version and code commit.

import kfp
from kfp import dsl
from kfp.components import func_to_container_op

@func_to_container_op
def validate_data(data_path: str) -> str:
    import great_expectations as ge
    df = ge.read_parquet(data_path)
    results = df.validate()
    assert results.success
    return data_path

@func_to_container_op
def train_model(data_path: str, learning_rate: float, n_estimators: int) -> str:
    import mlflow
    import pandas as pd
    from sklearn.ensemble import GradientBoostingClassifier
    import joblib

    df = pd.read_parquet(data_path)
    X, y = df.drop("label", axis=1), df["label"]

    with mlflow.start_run():
        model = GradientBoostingClassifier(
            learning_rate=learning_rate, n_estimators=n_estimators
        ).fit(X, y)
        mlflow.sklearn.log_model(model, "model")
        run_id = mlflow.active_run().info.run_id

    model_path = f"/tmp/model_{run_id}.pkl"
    joblib.dump(model, model_path)
    return model_path

@func_to_container_op
def evaluate_and_register(model_path: str, test_path: str, min_f1: float) -> str:
    import joblib, pandas as pd, mlflow
    from sklearn.metrics import f1_score

    model = joblib.load(model_path)
    test = pd.read_parquet(test_path)
    f1 = f1_score(test["label"], model.predict(test.drop("label", axis=1)))

    assert f1 >= min_f1, f"F1 {f1:.3f} below threshold {min_f1}"
    mv = mlflow.register_model(f"runs:/{model_path}/model", "fraud-detector")
    return mv.version

@dsl.pipeline(name="fraud-detector-training")
def training_pipeline(data_path: str, min_f1: float = 0.82):
    validated = validate_data(data_path)
    model = train_model(validated.output, learning_rate=0.05, n_estimators=200)
    version = evaluate_and_register(model.output, data_path, min_f1)

Layer 3: Model Registry

The model registry is the single source of truth for promoted models. It stores:

• Model artifact (weights, preprocessing pipeline, schema)
• Metadata (training data version, code commit, evaluation metrics, responsible AI checks)
• Lifecycle stage (staging, production, archived)
• Deployment targets (which serving endpoints are running this version)

Promotion Workflow

Data Scientist → registers model from experiment run
         ↓
ML Engineer → runs evaluation suite (performance, latency, fairness)
         ↓
Approver → approves promotion in registry
         ↓
CD System → deploys to staging environment
         ↓
After canary validation → promotes to production

Layer 4: Serving Layer

The serving layer is responsible for model inference at production scale.

Serving Patterns

Pattern	Use Case	Tool
REST API	Standard JSON in/out	FastAPI, KServe
gRPC	High-throughput, low-latency	Triton, KServe
Batch scoring	Large datasets offline	Spark, Ray, SageMaker Batch
Streaming	Kafka event-driven inference	Flink ML, Kafka Streams

KServe (Kubernetes Serving)

KServe is the standard model serving runtime on Kubernetes. It supports multiple frameworks (sklearn, TensorFlow, PyTorch, ONNX, HuggingFace) through a unified API and handles:

• Auto-scaling (including scale-to-zero)
• Canary deployments
• Request batching
• Pre/post-processing pipelines

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: fraud-detector
  namespace: ml-serving
spec:
  predictor:
    # Production model (90% traffic)
    sklearn:
      storageUri: "s3://my-models/fraud-detector/v2.1/"
      resources:
        requests:
          cpu: "1"
          memory: "2Gi"

  # Canary: new model gets 10% traffic
  canaryTrafficPercent: 10
  canary:
    predictor:
      sklearn:
        storageUri: "s3://my-models/fraud-detector/v2.2/"
        resources:
          requests:
            cpu: "1"
            memory: "2Gi"

  # Transformer for pre/post processing
  transformer:
    containers:
      - name: transformer
        image: my-registry/fraud-transformer:1.0
        env:
          - name: FEATURE_STORE_URL
            value: "http://feast-online-server:6566"

Layer 5: Monitoring & Observability

Production ML systems need three types of observability:

Infrastructure Monitoring

• Pod CPU/memory/GPU utilization (Prometheus + Grafana)
• Request latency (p50, p95, p99) and error rates
• Queue depth and throughput

ML Monitoring

• Data drift (PSI, KS-test) on input features
• Prediction distribution shift
• Model performance on labeled windows (when ground truth available)

Business Monitoring

• Business KPIs: fraud prevented, revenue impacted, customer satisfaction
• SLA breaches: predictions taking > X ms
• Model fairness metrics by demographic segment

Reference Architecture: Fraud Detection Platform

Putting all five layers together for a fraud detection use case:

[Kafka: transaction events]
        ↓
[Flink: real-time feature computation]
        ↓
[Feast Online Store (Redis)]          [Feast Offline Store (BigQuery)]
        ↓                                         ↓
[KServe: real-time inference]         [Vertex AI Training Pipeline]
        ↓                                         ↓
[Prediction Logs → BigQuery]          [MLflow Model Registry]
        ↓                                         ↓
[Evidently + Arize: drift monitoring] [Canary → Production promotion]
        ↓
[Grafana Dashboard + PagerDuty alerts]

Build vs. Buy Decision Matrix

Component	Build	Buy / OSS
Feature store	❌ expensive	Feast (OSS) or Tecton (managed)
Experiment tracking	❌	MLflow (OSS) or W&B
Model registry	Sometimes	MLflow, Vertex AI, SageMaker
Serving	Sometimes (FastAPI)	KServe, Triton (GPU)
Monitoring	❌	Evidently + Grafana, or Arize
Orchestration	❌	Airflow, Prefect, Kubeflow

Rule: Buy or use OSS for infrastructure; build only for business logic (feature definitions, model architecture, evaluation criteria).

Summary: MLOps Engineering Module

You've now covered the complete MLOps & AI Engineering stack:

1. ML Project Lifecycle — CRISP-DM, feasibility, success metrics, data leakage prevention
2. Data Versioning & Pipelines — DVC, Airflow, Prefect, Kedro for reproducible workflows
3. Experiment Tracking — MLflow, W&B, Neptune for runs, metrics, and artifact management
4. Feature Stores — Feast, Tecton, Hopsworks with point-in-time correct offline/online serving
5. Model Serving — FastAPI, ONNX, TorchServe, Triton with batching and quantization
6. CI/CD for ML — model testing, GitHub Actions pipelines, automated retraining triggers
7. Production Monitoring — data drift, concept drift, Evidently, Arize, Fiddler
8. A/B Testing — statistical power, holdback groups, Thompson Sampling bandits, interleaving
9. Cost Optimization — spot instances, auto-scaling, right-sizing, cost-per-prediction
10. ML Platform Architecture — five-layer reference design connecting all components

The mark of a production-grade ML system is not the model itself, but the infrastructure that makes the model reliable, reproducible, and improvable over time.