ML Project Lifecycle

The ML Project Lifecycle

Building a machine learning system is not just about training a model. A production ML project spans business understanding, data engineering, modelling, deployment, and ongoing monitoring — in cycles that repeat as the system evolves.

The most widely used framework for structuring this work is CRISP-DM (Cross-Industry Standard Process for Data Mining), but modern MLOps practice extends it significantly.

The Six CRISP-DM Phases

Phase	Key Activities
Business Understanding	Define objectives, success criteria, constraints
Data Understanding	Explore, profile, and assess data quality
Data Preparation	Clean, transform, feature engineer
Modelling	Select algorithms, train, tune
Evaluation	Validate against business goals, not just metrics
Deployment	Serve predictions, integrate with downstream systems

The process is iterative — evaluation findings send you back to earlier phases. Plan for this.

Phase 1: Business Understanding

The most important phase, and the most often skipped. Before writing a line of code:

Define the Problem

• What decision will the model improve or automate?
• Who is the end user and what is their workflow?
• What does "wrong" cost? (false positives vs. false negatives)

Assess Feasibility

Three questions determine whether ML is the right tool:

1. Is there training data? ML requires examples of the input-output relationship you want to learn. If historical data doesn't exist, you may need to collect it first.
2. Is the pattern learnable? The relationship must be consistent enough to generalize. If humans can't reliably make the prediction from the same inputs, ML probably can't either.
3. Is the ROI positive? The cost of building, deploying, and maintaining an ML system must be less than the value it creates.

Define Success Metrics

A good ML project has two layers of metrics:

Layer	Examples
ML metrics	Precision, recall, F1, RMSE, AUC-ROC
Business metrics	Revenue lift, churn reduction, cost per prediction

ML metrics measure model quality. Business metrics measure actual impact. Both matter, but business metrics are the real goal.

Phase 2: Data Understanding

Before modelling, spend significant time understanding your data.

Data Profiling

• Distribution of each feature (mean, std, min, max, percentiles)
• Missing value rates per column
• Cardinality of categorical features
• Target variable distribution (class imbalance?)
• Temporal patterns if data is time-ordered

Data Quality Issues to Catch Early

• Leakage: features that implicitly contain the target or future information
• Drift: training data from a different distribution than production
• Staleness: labels that were correct at collection time but no longer are
• Bias: systematic under- or over-representation of subgroups

import pandas as pd
import numpy as np

df = pd.read_parquet("data/raw/transactions.parquet")

# Basic profile
print(df.shape)
print(df.dtypes)
print(df.isnull().mean().sort_values(ascending=False).head(10))

# Target distribution
target = "is_fraud"
print(df[target].value_counts(normalize=True))

# Check for potential leakage: columns correlated perfectly with target
correlations = df.select_dtypes(include=np.number).corrwith(df[target]).abs()
print(correlations.sort_values(ascending=False).head(10))

Phase 3: Data Preparation

Data preparation typically consumes 60–80% of project time. It includes:

• Cleaning: handle missing values, outliers, duplicates
• Transformation: scaling, encoding, normalization
• Feature engineering: create informative features from raw data
• Train/validation/test split: preserve temporal order for time-series data

The Critical Rule: No Data Leakage

All preprocessing steps (scalers, encoders, imputers) must be fit on training data only and then applied to validation and test sets. Fitting on the full dataset leaks future information into your model.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# Fit ONLY on training data — transform applies to both
pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler()),
    ('model', GradientBoostingClassifier()),
])

pipeline.fit(X_train, y_train)
print(pipeline.score(X_test, y_test))

Phases 4–6: Modelling, Evaluation, Deployment

Modelling

• Start simple: a linear model or decision tree sets a strong baseline
• Use cross-validation to estimate generalization
• Track every experiment — model type, hyperparameters, metrics (Chapter 3 covers this)

Evaluation

Before declaring success, validate against your business metrics:

• Run an offline simulation or backt est on held-out time periods
• Check performance across subgroups (fairness check)
• Confirm the model doesn't degrade on recent data

Deployment

Modern ML deployment is covered in depth in Chapter 5. Key questions:

• Batch or real-time prediction?
• What SLA (latency, throughput) is required?
• How will the model be updated when it degrades?

The Modern ML Lifecycle

Production ML systems add two phases beyond CRISP-DM:

7. Monitoring — detect data drift, model degradation, and infrastructure failures (Chapter 7)
8. Retraining — trigger new training runs when performance drops (Chapter 6)

These phases run continuously and feed back into all earlier phases.

Summary

1. CRISP-DM provides a proven iterative framework: Business Understanding → Data → Modelling → Evaluation → Deployment
2. Feasibility assessment should happen before any code: check for data, learnability, and positive ROI
3. Success metrics must have both an ML layer (precision, recall, AUC) and a business layer (revenue, cost)
4. Data leakage is the #1 silent failure mode — always fit preprocessing on training data only
5. Production systems add Monitoring and Retraining phases that run continuously

Next chapter: how to version your data and build reproducible pipelines with DVC, Airflow, and Kedro.