Drift with PSI
Shift the production distribution and watch the PSI and the alert that triggers retraining.
Production scenario
Pick a case or tune it by hand.
Training: 4,000 customers, mean 35, sd 11.
Nothing to do: the model sees customers like the ones it was trained on.
Customer age: training × production
In training every bin holds 10% of customers (the dashed line). Bars are production: the warmer the colour, the more the bin weighs in the PSI.
Week by week in production
If the audience drifts gradually toward the chosen scenario, this is how monitoring sees the PSI climb.
See the table with each bin’s numbers
The PSI is the sum of the right-hand column.
| Age binAge | Training | ProductionProd. | ContributionContrib. |
|---|---|---|---|
| PSI | |||
How PSI works
- 1
Slice the training data
Training customer ages are split into 10 bins holding 10% of customers each (deciles).
- 2
Count production
Customers arriving in production are placed into those same bins.
- 3
Add up the gap
Each bin adds (production − training) × ln(production ÷ training). An unchanged bin adds zero.
PSI = Σ (pprod − ptrain) · ln(pprod / ptrain)
In mlops-platform this runs for every feature of the credit model. Above 0.25 it opens a retrain, and the new model only replaces the champion if it wins on quality and fairness criteria.