A bank's credit model might be 94% accurate and still create a compliance liability if no one can explain a specific denial in plain terms.
The individual explanation requirement
Under Canadian consumer protection frameworks and OSFI model risk guidance, financial institutions must be able to explain adverse credit decisions to affected individuals. Population-level explanations — the model generally weights debt-to-income ratio heavily — do not satisfy this requirement. What compliance teams actually need is a structured explanation for each declined applicant, reproducible on demand, stored for audit. Most off-the-shelf ML platforms do not produce this by default.
Where institutions get stuck
The bottleneck is usually not the model itself but the data pipeline feeding it. When explanations reference input features that map poorly to human-readable concepts — encoded categorical variables, interaction terms, time-aggregated behavioral flags — the explanation becomes accurate but unusable. One regional lender spent four months post-deployment building a feature dictionary just to make SHAP outputs interpretable for their compliance team. That cost was not in the original project budget.
A more workable path
Banks that have moved past this problem typically made two decisions early. First, they constrained their feature set to variables with clear real-world meaning before model training. Second, they assigned a business analyst — not a data scientist — to own the explanation output layer. That person's job was to ensure every feature name and contribution value could be communicated verbally. Models built under those constraints performed slightly less well on AUC benchmarks but significantly better on deployment timelines and audit outcomes.