The Hidden Risk In Automating Financial Models With AI

Aritra Pal, CFA, builds machine learning and risk models for regulated banks, with 17+ years in treasury and capital markets tech.

​Every bank I’ve worked at wants to move faster: credit decisions due the same afternoon, forecasts ready before the quarter they cover. I’ve spent more than a decade building machine learning models inside regulated financial institutions, and the failure I’ve come to fear isn’t a wrong number. It’s a model still running smoothly after everyone who understood it has left.

I’ve watched it happen. A model my team built ran in production for months without incident. Then a routine validation review raised a question about a design decision made in the project’s first week, a choice about how certain inputs were structured to work around a gap in the source data. Nobody could fully explain it. The decision made sense at the time, but it was never written down, and the person who made it had since moved to another team. Without that reasoning, we couldn’t satisfy the reviewer, and the model came out of production.

I still think it was a reasonable model. That opinion didn’t count for much. What stayed with me wasn’t really the loss of that one project. It was how little time it took for that knowledge to disappear. According to Gartner, “through 2026, organizations will abandon 60% of AI projects unsupported by AI-ready data.” The reasoning behind a model tends to live in people’s heads rather than in documentation; people move on to other teams, and unless someone deliberately holds onto that reasoning, it leaves with them.

Where The Hours Actually Go

The first question I usually get from leadership on a new AI initiative is which algorithm we’re using. Almost nobody asks where the data comes from, and that’s usually where most of the real work sits. Pulling consistent, reliable data out of six or seven internal systems that were never built to talk to each other takes far more time than most people expect.