Silent Data Errors That Survive Every Cleaning Pipeline

A quarter of organizations say poor data quality costs them more than \$5 million a year, and 7% put that number above \$25 million, a Forrester finding IBM cites in its own data quality research. Most of that damage never shows up on a dashboard.

Standard cleaning catches nulls, duplicate rows, and type mismatches. Those checks run fast and catch real problems, then stop looking the moment every column passes.

Four bugs clear those checks anyway and still break the model, the dashboard, or the finance report downstream: entities duplicated without ever looking identical, values that sit inside range but make no sense in context, timestamps that leak information from the future, and denominators that drop the people who would have changed the answer.

Each one shows up in a real interview question on StrataScratch. Here's what we'd miss on a naive read of each question, and what to check for instead.

The Syntax, Semantics, and Pragmatics Gap

We can split data quality tests into three layers, borrowed from linguistics.

Syntax asks whether a value is well-formed: is the zip code 5 digits, does the date parse, is the column non-null.

Semantics asks whether the value makes sense for what it represents: is a 214-year-old customer possible, is a 3 am login from a store that closes at 10 pm believable.

Pragmatics asks whether the same kind of value needs a different test depending on who's using it: for example, a pharma company where a drug discovery team and a manufacturing team look at the same kind of record and don't agree on what counts as valid.

We see most production pipelines stop at syntax.

Pandera and dbt tests (SQL queries that check data quality) default to type, null, and range checks, the ones a generic library can write without knowing the business. Semantic checks need someone who understands what the data means. That gap is where the 4 bugs below live.

Silent Data Error #1: Duplicated Entities

A duplicate check tests whether 2 rows are byte-for-byte identical.