When API fields change meaning but not name — the $400K silent killer that no existing tool can detect
Stripe upgraded their API from version 2024-12-18 to 2025-04-16. The payment_intent.amount field changed from presentment currency (EUR amount for EUR payments) to settlement currency (always USD).
The column name didn't change. The data type didn't change. The schema didn't change. Every existing monitoring tool — Fivetran alerts, dbt tests, Elementary anomaly detection — saw nothing wrong.
The dbt model had a CASE WHEN currency != 'usd' THEN amount * fx_rate that was now double-converting already-USD amounts. Result: $400K revenue underreported. The CFO found it, not the data team.
Across all 3 tooling conditions (standard, dbt-tests, full observability), severity remained at 9/10. Even with Elementary anomaly detection, the engineer could see that something was wrong but not why. The root cause required understanding API changelog semantics — something no existing tool does.
"Numbers look wrong but nothing technically failed" — the #1 complaint pattern on r/dataengineering. Survey data: 2/3 of data engineers blame poor source data as root cause. MotherDuck blog: "'It's just a small schema change' strikes fear into every data engineer."
| Tool | What It Detects | Why It Misses Semantic Drift |
|---|---|---|
| Fivetran Alerts | Schema changes (column add/remove/rename) | Schema didn't change — still called amount, still integer |
| dbt Tests | not_null, unique, accepted_values | Amount is not null, is unique per payment, value is valid integer |
| Elementary | Statistical anomalies, freshness | Detects the anomaly but cannot explain WHY or connect to API changelog |
| Great Expectations | Distribution checks, range validation | Can flag unusual distribution but not map to upstream semantic change |
| Monte Carlo | Volume, freshness, schema, distribution | Same as Elementary — symptom detection, not root cause |
The gap: All existing tools operate at the data layer (values, schema, statistics). Semantic drift operates at the meaning layer (what a field represents). Bridging this requires understanding API documentation — which is a natural language task that requires LLM capability.
⚠️ SEMANTIC DRIFT DETECTED
Source: Stripe API version 2024-12-18 → 2025-04-16
Field: payment_intent.amount
Change: Now returns settlement currency (USD) instead of
presentment currency (local currency)
Affected dbt models:
- stg_payments (line 23): CASE WHEN currency != 'usd'
THEN amount * fx_rate
⚠️ This will DOUBLE-CONVERT non-USD payments because
amount is already in USD in the new API version.
- fct_daily_revenue (depends on stg_payments)
- fct_monthly_revenue (depends on stg_payments)
- fct_customer_ltv (depends on stg_payments)
Suggested fix:
Remove the FX conversion for the 'amount' field.
Use amount directly as it is now always USD.
OR use the new 'presentment_amount' field if you need
local currency values.
| Competitor | What They Do | Gap |
|---|---|---|
| Monte Carlo ($1.6B valuation) | Data observability: anomaly detection | Detects symptom, not root cause. No API changelog parsing. |
| Elementary (open source) | dbt-native observability | Same gap — operates on data layer only |
| Soda | Data quality checks | Rule-based checks, no semantic understanding |
| This Product | API changelog → SQL impact analysis | Only product that bridges meaning layer to code layer |