AI Models Drift Under Your App and Nobody Tells You

Your app calls an AI model by name. It’s a string in a config file, gpt-4o or claude-sonnet-4-5, and it was right the day someone typed it. The model behind that string lives on somebody else’s servers, on somebody else’s schedule.

Providers retire models. They rename them, or point an alias at a newer one. They cut a context window, add a retirement date, move a model from active to deprecated, raise the price per token. None of that is a secret. It goes into a changelog, a deprecations page, maybe an email to whoever signed up for the account three years ago. And the team that owns the app finds out from a stack trace.

The app’s config doesn’t change. The list it depends on does.

The list is already published

Here’s the part that makes this fixable. Every provider publishes its model list through its API. OpenAI, Anthropic, Google and OpenRouter all answer a plain request with the models they serve right now, and when a model goes, it goes from the list. Anthropic’s list says which models are deprecated and when they retire. OpenRouter’s has limits, prices and retirement dates for hundreds of models. Google’s has token limits and what each model supports.

So nobody needs to read changelogs. A small program can read the lists on a schedule, keep the last one, compare, and tell you what moved. That’s RadarModel AI.

Only the models you use

The catch is noise. OpenRouter alone listed 470 models when we scanned it on October 17, 2026, and the lists change often. Nobody wants an alert because a provider added forty models they’ll never call.

So you tell it which models your app calls. Those are the only ones that can raise anything. A watched model gone from the list is breaking. A cut in its context or output limit is breaking, because requests that fit yesterday may not fit today. A price rise, a retirement date or a move to deprecated is a warning. Everything else, the new models, the price cuts, the bigger windows, is info you can look at when you want.

A broken answer is not a removed model

The other catch is crying wolf. A watcher that pages you because a provider had a bad minute is worse than no watcher. You’d mute it by Wednesday.

RadarModel AI treats a bad answer as a failed scan. A 500, a rate limit, a maintenance page, a cut-off body, an empty list, a page in the middle of a paged list that didn’t load: each one is reported as “couldn’t scan this provider”, the stored snapshot stays exactly as it was, and the next good scan compares against it. And a model has to be missing from two scans in a row before it counts as breaking, which covers the one failure nobody can see: a list that comes back short with no error at all. How that was tested

Where it fits

It’s one binary with no dependencies. Run check from cron, from CI as a gate (it exits 1 on a breaking change), or leave watch running. Reports go to the terminal, a webhook, Slack or email. It reads the lists and never calls a model, so it costs nothing to run.

The same engine will watch something else next. RadarModel Schema points the same snapshot, compare and alert loop at databases, for the columns and indexes that change outside your migrations. Different model, same drift.

Start with the quick start.