Why API tests get abandoned: compare response shape, not values
Most teams that write API tests end up deleting them. The reason is usually not that the tests were wrong. It is that they were noisy. A test that fails every Monday because an id or a token changed trains everyone to ignore it.
I ran into this while building HTTP Runner, a VS Code extension that re-runs the .http files you already have. This post is about one design choice that fixed the noise for me.
The problem: values change, structure does not
A typical response from a login endpoint has a fresh token and a new id every time. If you save the whole body as the expected result, the next run fails even though nothing is broken.
The fix: save the shape
Instead of saving the response, I save its shape: the keys, the types, and the status code. The first response becomes the baseline. On the next run I only report what changed in structure:
a field that disappeared
a field whose type changed (a number that is now a string)
a different status code
A new id or a new token gives zero difference, because both are still strings in the same place. A removed field gives a clear failure, and that is the kind of change that breaks real clients.
What I kept optional
Sometimes you do want exact values. So there are two switches: an exact mode that compares values too, and an off mode that turns the check off for one request. The default is the quiet one, because a check people leave on is worth more than a strict one they switch off.
What I would tell you to try
If you maintain API tests, count how many failures last month were real. If most were values that change by design, move those checks to shape comparison first and keep exact checks only for the few fields that must never change.
HTTP Runner (VS Code): https://marketplace.visualstudio.com/items?itemName=sellerkit.httprunner
About me: full-stack developer in South Korea (UTC+9), 13 years, open to remote contracts. I work fully in writing (async), so everything stays documented. CV: https://cv.namubase.com