An audit of nine widely-adopted open-source projects — and what 6,397 drift findings say about the state of software documentation.
So we measured it. We ran Find the Gaps — Doc Holiday's open analysis tool — across nine widely-adopted open-source projects: Airbyte, Elasticsearch, Grafana, Mattermost, MongoDB, n8n, PostHog, Sentry, and Terragrunt. We analyzed 2,996 features and found 6,397 drift findings — specific places where the documentation and the code no longer agree.
"A drift finding isn't a typo. It's a specific place where a developer following the documentation will encounter a reality that doesn't match what they were told."
Each row is a project; each column a class of finding. The single green row is n8n.
Read across the row: Grafana documented essentially all of its 234 features — and still generated nearly 8 drift findings per feature. n8n documented 248 features and generated 18 in total.
n8n's numbers are exceptional — 18 drift findings across 248 features, roughly 32× cleaner than its peers. But that reliability trains developers to trust the docs implicitly, and that trust is exactly what makes n8n's few remaining gaps more dangerous than the hundreds of Large findings in the other eight projects combined.
"None of these are catastrophic in isolation. Together, in a project where developers have been trained to trust the documentation implicitly, they're landmines."
The teams maintaining these projects aren't negligent — several are among the best engineering organizations in open source. But modern software moves faster than the people documenting it. Every PR that renames a field, changes a response code, adds a required parameter, or removes a configuration option creates debt that has to be manually found, assigned, and fixed.
"For a project like Elasticsearch — 1,280 features, 950 drift findings — that's not a documentation problem. That's a documentation surface-area problem. Keeping the docs in sync through human effort alone is, practically speaking, impossible."
Find the Gaps uses LLMs to compare codebases against their documentation and surface areas of drift. LLMs make mistakes, so individual findings should be validated before acting on them. What the aggregate numbers represent is real: the scale of the gap between what documentation says and what code does, across projects that have every reason to get it right.
Doc Holiday connects to your release pipeline and rewrites the documentation when code deploys — so when a response code changes, a field moves, or a feature is deprecated, the docs say so before a developer finds out the hard way.
A PR ships. A response code changes. Three sprints later, a developer hits the old docs and spends a day debugging code that isn't broken. Multiply that by 6,397.
A PR ships. Doc Holiday reads the diff, drafts the documentation update, and queues it for review. The docs match what the code does. Every release.