Written by a person from the record · 2026-07-28 09:55:00 UTC
Eight requests from OpenAI's two automated crawlers asked only for the rules and the map
Between 25 and 27 July 2026, requests declaring OAI-SearchBot arrived here six times from five addresses on three separate days, and every one of them asked for /robots.txt. Requests declaring GPTBot arrived twice and both asked for /sitemap.xml. Neither crawler has fetched a single page of this site's content. The only OpenAI identity that did is ChatGPT-User, which fetches when a person asks it something. A further eleven requests declaring OAI-SearchBot are excluded: they came from one address that presented thirteen different companies' crawler identities inside a minute.
What was observed
Three OpenAI crawler identities have reached this site. Their requests separate cleanly, and the separation is the finding.
| Declared identity | Requests | Addresses | Days | What it asked for |
|---|---|---|---|---|
OAI-SearchBot | 6 | 5 | 3 | /robots.txt, every time |
GPTBot | 2 | 1 | 2 | /sitemap.xml, both times |
ChatGPT-User | 2 | 2 | 1 | / and /questions |
Requests declaring OAI-SearchBot or GPTBot have never asked for a page of this site's content — not once in eight requests across three days. They asked for the file that states the rules and the file that lists the pages, and stopped there.
Twice, the two arrived together. On 26 July a request for /robots.txt was followed 52 seconds later by a request for /sitemap.xml from a different address; on 27 July the two arrived in the same second. Both addresses fall inside ranges OpenAI publishes, and they are not the same range.
What is excluded, and why
A further eleven requests declared OAI-SearchBot and are left out of every figure above. All eleven came from one address which, in a single minute, also presented twelve other companies' crawler identities while requesting /.env, /.git/config and sixty other credential paths. That incident is its own finding.
Including them would have produced the opposite result — the same identity would appear to read source control and configuration files eagerly. A user agent is a string the sender chooses, and a habit attributed without checking the address is a habit attributed to whoever last borrowed the name.
What it means
Being read by an AI crawler and being read at all are different events, and this record separates them for one company over three days. The crawlers that arrive on their own schedule took the rules and the index. The pages were fetched only when a person asked a question and an agent went to look — a different mechanism, a different identity, and one request each time.
For anyone measuring whether AI systems see their site, that distinction matters more than a visit count. Eight arrivals from OpenAI's infrastructure look like attention. None of them read a sentence of what is here.
What this does not establish
Four readings of the same observation remain, and the record separates only two of them. All four are published with the observation that would settle each.
- Delegation is supported. Content was fetched from OpenAI's infrastructure — under ChatGPT-User, twice.
- Refusal is contradicted. This site's
robots.txt, read back from what the network actually delivers rather than from the code that generates it, welcomes these crawlers by name. - "Not yet" is open. Three days settles nothing about a longer horizon.
- "This is what the crawler is for" is open, deliberately. It could be closed by consulting OpenAI's documentation, and a company's account of its own crawler is not evidence about that crawler's behaviour here.
Limits
- Eight requests over three days, on one site. This is what happened here. It supports no rate, no proportion, and nothing about how these crawlers behave on any other domain.
- Corroboration means the connecting address falls inside a range OpenAI publishes, checked against a dated snapshot rather than a live fetch. It establishes that the request came from that infrastructure, and nothing about what any system did afterwards.
- This site is four days old, small, and mostly about crawlers. A crawler's behaviour here need not resemble its behaviour on a large site with ordinary content.
- The site publishes a
Crawl-delayof 10 seconds and several major crawlers document that they ignore it. Nothing above depends on whether it was honoured. - Absence is not refusal. That a page was never fetched in three days does not establish that it will not be, and no sentence here should be read as saying so.
Why this was worth recording
The question this site exists to answer is whether AI systems read the web, and the usual proxy for it is whether their crawlers show up. Here they showed up eight times and read nothing — which is only visible because the requests were kept whole, with the path attached, and because eleven requests wearing the same name were checked against an address list before being counted.
Evidence chain
This finding was not written from memory. Each figure below came from a query against the immutable request record, and each query is shown so it can be run again — by us, on this page, or by anyone reconstructing the same instrument.
| Stage | What it produced |
|---|---|
| 1. Reality | Requests recorded by the server as they arrived, written once and never edited. |
| 2. Evidence | The rule human_analysis selected the observations that bear on this question and emitted each figure with the means of reproducing it. |
| 3. Statement | A fixed template turned that evidence into the prose above. No language model was involved and no figure was altered. |
| 4. Verification | Every figure was recomputed against the record. A single mismatch would have discarded the draft. |
| 5. Publication | 2026-07-28 09:55:00 UTC, after a person wrote and reviewed it. |
What was measured
Each figure is stated first as what it means, which does not depend on how this system happens to store anything today. The query underneath it is implementation — it would be written differently on different storage, and it is shown so the figure can be checked, not because it is part of the claim.
Evidence
How many requests declared OAI-SearchBot
At publication: 17 · recomputed now: 23 window still open
Implementation — how this is computed on today's storage
SELECT COUNT(*) AS n FROM RequestReality WHERE
cfRay IS NOT NULL
AND cfConnectingIp IS NOT NULL
AND cfConnectingIp NOT LIKE '203.0.113.%'
AND cfConnectingIp NOT LIKE '192.0.2.%'
AND cfConnectingIp NOT LIKE '198.51.100.%'
AND cfConnectingIp NOT LIKE '10.%'
AND cfConnectingIp NOT LIKE '192.168.%'
AND cfConnectingIp NOT LIKE '127.%'
AND cfConnectingIp NOT LIKE '2a00:1d34:4896:b600:%'
AND cfConnectingIp NOT IN (
SELECT cfConnectingIp FROM RequestReality
WHERE cfConnectingIp IS NOT NULL AND (userAgent LIKE 'curl%' OR userAgent LIKE 'Wget%' OR userAgent LIKE 'Python-urllib%' OR userAgent LIKE '%python-requests%' OR userAgent LIKE 'Go-http-client%' OR userAgent LIKE 'node-fetch%')) AND userAgent LIKE '%OAI-SearchBot%'Evidence
How many of those came from the address that presented thirteen crawler identities
At publication: 11 · recomputed now: 11 unchanged
Implementation — how this is computed on today's storage
SELECT COUNT(*) AS n FROM RequestReality WHERE
cfRay IS NOT NULL
AND cfConnectingIp IS NOT NULL
AND cfConnectingIp NOT LIKE '203.0.113.%'
AND cfConnectingIp NOT LIKE '192.0.2.%'
AND cfConnectingIp NOT LIKE '198.51.100.%'
AND cfConnectingIp NOT LIKE '10.%'
AND cfConnectingIp NOT LIKE '192.168.%'
AND cfConnectingIp NOT LIKE '127.%'
AND cfConnectingIp NOT LIKE '2a00:1d34:4896:b600:%'
AND cfConnectingIp NOT IN (
SELECT cfConnectingIp FROM RequestReality
WHERE cfConnectingIp IS NOT NULL AND (userAgent LIKE 'curl%' OR userAgent LIKE 'Wget%' OR userAgent LIKE 'Python-urllib%' OR userAgent LIKE '%python-requests%' OR userAgent LIKE 'Go-http-client%' OR userAgent LIKE 'node-fetch%')) AND userAgent LIKE '%OAI-SearchBot%' AND cfConnectingIp = '45.45.237.206'Evidence
How many came from any other address
At publication: 6 · recomputed now: 12 window still open
Implementation — how this is computed on today's storage
SELECT COUNT(*) AS n FROM RequestReality WHERE
cfRay IS NOT NULL
AND cfConnectingIp IS NOT NULL
AND cfConnectingIp NOT LIKE '203.0.113.%'
AND cfConnectingIp NOT LIKE '192.0.2.%'
AND cfConnectingIp NOT LIKE '198.51.100.%'
AND cfConnectingIp NOT LIKE '10.%'
AND cfConnectingIp NOT LIKE '192.168.%'
AND cfConnectingIp NOT LIKE '127.%'
AND cfConnectingIp NOT LIKE '2a00:1d34:4896:b600:%'
AND cfConnectingIp NOT IN (
SELECT cfConnectingIp FROM RequestReality
WHERE cfConnectingIp IS NOT NULL AND (userAgent LIKE 'curl%' OR userAgent LIKE 'Wget%' OR userAgent LIKE 'Python-urllib%' OR userAgent LIKE '%python-requests%' OR userAgent LIKE 'Go-http-client%' OR userAgent LIKE 'node-fetch%')) AND userAgent LIKE '%OAI-SearchBot%' AND cfConnectingIp <> '45.45.237.206'Evidence
How many of those asked for a path other than /robots.txt
At publication: 0 · recomputed now: 0 unchanged
Implementation — how this is computed on today's storage
SELECT COUNT(*) AS n FROM RequestReality WHERE
cfRay IS NOT NULL
AND cfConnectingIp IS NOT NULL
AND cfConnectingIp NOT LIKE '203.0.113.%'
AND cfConnectingIp NOT LIKE '192.0.2.%'
AND cfConnectingIp NOT LIKE '198.51.100.%'
AND cfConnectingIp NOT LIKE '10.%'
AND cfConnectingIp NOT LIKE '192.168.%'
AND cfConnectingIp NOT LIKE '127.%'
AND cfConnectingIp NOT LIKE '2a00:1d34:4896:b600:%'
AND cfConnectingIp NOT IN (
SELECT cfConnectingIp FROM RequestReality
WHERE cfConnectingIp IS NOT NULL AND (userAgent LIKE 'curl%' OR userAgent LIKE 'Wget%' OR userAgent LIKE 'Python-urllib%' OR userAgent LIKE '%python-requests%' OR userAgent LIKE 'Go-http-client%' OR userAgent LIKE 'node-fetch%')) AND userAgent LIKE '%OAI-SearchBot%' AND cfConnectingIp <> '45.45.237.206' AND path <> '/robots.txt'Evidence
How many distinct addresses sent them
At publication: 5 · recomputed now: 9 window still open
Implementation — how this is computed on today's storage
SELECT COUNT(DISTINCT cfConnectingIp) AS n FROM RequestReality WHERE
cfRay IS NOT NULL
AND cfConnectingIp IS NOT NULL
AND cfConnectingIp NOT LIKE '203.0.113.%'
AND cfConnectingIp NOT LIKE '192.0.2.%'
AND cfConnectingIp NOT LIKE '198.51.100.%'
AND cfConnectingIp NOT LIKE '10.%'
AND cfConnectingIp NOT LIKE '192.168.%'
AND cfConnectingIp NOT LIKE '127.%'
AND cfConnectingIp NOT LIKE '2a00:1d34:4896:b600:%'
AND cfConnectingIp NOT IN (
SELECT cfConnectingIp FROM RequestReality
WHERE cfConnectingIp IS NOT NULL AND (userAgent LIKE 'curl%' OR userAgent LIKE 'Wget%' OR userAgent LIKE 'Python-urllib%' OR userAgent LIKE '%python-requests%' OR userAgent LIKE 'Go-http-client%' OR userAgent LIKE 'node-fetch%')) AND userAgent LIKE '%OAI-SearchBot%' AND cfConnectingIp <> '45.45.237.206'Evidence
How many separate days they arrived on
At publication: 3 · recomputed now: 6 window still open
Implementation — how this is computed on today's storage
SELECT COUNT(DISTINCT substr(observedAt,1,10)) AS n FROM RequestReality WHERE
cfRay IS NOT NULL
AND cfConnectingIp IS NOT NULL
AND cfConnectingIp NOT LIKE '203.0.113.%'
AND cfConnectingIp NOT LIKE '192.0.2.%'
AND cfConnectingIp NOT LIKE '198.51.100.%'
AND cfConnectingIp NOT LIKE '10.%'
AND cfConnectingIp NOT LIKE '192.168.%'
AND cfConnectingIp NOT LIKE '127.%'
AND cfConnectingIp NOT LIKE '2a00:1d34:4896:b600:%'
AND cfConnectingIp NOT IN (
SELECT cfConnectingIp FROM RequestReality
WHERE cfConnectingIp IS NOT NULL AND (userAgent LIKE 'curl%' OR userAgent LIKE 'Wget%' OR userAgent LIKE 'Python-urllib%' OR userAgent LIKE '%python-requests%' OR userAgent LIKE 'Go-http-client%' OR userAgent LIKE 'node-fetch%')) AND userAgent LIKE '%OAI-SearchBot%' AND cfConnectingIp <> '45.45.237.206'Evidence
How many requests declared GPTBot, excluding that address
At publication: 2 · recomputed now: 60 window still open
Implementation — how this is computed on today's storage
SELECT COUNT(*) AS n FROM RequestReality WHERE
cfRay IS NOT NULL
AND cfConnectingIp IS NOT NULL
AND cfConnectingIp NOT LIKE '203.0.113.%'
AND cfConnectingIp NOT LIKE '192.0.2.%'
AND cfConnectingIp NOT LIKE '198.51.100.%'
AND cfConnectingIp NOT LIKE '10.%'
AND cfConnectingIp NOT LIKE '192.168.%'
AND cfConnectingIp NOT LIKE '127.%'
AND cfConnectingIp NOT LIKE '2a00:1d34:4896:b600:%'
AND cfConnectingIp NOT IN (
SELECT cfConnectingIp FROM RequestReality
WHERE cfConnectingIp IS NOT NULL AND (userAgent LIKE 'curl%' OR userAgent LIKE 'Wget%' OR userAgent LIKE 'Python-urllib%' OR userAgent LIKE '%python-requests%' OR userAgent LIKE 'Go-http-client%' OR userAgent LIKE 'node-fetch%')) AND userAgent LIKE '%GPTBot%' AND cfConnectingIp <> '45.45.237.206'Evidence
How many of those asked for a path other than /sitemap.xml
At publication: 0 · recomputed now: 56 window still open
Implementation — how this is computed on today's storage
SELECT COUNT(*) AS n FROM RequestReality WHERE
cfRay IS NOT NULL
AND cfConnectingIp IS NOT NULL
AND cfConnectingIp NOT LIKE '203.0.113.%'
AND cfConnectingIp NOT LIKE '192.0.2.%'
AND cfConnectingIp NOT LIKE '198.51.100.%'
AND cfConnectingIp NOT LIKE '10.%'
AND cfConnectingIp NOT LIKE '192.168.%'
AND cfConnectingIp NOT LIKE '127.%'
AND cfConnectingIp NOT LIKE '2a00:1d34:4896:b600:%'
AND cfConnectingIp NOT IN (
SELECT cfConnectingIp FROM RequestReality
WHERE cfConnectingIp IS NOT NULL AND (userAgent LIKE 'curl%' OR userAgent LIKE 'Wget%' OR userAgent LIKE 'Python-urllib%' OR userAgent LIKE '%python-requests%' OR userAgent LIKE 'Go-http-client%' OR userAgent LIKE 'node-fetch%')) AND userAgent LIKE '%GPTBot%' AND cfConnectingIp <> '45.45.237.206' AND path <> '/sitemap.xml'Evidence
How many requests declared ChatGPT-User, excluding that address
At publication: 2 · recomputed now: 2 unchanged
Implementation — how this is computed on today's storage
SELECT COUNT(*) AS n FROM RequestReality WHERE
cfRay IS NOT NULL
AND cfConnectingIp IS NOT NULL
AND cfConnectingIp NOT LIKE '203.0.113.%'
AND cfConnectingIp NOT LIKE '192.0.2.%'
AND cfConnectingIp NOT LIKE '198.51.100.%'
AND cfConnectingIp NOT LIKE '10.%'
AND cfConnectingIp NOT LIKE '192.168.%'
AND cfConnectingIp NOT LIKE '127.%'
AND cfConnectingIp NOT LIKE '2a00:1d34:4896:b600:%'
AND cfConnectingIp NOT IN (
SELECT cfConnectingIp FROM RequestReality
WHERE cfConnectingIp IS NOT NULL AND (userAgent LIKE 'curl%' OR userAgent LIKE 'Wget%' OR userAgent LIKE 'Python-urllib%' OR userAgent LIKE '%python-requests%' OR userAgent LIKE 'Go-http-client%' OR userAgent LIKE 'node-fetch%')) AND userAgent LIKE '%ChatGPT-User%' AND cfConnectingIp <> '45.45.237.206'6 figures have moved since publication. That is expected where a window is still open: the record grew, so the count grew. It means the sentence has aged, not that it was wrong. The published value is what the record showed at the moment it was published, and both are kept.
Governed by
- I. Reality is not interpretation — Reality ≠ Interpretation
- IV. Replay guesses nothing — Every figure must be reproducible from the record
- VI. The record is honest about its size — Sample size is stated; absence is reported as absence
These are the standards this finding was held to, not a claim that it is beyond question. The constitution states them in full.