Short answer: “Pages that did not respond” is a warning in the SEO area of a Site AI Audit report. It lists pages for which the crawler got no HTTP answer at all: the connection timed out, was refused or was cut off. It is different from a broken page, which answers with an error code, and from a slow page, which answers late. The usual causes are an overloaded server, a firewall or security plugin that blocks the crawler and pages that crash before they can send anything. The advice in the report is to check server stability and slow pages.
Three kinds of trouble, three findings
Site AI Audit separates pages that go wrong in different ways, because the causes and the fixes differ:
- Broken pages answer, but with an error status such as 404 or 500. The report treats links to them as critical, and the article on broken internal links explains the repair.
- Slow pages answer, but take more than two seconds. That is a notice, covered in the article on the slow pages finding.
- Pages that did not respond give no answer. There is no status code to read, only a failure to connect or to receive anything within the time allowed.
The third kind is the most puzzling for site owners, because the page may open fine in a browser a minute later. The report is honest about it: during the check, something stopped the answer from reaching the crawler.
What counts as no response
The crawler asks for each page and waits. If the server sends nothing for 15 seconds, or the whole transfer takes more than 25 seconds, the request is given up. Other failures are caught as well: the connection is refused, the connection is reset in the middle of the transfer, the name cannot be resolved or the secure handshake fails for that address. In all of these cases, the audit has no status code to record, so the page is listed as one that did not respond.
The finding is a warning, not a critical item, because a few such pages may be a short hiccup. But the report keeps the addresses, and it is worth looking at them, since a page that cannot be fetched cannot be indexed either, and visitors may have had the same experience.
Common causes
- An overloaded server. On cheap shared hosting, a burst of visitors, a backup or a heavy plugin can leave no capacity for new requests. The crawler makes about three requests a second, which is gentle, but a server that is already at its limit may still fail to answer. The article on slow server findings explains what to ask a host.
- Rate limits and blocks. A firewall, a security plugin or a hosting rule may treat the crawler as a bot and drop its connections after some number of requests. Then the first pages answer, and later ones do not.
- Pages that crash. A script that runs out of memory or time can end without sending a response. The visitor sees a blank page or a connection error.
- Heavy pages. A page that builds a large report, a huge search result or a big export may take longer than 25 seconds.
- Problems with part of the site. A subdomain, a separate application or an old section that points to a server that is switched off or has an expired certificate.
- Network trouble. A temporary routing problem between the crawler and your host. These are rare and usually do not repeat.
Step 1: look at the list and the pattern
Open the finding in the report. Do not stop at the number; read the addresses. A pattern usually appears:
- All from one section. For example, only addresses under /shop/ or /search/. The cause is probably in that part of the application.
- Only late in the crawl. If the first pages answered and the later ones did not, a rate limit or a block is likely.
- Only heavy pages. Reports, exports and big archives suggest timeouts due to the work they do.
- Scattered with no pattern. That suggests general load or short interruptions.
Step 2: try the pages yourself
Open several of the listed addresses in a browser. If they load, time how long it takes. A page that needs ten seconds on your computer is a candidate for the crawler’s limit. If they load fast, the problem was temporary or related to the crawler specifically. If they fail for you too, you have found a real fault: look at the error message and the server log.
Step 3: check the server logs
Your hosting panel or your host can show the access log and the error log. Look at the time of the audit, which is shown in the report. Three things to look for are lines for the listed addresses with a long response time, errors about memory, timeouts or process limits, and requests from the crawler that were refused. The crawler introduces itself as SiteAuditBot, a name that you can search for. The article on what SiteAuditBot does on your site describes its behaviour and its limits.
Step 4: make sure the crawler is not blocked
If the log or the pattern suggests blocking, check what protects your site. Security plugins, web application firewalls, hosting rate limits and bot protection services can all drop connections from automated visitors. Allow the crawler by name or by its behaviour, or raise the rate limit a little. Do this carefully: you want to let your own audit in without opening the door to scrapers. If the crawler is blocked by design, for example on a staging site, the audit will not be able to read it. The article on robots.txt and sitemap findings explains the rules that the crawler follows.
Step 5: fix the real cause
- Load. Add page caching, reduce heavy plugins, compress images, or move to a plan with more resources. A CDN can take static files off the server.
- Heavy pages. Add pagination or filters, cache the result or run the work in the background.
- Crashes. Read the error log and fix the script, or raise the memory limit if the work is legitimate.
- Abandoned sections. Remove links to switched-off parts of the site, or restore them.
- Blocks. Adjust the firewall rule or rate limit.
Then run the audit again. If the finding stays, compare the list with the first one. Pages that still fail are the real problems, and pages that disappeared from the list were short interruptions.
Prevent it from coming back
Once the cause is fixed, keep an eye on it. Server trouble tends to return when traffic grows, when a plugin is added or when a backup job moves to a busy hour. Three habits help. Run an audit after any larger change to the site or the hosting plan. Ask your host to alert you when the server reaches its process or memory limits. And when you add an interactive feature, such as a calendar, a search or a filter with many combinations, test how a single request behaves under load before you link to it from every page. A feature that needs twenty seconds to build a result is a risk for visitors as well as for the audit.
If the same pages fail again after a fix, do not assume that the crawler is at fault. Compare the time of the check with the server log, and look at what else was running at that time. Scheduled tasks, imports and backups are common hidden causes.
Why it matters beyond the audit
Search engine crawlers fail in the same way. If Googlebot meets timeouts, it slows down and visits less often, and pages it cannot fetch may drop out of the index. Visitors meet the same wall and do not come back. The Search Console reports on server errors, discussed in the article on 5xx errors in Search Console, are the other side of the same problem. A clean result in the audit is a good sign for both.
A worked example
A cleaning company with a booking calendar runs on shared hosting. The audit shows a warning: 23 pages did not respond. All of them are under /calendar/ and have long parameter strings. The owner opens three of them and sees that each takes about 20 seconds, because the calendar plugin builds a month view from the database each time. The host’s log confirms the process limit being reached. The agency adds caching for the calendar views and removes the links that generate endless date combinations. A new audit lists no pages without a response, and the calendar loads in two seconds.
To see whether your own pages answer, run a Site AI Audit with a free account. The plans unlock the full report with the list of affected pages.
Related reading
- Resource Limit Reached and 508 Errors on Shared Hosting
- Server Load Average and CPU Limits: Reading Hosting Stats
- Shared Hosting vs VPS: Which Is Faster for Your Website?
- Crawl Budget Explained: Does It Matter for Your Website?
The bottom line
Pages that did not respond gave the crawler no answer at all. Look for the pattern, try the pages yourself, read the logs, make sure nothing blocks the crawler and fix the real cause, then run the audit again.
DUK
How long does the crawler wait for a page?
It gives up when nothing arrives for 15 seconds, or when the whole transfer takes longer than 25 seconds.
Is a page that did not respond the same as a 404?
No. A 404 is an answer that says the page does not exist. A page that did not respond gave no answer at all.
Can the problem be on the crawler’s side?
Rarely. Short network problems happen, so run the audit again. If the same pages fail twice, the cause is on your side.
Could my firewall be the cause?
Yes. Firewalls and security plugins sometimes drop connections from automated visitors after a number of requests. The log will show it.
Should I worry about one or two pages?
Check them, but a single failure may be a short hiccup. A pattern or a growing number is the real signal.



