Technical eligibility · Branch T.A, Crawl, HTTP and indexable content
T-01 Public URLs must be technically reachable and carry real main content
A public URL intended for indexing must not be blocked by an access barrier, must return a successful HTTP response and must carry real main content that crawlers can see.
- Scope
- All public content
- Priority
- Gate. A FAIL stops publish or launch.
- Quality bucket
- Technical eligibility
- Legacy coverage
- SEO-31, SEO-32, SEO-33
Why it matters
A page that crawlers cannot reach, or that returns an error or an empty shell, cannot be indexed or used in answers, however good the content is meant to be. Visitors can also be turned away by blocks they never see. Google's technical requirements and JavaScript guidance exist because crawlers must be able to see the real main content.
How to implement
- Test robots.txt and check noindex and X-Robots headers on every URL meant for indexing.
- Inspect key URLs in Search Console and confirm Googlebot access in server and edge logs.
- Remove incorrect blocks based on user agent or country, and review WAF events.
- Crawl every sitemap URL, fix error status codes, identify soft 404s and remove unnecessary redirect chains.
- Check the rendered HTML and keep critical copy, prices, product names and links in the accessible DOM.
Checklist
- Test the robots.txt rules.
- Use Search Console URL Inspection.
- Verify Googlebot access in server and edge logs.
- Remove incorrect blocks based on user agent or country.
- Crawl every URL in the sitemap and check its status code.
- Identify soft 404s.
- Remove unnecessary redirect chains.
- Check the rendered HTML.
- Keep critical copy, prices, product names and links in the accessible DOM.
- Verify noindex and X-Robots headers.
How to audit
- Check URL Inspection, server logs and WAF events together.
- Run a sitemap crawl and compare it with the Search Console indexing status.
- Compare the Search Console rendered HTML and screenshot with a real browser.
How UltimateSEO checks this automatically
Automated. Checked automatically: HTTP status of every crawled URL, soft-404 heuristics, redirect chains longer than two hops and the presence of real main content.
States used: PASS, FAIL, UNKNOWN, NOT_APPLICABLE or BLOCKED. UNKNOWN is never shown as a pass. How states work.
Sources
Part of The UltimateSEO Framework v4.1, revised 2026-10-07. Evidence states, control states and the publish gate are explained on the hub.