Site crawl

Glossary

Site crawl

A read of a company's own website, page by page, as an AI engine would reach it. It shows how readily an engine can reach, read, and quote the site.

What it tells a buyer or a marketer

AI engines read websites differently from people. They fetch pages, read the text and the data behind it, and quote what they can make sense of. A site crawl reads a company's own website the same way, page by page, to show what an engine would find.

For a marketer, the crawl answers a practical question: can an engine reach, read, and quote this site? A site can look polished to a visitor and still give an engine very little, because the facts sit in images, behind scripts, or on pages the engine cannot reach.

How it appears on the AI Authority Index™

Source is the factor that reads a company's own site, and the site crawl is how that reading is made. It asks whether the content a company owns stands up as the record of what is true about it, and whether an engine can use it.

The free AI Authority Check runs a live site crawl on the company it reads, so its Source reading reflects the site as it stands that day.

What moves it

Clear pages move it. A site helps itself when its facts are written in plain text, when each page says what the company does and for whom, and when its details match what other sources say. Pages an engine cannot reach, thin pages, and pages that contradict each other all count against it. A site update of this kind helps the engines describe a company accurately. Who they recommend depends more on what independent sources say.

Why Third Hemisphere® defines it

Third Hemisphere runs the site crawl with its own technology, reading each page as an AI engine would reach it. The same crawl serves every company, so one site's reading can be set beside another's.

Run the free AI Authority Check for a live crawl of your own site.

Related terms

A site crawl is how Source is read, while cited sources are what Influence reads. The free AI Authority Check runs a live site crawl, and it shows what an AI engine would find on a company's own site.

See also

Run the free AI Authority Check

Questions

What is a site crawl?

A read of a company's own website, page by page, as an AI engine would reach it. It shows how readily an engine can reach, read, and quote the site.

Which CRISP™ factor uses the site crawl?

Source, which reads whether a company's own site stands up as the record of what is true about it.

Does the free AI Authority Check crawl my website?

Yes. The Check reads Source from a live site crawl of the company it is run on.

Defined facts

  • Defined by Third Hemisphere Pty Ltd (ABN 60 616 085 673).
  • First published by Third Hemisphere on 16 October 2026.
  • Used in the AI Authority Index™ since 16 October 2026.
  • Last updated 16 November 2026.

Questions about a term, or about where your own company stands? Book a discovery call.

Want to measure and improve these yourself? Do it yourself with our tools.

Written and maintained by the Third Hemisphere team.

© 2026 Third Hemisphere Pty Ltd. All rights reserved. The definitions, the CRISP™ method, and the AI Authority Index™ are the copyright of Third Hemisphere Pty Ltd.

Third Hemisphere® and the Third Hemisphere logo are registered trade marks of Third Hemisphere Pty Ltd (Reg. No. 2005138, 2005141).

CRISP™ and AI Authority Index™ are trade marks of Third Hemisphere Pty Ltd.