Canonical Fold
Three duplicate URL rows send converging arrows into a single canonical card, which lights as the duplicates recede.
CSS
Crawlindex
15 animationsConditional Fetch
A conditional request carries an ETag to the origin and returns either as a full response body or as a slim not-modified reply.
CSSCrawl Budget
Three per-host budget meters clear at the crawl-window rollover and refill to the share each host is allowed.
CSSCrawl Frontier
A vertical queue of pending URL rows drains from the top into a fetch slot while a fresh row is discovered at the base, so the stack never empties.
CSSCrawl Rate
A host-load gauge needle swings across the dial while the request pulses beneath it run out at whatever rate it is reading.
CSSFetch Pool
Four parallel fetcher lanes each carry a request across to the origin on their own phase, so the pool is always partly in flight.
CSSIndex Coverage
A coverage ring sweeps round to its indexed ceiling, falls back as newly discovered URLs land, and climbs again.
CSSLink Extraction
A parsed page is read line by line and each href found drops out of the text into the outlink tray below.
CSSRecrawl Freshness
Staleness bars climb across a row of tracked pages while a recrawl head sweeps through and drops each one back to fresh.
CSSRender Pass
A page frame draws its raw HTML skeleton, then a second rendering pass paints the JavaScript-injected block over it before the cycle restarts.
CSSRetry Backoff
Retry hops march along an attempt rule at intervals that double until the host answers and the trace reaches it.
CSSRobots Directive
Path tokens travel a lane toward a directive gate arm that lifts to admit the allowed ones and stays down to turn the disallowed ones aside.
CSSShard Merge
Two columns of index segments slide apart and interleave back into one merged shard, then split again for the next merge.
CSSSitemap Claim
A matrix of submitted sitemap URLs is claimed row by row by a scanning head, filling as accent until the whole submission is taken up.
CSSURL Discovery
A discovery wave leaves the seed URL, follows the first ring of links and lights it, then pushes out to the next ring.
CSS