CitinoBot

What it is

CitinoBot is the crawler of Citino, an AI search analytics product operated by Tambe Technologies. Its user agent is CitinoBot/1.0 (+https://citino.ai/bot).

It checks which AI crawlers a website's robots.txt allows, for example GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and Googlebot. Citino customers use this to see whether their own site, and competitor sites they choose to track, can be read by the crawlers behind AI answer engines.

What it fetches

Only one file per site: https://<domain>/robots.txt. It never requests any other page, never follows links, never runs JavaScript and never submits forms. It follows up to five redirects to reach robots.txt, for example from example.com to www.example.com, and nothing else.

How often

About once a day per site, for each Citino project that tracks that site. Occasionally once more, when a customer adds the site or asks for a re-check, and at most once every ten minutes per project.

How it behaves

• It identifies itself honestly as CitinoBot. It never pretends to be another crawler, such as GPTBot or Googlebot.
• Every request is a plain HTTPS GET, with no cookies, logins or credentials.
• It gives up after 15 seconds and reads at most the first 512 KB of the file.
• It only fetches public sites. It refuses private and internal network addresses.

What happens to the file

The robots.txt text is stored so the Citino customer can see which crawlers it allows, and when it changed. Only a few recent versions are kept. It is not used to train AI models, and nothing else from the site is collected.

Questions about CitinoBot? Write to support@citino.ai.