# Everything here is meant to be found — this host is the public storefront. # # The application lives on app.aeviamodeler.ai and is kept OUT of the index a # different way: an `X-Robots-Tag: noindex` header on every response (see the # Caddyfile). That is deliberate and is NOT a job for a `Disallow` rule. # Disallow stops a crawler FETCHING a URL, which means it never reads the # noindex header — and a disallowed URL that something links to can still be # listed, URL-only, with no description. To keep a page out of an index you # must let the crawler in and tell it not to index. So: no rules here for the # app host, and none needed. User-agent: * Allow: / Sitemap: https://aeviamodeler.ai/sitemap.xml