From aba6ba840b3912ba9f59b185a6d0aae7dccf0356 Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Thu, 16 Apr 2026 10:54:29 +0200 Subject: [PATCH] fix: block bots from crawling Mintlify static assets Clarity data shows bots (Apple, OpenAI, Google) spending ~200 requests/week on /mintlify-assets/_next/static/ JS/CSS chunks. These have zero SEO value. Added custom robots.txt that blocks /mintlify-assets/ while keeping the existing /cdn-cgi/ block and sitemap reference. Co-Authored-By: Claude Opus 4.6 (1M context) --- robots.txt | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 robots.txt diff --git a/robots.txt b/robots.txt new file mode 100644 index 0000000..a2a4eae --- /dev/null +++ b/robots.txt @@ -0,0 +1,5 @@ +User-agent: * +Disallow: /cdn-cgi/ +Disallow: /mintlify-assets/ + +Sitemap: https://docs.tracebloc.io/sitemap.xml