तकनीकी SEO
robots.txt
यह क्यों ज़रूरी है
robots.txt controls which parts of a site crawlers may visit. A wrong Disallow rule can block important pages from Google; a missing file is fine but wastes the chance to list your sitemap.
इसे कैसे ठीक करें
- Make sure no Disallow rule blocks pages that should rank.
- Add a “Sitemap:” line pointing to your XML sitemap.
- Test changes before publishing them.
इससे जाँचें
- Robots.txt Tester – Fetch and validate robots.txt, and test whether any URL is blocked for Googlebot or AI bots.
- Robots.txt Generator – Build a robots.txt file with sitemap, blocked folders and AI crawler controls.
स्कोर: पास होने वाली जाँच को पूरा भार मिलता है, चेतावनी को आधा और त्रुटि को कुछ नहीं। SEO स्कोर कैसे काम करता है