Robots.txt Validator & Tester
robots.txt 파일이 특정 URL 및 사용자 에이전트를 허용 또는 차단하는지 테스트
robots.txt Content
Test URLs
Robots.txt Tester & Validator
Use this free robots.txt tester, validator, and checker to see whether a URL is allowed or blocked for Googlebot, Bingbot, or any custom crawler. Paste your robots.txt, enter a URL, choose the User-agent, and get an instant rule-by-rule result.
How to test your robots.txt
- 1
Paste your robots.txt
Copy the full content from your live /robots.txt file or draft rules and paste them into the tester.
- 2
Enter a URL to test
Provide the exact path you want to check, such as /admin/dashboard, /products/item, or a full URL.
- 3
Choose a User-agent
Select Googlebot, Bingbot, Googlebot-Image, or type a custom bot name to simulate how that crawler reads the file.
- 4
Read the matched rule
The tool shows the specific Allow or Disallow rule that wins, whether crawling is allowed, and why.
What the robots.txt checker validates
Allow vs Disallow
See which directive matches the longest path and whether it permits or blocks the tested URL.
Crawler-specific rules
Compare rules for Googlebot, Bingbot, all crawlers (*), or custom bots before publishing changes.
Syntax issues
Catch malformed directives, empty groups, unexpected spacing, and confusing rule order.
Crawl, not index
Understand whether the URL can be crawled. Use noindex separately when the page should not appear in search.
FAQ
What is a robots.txt validator?
It checks your file for syntax errors and confirms rules work as intended for specific URLs and bots.
How do I check if Googlebot can crawl a page?
Set User-agent to Googlebot, enter the URL path, paste your robots.txt — the tester shows if Googlebot can crawl it.
Why does a URL show as blocked?
A matching Disallow rule may be more specific than your Allow rule, or the URL may be inside a blocked folder. Check the matched rule shown by the tester.
Why test robots.txt before deploying?
Manual rules are easy to get wrong. A tester prevents you from accidentally blocking important pages from Google.
계속 탐색
추천 개발자 도구 도구…
JSON 포맷터
구문 강조 및 오류 감지로 JSON 형식화, 유효성 검사 및 압축
Base64 인코더/디코더
텍스트 또는 파일을 Base64로 인코딩하고 Base64 문자열을 읽기 가능한 텍스트로 디코딩
URL 인코더/디코더
URL 구성 요소 및 쿼리 문자열 매개변수 인코딩 및 디코딩
UUID 생성기
무작위 UUID(v1, v4) 생성 또는 여러 UUID 일괄 생성
해시 생성기
텍스트 또는 파일에서 MD5, SHA-1, SHA-256, SHA-512 해시 생성
정규식 테스터
실시간 일치 강조 표시로 정규식 테스트
JWT 디코더
JSON Web Token 디코딩 및 검사 — 헤더, 페이로드 및 서명 확인
HTML 포맷터
적절한 들여쓰기와 구문 강조로 HTML 코드 형식화 및 정리