A lightweight, no-fluff SEO crawler built for clarity, speed, and control.

What is Curious Badger?

Curious Badger is a Python-built SEO crawler designed to scan websites quickly and return only the stuff that matters — titles, metadata, image alt tags, canonical links, headers, schema, and error pages — all exported into clean, structured CSVs.

I built it to scratch my own itch: I needed a fast, local tool to audit site structure and content for projects, without the overhead (or cost) of enterprise SEO platforms.

What it does

  • Crawls sites quickly using multithreading or asyncio
  • Caches pages to avoid fetching duplicates
  • Skips broken link checks to keep things fast
  • Limits content size for better memory performance
  • Exports separate CSVs for:
    • Metadata (title, description, canonical, headers)
    • 4xx and 5xx errors
    • Missing image alt tags
    • Pages without H1s or canonical tags

Why I built it

I wanted something that gave me just enough — fast, transparent, and flexible. Curious Badger runs locally, doesn’t track anything, and outputs exactly what I need to make decisions (and nothing more).

It’s ideal for:

  • Spotting structural SEO issues
  • Content audits before redesigns
  • Lightweight QA for dev teams
  • Debugging client sites without loading up a heavy platform

Tech Stack

  • Python
  • requests, aiohttp, beautifulsoup4, concurrent.futures
  • CSV and JSON output
  • Local-first, no external dependencies or services

Want to try it?

I’m not open-sourcing Curious Badger (yet), but if you’d like a copy to run locally or want help adapting it to your needs, feel free to reach out:

📩 hello@studiojwd.com