Software Engineer, Data Mining

NationGraph is building the data and intelligence layer for the public sector. We are looking for a Software Engineer, Data Mining to own the infrastructure for acquiring public-sector information from across the internet at massive scale. You will build systems for creating, deploying, scheduling, monitoring, and maintaining hundreds of thousands of scrapers, design abstractions for scaling to millions of sources, handle messy government websites and legacy systems, use AI to automate scraping,

You are an unusually strong engineer who enjoys figuring out how things work. You have built production web crawlers, scraping systems, browser automation, or large-scale external data pipelines. You are strong in Python, Go, TypeScript, or another backend/systems language. You understand the realities of scraping modern websites including JavaScript rendering, sessions, rate limits, proxies, authentication, and changing schemas. You understand distributed systems including orchestration, queues