Software Engineer, Applied AI- Evals

Mercor is seeking a Software Engineer for the Evals product team to build scorecards, failure reports, and dataset views that help customers evaluate AI models. The role involves shipping APIs, data models, and UI, owning APEX benchmark launches, building failure analysis, and working directly with customers. The team runs evals at scale and grades results to explain model failures.

Bachelor's Degree in Computer Science or related field, strong full-stack skills in Python, Go, or TypeScript and React, track record of owning product features end to end, product sense, experience building data-heavy products, excellent communication, comfort with ambiguity, experience with LLM evals or AI developer tools is a plus, startup or B2B developer product experience is a plus.