AI Evaluation Specialist
micro1 Australia, Canada, Ireland, New Zealand, United Kingdom, United States
This one is closed
Live roles like this one
-
D
7h ago
Digital Science United States
-
B
7h ago
Upstart Anywhere in the World Salary Range $141,000
- B 9h ago
- D 19h ago
See every "AI Evaluation Specialist" role →
Get new “AI Evaluation Specialist” roles by email
One email a day with what is new in "AI Evaluation Specialist". Nothing new, no email.
We confirm the address first, and every mail carries an unsubscribe link. Alerts are ours, not a third party's.
Why this grade This listing scored 45/100, which is a D. It lost the most ground on pay transparency. See the breakdown
- Description depth 20 / 20 How much the posting actually says about the work, measured in characters of real text.
- Freshness 12 / 15 How recently it was posted. Older postings are likelier to be filled or abandoned.
- Remote clarity 8 / 15 Whether "remote" means anywhere, or is quietly restricted to one country.
- Corroboration 5 / 10 Whether more than one source carries this listing.
- Role specificity 0 / 10 Whether the listing is tagged well enough to tell what the role actually is.
- Pay transparency 0 / 25 A published salary range, worth more than any other single factor because it is what a candidate cannot find out without applying.
Every figure above is arithmetic over the posting itself — its salary field, its text, its age, its tags and how many sources carry it. How the grades work →
Role Title: AI Evaluation Specialist
Role Type: Contractor
Location: Remote (US, CA, UK, IE, AU, NZ)
micro1 is engaging AI Evaluation Specialists to assess and elevate the quality of AI assistant outputs for an enterprise AI training initiative. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.
Scope of Work
- Evaluate AI-generated outputs against detailed rubrics and defined quality standards, focusing on accuracy, relevance, and adherence to guidelines.
- Apply consistent, impartial judgment across a high volume of examples, ensuring a fair and reliable assessment process.
- Identify reasoning gaps, tool-use failures, or logic errors in AI assistant responses, providing actionable feedback for iterative improvement.
- Produce clear, concise written feedback on both strengths and areas for improvement, directly influencing model refinement and AI adoption practices.
- Participate in discussions regarding rubric interpretation and evolving quality standards, contributing to process optimization and best practices.
- Maintain meticulous documentation of evaluations and recommendations, ensuring transparency and traceability in assessment workflows.
Preferred Qualifications
- Experience in grading, quality assurance, editorial review, assessment, annotation, or similar fields demanding careful analysis and detailed feedback.
- Advanced, daily use of AI assistants (such as ChatGPT, Claude, or similar) as an essential work and productivity tool.
- Demonstrated ability to synthesize complex information and communicate findings effectively in writing.
- Background in process improvement, rubric development, or operational quality assessment in an enterprise or educational context.
- Strong critical thinking skills with a focus on consistency, integrity, and fairness in evaluations.
- Comfort working independently on large volumes of similar examples while maintaining high attention to detail.
- Collaborative mindset for sharing insights, discussing ambiguous cases, and refining evaluation criteria as models evolve.
Originally posted on Himalayas
Apply for this role Opens himalayas.app — the link as listed; we have not yet verified it is the employer's own page
Quick question · anonymous · one tap
Would you apply to this job?
Answer to see what other job seekers said.
Your turn · no account needed
Help the next applicant
You may know something about this listing that we cannot see from here. One tap. No account needed. Signed-in reports earn points once the evidence agrees with you.
I know what it pays
Sign in with Google to earn points for reports — 100 confirmed points buy a week of Early Access.
Where this listing came from
- 12 Sep 2026 Himalayas first sighting
Seen on 1 board over 0 days.