This one is closed
Live roles like this one
-
B
3d ago
Supabase Anywhere in the World
See every "AI Rubric Writer" role →
Get new “AI Rubric Writer” roles by email
One email a day with what is new in "AI Rubric Writer". Nothing new, no email.
We confirm the address first, and every mail carries an unsubscribe link. Alerts are ours, not a third party's.
Why this grade This listing scored 34/100, which is an F. It lost the most ground on pay transparency. See the breakdown
- Description depth 20 / 20 How much the posting actually says about the work, measured in characters of real text.
- Remote clarity 15 / 15 Whether "remote" means anywhere, or is quietly restricted to one country.
- Corroboration 5 / 10 Whether more than one source carries this listing.
- Freshness 4 / 15 How recently it was posted. Older postings are likelier to be filled or abandoned.
- Role specificity 0 / 10 Whether the listing is tagged well enough to tell what the role actually is.
- Pay transparency 0 / 25 A published salary range, worth more than any other single factor because it is what a candidate cannot find out without applying.
-10 Ghost-job penalty — Deducted for signals that this posting may not be a real, currently-open role — staleness, repeated relisting, or talent-pool language.
Every figure above is arithmetic over the posting itself — its salary field, its text, its age, its tags and how many sources carry it. How the grades work →
Role Title: AI Rubric Writer
Role Type: Contractor
Location: Remote
We are seeking an experienced AI Rubric Writer to develop high-quality evaluation rubrics and scoring frameworks for AI and Large Language Model (LLM) assessments. The ideal candidate will create clear, objective evaluation criteria to assess AI-generated responses and help improve model performance across a variety of subject areas.
Responsibilities
- Design and develop evaluation rubrics and scoring guidelines for AI-generated content.
- Create objective, consistent, and well-structured assessment criteria.
- Evaluate AI responses for accuracy, reasoning, completeness, and quality.
- Provide detailed feedback to improve AI model performance.
- Collaborate with cross-functional teams to refine evaluation standards.
- Ensure rubrics are clear, unbiased, and easy to apply.
Required Qualifications
- Bachelor's degree or higher in a relevant field.
- Strong analytical, critical thinking, and written communication skills.
- Experience creating evaluation criteria, scoring frameworks, assessment guidelines, or rubrics.
- Subject matter expertise in Software Engineering, Finance, Law, Consulting, or other quantitative/STEM disciplines.
Preferred Qualifications
- Experience as an AI Evaluator, Prompt Evaluator, or similar AI quality assessment role.
- Experience evaluating Large Language Models (LLMs) or Generative AI systems.
- Familiarity with rubric development, AI quality assurance, or data annotation.
- Experience reviewing and assessing technical or domain-specific content.
Originally posted on Himalayas
Apply for this role Opens himalayas.app — the link as listed; we have not yet verified it is the employer's own page
Quick question · anonymous · one tap
Would you apply to this job?
Answer to see what other job seekers said.
Your turn · no account needed
Help the next applicant
You may know something about this listing that we cannot see from here. One tap. No account needed. Signed-in reports earn points once the evidence agrees with you.
I know what it pays
Sign in with Google to earn points for reports — 100 confirmed points buy a week of Early Access.
Where this listing came from
- 05 Aug 2026 Himalayas first sighting
Seen on 1 board over 0 days.