This one is closed
Live roles like this one
-
B
3h ago
Platform Tester (Back-end Systems) - Canada
Intelerad Canada CA$75k - CA$100k/yr
- C 19h ago
- D 19h ago
- C 19h ago
See every "AI Evaluators Assessing" role →
Get new “AI Evaluators Assessing” roles by email
One email a day with what is new in "AI Evaluators Assessing". Nothing new, no email.
We confirm the address first, and every mail carries an unsubscribe link. Alerts are ours, not a third party's.
Why this grade This listing scored 48/100, which is a D. It lost the most ground on pay transparency. See the breakdown
- Description depth 20 / 20 How much the posting actually says about the work, measured in characters of real text.
- Pay transparency 12 / 25 A published salary range, worth more than any other single factor because it is what a candidate cannot find out without applying.
- Freshness 8 / 15 How recently it was posted. Older postings are likelier to be filled or abandoned.
- Remote clarity 8 / 15 Whether "remote" means anywhere, or is quietly restricted to one country.
- Corroboration 5 / 10 Whether more than one source carries this listing.
- Role specificity 0 / 10 Whether the listing is tagged well enough to tell what the role actually is.
-5 Ghost-job penalty — Deducted for signals that this posting may not be a real, currently-open role — staleness, repeated relisting, or talent-pool language.
Every figure above is arithmetic over the posting itself — its salary field, its text, its age, its tags and how many sources carry it. How the grades work →
This listing does not state a salary
$80k – $121k
That is the middle half of what comparable roles paid on this board over the last 90 days — 16 listings that did publish a figure, median $99k. It is not this employer's offer, and we have no idea what they pay. It is only what the rest of the market advertised.
What We're Researching
We're hiring AI evaluators to assess the accuracy and helpfulness of a new digital shopping assistant. This project focuses on understanding how well the system handles real-world e-commerce queries and where it falls short in its logic. Your analysis will directly feed into improving the underlying model and its response quality.
How It Works
You will review real interaction traces between users and the shopping assistant within our custom platform. As you analyze these conversations, you will pinpoint specific failures, logical errors, or unhelpful product recommendations. From there, you will create structured rubrics and verifiers to consistently judge future response quality. This is an ongoing remote engagement requiring 20+ hours per week.
Who This Is For
This opportunity is ideal for quality assurance specialists, AI data evaluators, and e-commerce professionals with a strong eye for detail. We welcome applicants with prior experience in prompt engineering, complex data annotation, or software testing. You should be comfortable analyzing text interactions deeply and building structured evaluation frameworks from scratch.
What You'll Do
Review real user interaction traces with an AI shopping assistant
Identify logical failures, inaccuracies, or poor recommendations in the text
Create structured rubrics and verifiers to judge response quality
Commit to 20+ hours per week of evaluation work on our internal platform
Who Should Apply
Experience in data evaluation, quality assurance, or AI training
Strong analytical skills with the ability to spot subtle errors in text
Familiarity with e-commerce search and digital shopping experiences
Ability to commit to a sustained workload of 20+ hours per week
Compensation
$50 per hour
Ready to participate?
Start your paid interview now
About Terac
Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.
Learn more at or on YouTube at @jointerac.
Originally posted on Himalayas
Apply for this role Opens jobs.ashbyhq.com — verified as the employer's own application page
Quick question · anonymous · one tap
Would you apply to this job?
Answer to see what other job seekers said.
Your turn · no account needed
Help the next applicant
You may know something about this listing that we cannot see from here. One tap. No account needed. Signed-in reports earn points once the evidence agrees with you.
I know what it pays
Sign in with Google to earn points for reports — 100 confirmed points buy a week of Early Access.
Where this listing came from
- 03 Sep 2026 Himalayas first sighting
Seen on 1 board over 0 days.