AI Evaluator -Arabic | $11.5/hr Remote

Position
: LLM AI Quality Analyst (Personalization) Arabic
Type
: Short-Term Contract
Location
: Remote
Compensation
:$11.5 per hour
Commitment
: 30 40 hours/week, 4-hour overlap with PST
Engagement Length
: 2 months
Start Date
: Immediate
Role Responsibilities
Design multi-turn conversational prompts based on personal context
Evaluate personalized AI responses for relevance, grounding, and helpfulness
Assess correct and incorrect use of personal data in model outputs
Perform side-by-side (SxS) evaluation and ranking of AI responses
Identify grounding errors, poor inferences, and forced personalization
Write clear, structured rationales referencing specific conversation turns
Extract and verify model debug information and data source usage
Maintain strict data hygiene by deleting evaluation conversations
Requirements
Arabic fluency (reading and writing)
Strong experience in data annotati...