DESCRIPTION
The primary goal of Content Reviewers (Raters) is to evaluate various health related prompts, make individual judgments and decisions based on predefined options from task guidelines to support machine learning project objectives. Your feedback will directly inform positive and negative changes to the underlying AI models and contribute to the refinement of evaluation guidelines.
Key Responsibilities:
- Thoroughly read and understand a new set of evaluation guidelines, including specific criteria and examples for assessing model output.
- Evaluate the overall quality of responses based on provided criteria, focusing on areas such as:
- Ability to answer user questions (including identifying explicit refusals/punts).
- Helpfulness of responses in meeting user needs and providing actionable information.
- Understanding and interpretation of user queries.
- Factuality of information, including identifying hallucinations or fabricated knowledge.
- Provision of recommendations for health and wellness improvement or general health knowledge.
- Comprehensiveness (clarity and directness) in addressing all aspects of a query.
- Utilization of personal user data ( metrics, demographics, memories) in responses.
- Extraction and correct use of information from personal data.
- Overall sentiment and appropriateness of tone in messages.
- Overall quality and room for improvement of the response/model.
- Appropriateness of response length to the query.
- Cultural and regional appropriateness of the response (terminology, units, references, tone).
- Document any questions, concerns, or issues encountered while applying the guidelines to the new model output.
- Provide clear, concise, and constructive comments directly on the guidelines to facilitate fast-follow updates.
- Participate in calibration sessions with team members, other locations, or clients as instructed.
- Collaborate with team members to share best practices.
- Work effectively both independently and as part of a team.
- Synthesize evaluation findings into actionable recommendations for modeling and product teams.
- Communicate clearly with internal and external contacts as required.
- Manage multiple assignments concurrently.
- Adhere to Tech Mahindra and client procedures, policies, and guidelines.
- Follow and adhere to all KPI-related tasks to prevent business impact.
- Ensure breaks are pre-approved by the SME, Team Leader, or Managers.
- Comply with all instructions from the Team Leader & superiors.
- Adhere to all process-specific policies.
- Maintain confidentiality for all process-related work and critical details.
- Undertake any additional ad hoc tasks that contribute to improved client service or KPI enhancement.
- Identify opportunities that would help enhance the existing workflow.
REQUIREMENTS
- Language Proficiency: Expert written and verbal skills in the designated (AU & EN) language, including fluency in both writing and speaking.
- Cultural Affinity: Must be a native of the country where the selected language originated.
- Computer Skills: Basic computer proficiency with a typing speed of 35-40 Words per minute.
- Excellent verbal and written communication abilities.
- Strong analytical skills and attention to detail in scoring, identifying errors, and providing clear written justifications.
- Ability to multitask and adapt to rotational shifts.
- Familiarity with general health and wellness concepts & terms (specific medical knowledge is not required, but a general understanding is beneficial).
- Familiarity with common wearable device metrics (e.g., steps, heart rate, sleep duration) is a plus.
- Experience in content review, data annotation, or quality assurance is advantageous.
- A Must: Expertise of Australian culture, familiarity with local standards, places, important numbers, locally used conversion units (for eg miles and not KMs) like , metrics, slangs, nuances, festivals which are not appropriate for a user in Australia .
- Under-graduate or Graduate in any stream
OFFER
- Meal Allowance - € 9,13 (per working day)
- Health Insurance
- Transport Allowance or Shuttle
