Contribute human expertise to AI training, evaluation or data work. Check the official source for project requirements.
About the role
Project Overview
We are sourcing independent Information Specialists to provide their expertise for an AI benchmark evaluation project. As AI models increasingly generate professional-grade data management strategies, information retrieval systems, and content architecture deliverables, their accuracy relies entirely on robust, expert-crafted training data. The objective of this project is to autonomously produce high-quality evaluation tasks, strong prompts, and clear, well-structured rubrics that generate clean, reliable data for model training.
Project Deliverables & Scope
Operate autonomously to design complex evaluation frameworks and provide structured training data. Expected deliverables include:
Task & Prompt Creation: Generating realistic, high-quality prompts that compel the AI model to produce complex, professional-grade deliverables specific to information science, data management, and knowledge organization.
Rubric Development: Writing clear, well-structured evaluation rubrics with criteria that are highly specific, non-ambiguous, and easy to score.
Benchmark Evaluation Data Generation: Producing clean, reliable training data that directly aids in the evaluation and refinement of AI models handling complex information architecture and retrieval tasks.
Quality Assurance & Fact-Checking: Ensuring all generated tasks and scoring criteria reflect strict, real-world data governance standards, research methodologies, and information management best practices.
Required Expertise
To successfully fulfill the deliverables of this project, Contractors must possess deep industry knowledge to craft realistic professional scenarios.
Core skillset includes:
Demonstrable professional expertise within the information science, library science, data governance, or knowledge management sectors, with a deep understanding of industry standards, metadata schemas, and search methodologies. …
Official source: job-boards.eu.greenhouse.io. Details can change; the platform's page is the final word. How we check listings.
