About the job
Join Jobgether's partner company as an AI Benchmark Engineer | Native Language Specialist in India.
This pivotal role merges software engineering, linguistic expertise, and AI assessment, concentrating on the creation of robust benchmarks to evaluate the multilingual capabilities of advanced language models. You will design and implement terminal-based tasks to analyze how AI systems manage non-English inputs, encoding challenges, and locale-specific behaviors in practical coding contexts. The position is highly experimental and research-driven, necessitating a blend of technical proficiency and linguistic accuracy. Responsibilities include crafting realistic multilingual datasets, pinpointing model failure points, and establishing comprehensive evaluation criteria. You will collaborate in a distributed, quality-centric environment, with a strong focus on precision, reproducibility, and systematic validation. Your work will have a direct impact on the measurement and enhancement of next-generation AI systems on a global scale.

