About the job
We encourage you to submit your CV in English and specify your level of English proficiency.
Mindrift specializes in connecting talented professionals with project-based AI opportunities for top-tier tech firms, focusing on the testing, evaluation, and enhancement of AI systems. This position is project-based and not a permanent role.
Opportunity Overview:
In this role, you will be responsible for developing intricate coding test scenarios that challenge AI coding systems to excel:
- Assess and refine practical coding tasks derived from existing production codebases with realistic scopes, requirements, and sources of information.
- Create detailed functional tests that verify genuine end-to-end functionality and edge cases, moving beyond mere superficial checks.
- Design “fair yet challenging” tasks where the AI has all the necessary context but must work diligently to piece together information from various files and external resources, requiring complex reasoning.
- Examine AI failures to discern the model's weaknesses and strengths.
- Refine your tests based on feedback from expert QA reviewers who evaluate your work according to seven quality criteria.
Desired Qualifications:
This opportunity is ideal for seasoned developers, software engineers, or test automation specialists who are open to part-time, non-permanent projects. The preferred candidates will possess:
- A degree in Computer Science, Software Engineering, or a related field.
- Over 5 years of experience in software development, primarily in Python (including pytest, async/await, subprocess, and file operations).
- A background in Full-Stack development, with a balanced proficiency in developing React-based interfaces and robust back-end systems.
- Experience in writing tests (functional, integration—not just executing them).
- Knowledge of Docker containers (for local evaluation tasks).
- Familiarity with CI/CD processes (using GitHub Actions for triggers, labels, and result analysis).
- English proficiency at a B2 level or higher.
How the Process Works:
Apply → Complete required qualifications → Join a project → Execute tasks → Get compensated.
Estimated Effort:
The tasks for this project are estimated to require around 20 hours to complete, depending on their complexity. This is an estimate and not a strict schedule; you have the flexibility to choose when and how you work. Tasks must be submitted by the deadline and meet the specified acceptance criteria to be approved.
Compensation:
- Paid contributions, with rates up to $50/hour*.
- Fixed project rates or individual rates based on project specifics.
- Some projects may offer additional incentive payments.
*Note: Rates may vary depending on expertise, skill assessment, location, project requirements, and other factors. Higher rates may be offered for highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment specifics will be provided per project.