About the job
About the Role
Maincode is at the forefront of AI innovation, creating foundational models built from the ground up on Australian infrastructure. We specialize in designing architectures, managing our own computing resources, optimizing training processes, and deploying robust systems that power our models.
We are proud to have developed Matilda, the first large language model entirely built and trained in Australia. Our new compute cluster is operational, and we're excited to scale the next iteration of Matilda, making it accessible for public use.
We invite passionate AI researchers to join us in refining the core architecture, training, and evaluation of the large-scale language models that drive Matilda's capabilities.
This position emphasizes substantial contributions over incremental benchmarks. You will collaborate closely with engineers who manage large-scale training systems, focusing on designing models that learn efficiently and perform reliably in real-world applications.
Your Responsibilities
As part of our team, you will engage throughout the model development cycle, addressing research questions, conducting training runs, and performing evaluations. Your key tasks will include:
- Designing and testing architectural modifications and training methodologies for large language models.
- Executing controlled experiments at scale to isolate causal impacts.
- Investigating failure modes related to reasoning, generalization, robustness, and representation.
- Defining objectives, data mixes, and optimization strategies that significantly affect model behavior.
- Creating and enhancing evaluations that measure capabilities and reliability, beyond mere numerical scores.
- Analyzing training dynamics through logs, metrics, and model outputs.
- Partnering with machine learning systems engineers on distributed training and operational tasks.
- Documenting insights and experimental outcomes in clear internal notes to inform design decisions.
You will spend a considerable amount of time developing code, running training sessions, analyzing logs, and evaluating outputs to ensure clarity on what enhances the model's performance and the rationale behind it.
Qualifications
We value deep analytical reasoning, disciplined experimentation, and the capability to navigate ambiguity effectively. Ideal candidates will possess:
- Hands-on experience in developing and executing production-level machine learning or research code.
- Proficiency in Python, with experience in frameworks such as PyTorch or JAX.
- A strong foundation in transformer architectures and large-scale language models.
- Familiarity with distributed computing environments and data processing techniques.
- Excellent problem-solving skills and the ability to work collaboratively within a team.
