About the job
Backend Engineer - New Graduate Opportunity
Join LiteLLM, the leading AI Gateway, trusted by renowned organizations such as Adobe, Netflix, and NASA. Our innovative platform provides developers with secure and reliable access to Large Language Models (LLMs) and related services. We are seeking a passionate Backend Engineer (New Grad) to contribute to building robust guardrails and observability tools at an extensive scale.
Role Overview
In this role, you'll be instrumental in enhancing our guardrails and logging mechanisms. You will take ownership of the backend code ensuring that all guardrail calls are accurately logged, errors are made visible to users, and our observability tools are effective under high-traffic conditions. Your meticulous attention to detail in latency metrics, logging traceability, and backend guardrail registration will significantly influence user trust in our security and compliance features.
Key Responsibilities
Ensure all guardrail and policy enforcement calls (e.g.,
applyguardrail) are logged and traceable within our SpendLogs and relevant database tables.Proactively identify and resolve silent failures in guardrail creation, registration, and policy application, ensuring robust error handling and clarity for end-users.
Collaborate with observability integrations, including Datadog, Splunk, Prometheus, and OpenTelemetry, to maintain effective monitoring and logging for backend systems.
Refactor and enhance our Prometheus integration to facilitate configurable latency histogram buckets that can scale for high-traffic environments.
Work collaboratively across teams on backend engineering priorities such as performance, reliability, and security.
Qualifications
Recent graduate with a Bachelor’s or Master’s degree in Computer Science or a related field.
Proficient in Python and familiar with backend frameworks like FastAPI or Flask.
Knowledge of logging best practices, error handling, and secure backend development principles.
Exposure to monitoring and logging platforms such as Datadog, Splunk, Prometheus, or OpenTelemetry.
Familiarity with database integration and troubleshooting (e.g., PostgreSQL, Redis).
A strong drive to deliver high-quality backend code with attention to detail.