Staff Software Engineer, Observability at Databricks | Mountain View, CA

Join Databricks as a Staff Software Engineer focused on Backend development, where you'll play a crucial role in enabling data teams to tackle some of the world's most challenging problems, ranging from security threat detection to cancer drug development. Our mission is to build and operate the world's leading data and AI infrastructure platform, allowing our customers to concentrate on the high-value challenges central to their missions.Founded in 2013 by the original creators of Apache Spark, Databricks has rapidly expanded from a small office in Berkeley, California, to a global powerhouse with over 1,000 employees. Our platform is trusted by thousands of organizations, from startups to Fortune 100 companies, making us one of the fastest-growing SaaS companies worldwide.As part of our engineering team, you'll develop and maintain highly technical products that address significant real-world needs. We constantly push the limits of data and AI technology while ensuring the resilience, security, and scalability essential for our customers' success.Our software operates at an impressive scale, managing millions of virtual machines and processing exabytes of data daily. We frequently encounter cloud hardware, network, and operating system faults, and it is vital that our software effectively shields customers from these issues.As a Backend Software Engineer, you'll collaborate closely with your team and product management to prioritize, design, implement, test, and operate micro-services for the Databricks platform. Your responsibilities will include writing software in Scala/Java, building data pipelines using Apache Spark and Apache Kafka, integrating third-party applications, and interacting with cloud APIs such as AWS and Azure.Possible teams you could join include:Data Science and Machine Learning Infrastructure: Develop services and infrastructure that blend machine learning and distributed systems, empowering collaborative workspaces, notebooks, IDE integrations, and project management tools.Compute Fabric: Create resource management infrastructure that supports big data and machine learning workloads in a robust, flexible, secure, and cloud-agnostic manner, managing millions of virtual machines.

P-97 At Databricks, we are dedicated to empowering data teams to tackle the most significant challenges facing the world today. We achieve this by creating and managing the premier data and AI infrastructure platform, enabling our clients to harness profound data insights to enhance their operations. Our commitment to innovation drives us to constantly explore the edges of data and AI technology while maintaining the resilience, security, and scalability essential for our customers to thrive on our platform. With one of the largest software platforms globally, our infrastructure comprises millions of virtual machines, producing terabytes of logs and processing exabytes of data daily. Given our scale, we routinely encounter faults in cloud hardware, networks, and operating systems, and our software must adeptly shield our clients from these issues. In the role of Performance Engineer, you will collaborate extensively with various teams across the organization to assess the performance of products and features. You will identify performance bottlenecks and partner with engineers to address performance and scalability challenges. This includes establishing performance targets for different software releases, guiding teams in developing performance benchmarks, conducting competitive benchmark analyses for various Databricks products, and performing in-depth analyses to diagnose and rectify performance issues.

Join Databricks as a Lead Security Architect and play a pivotal role in fortifying our IT infrastructure. In this fast-paced environment, you will design and implement a secure and scalable architecture to safeguard our corporate assets. Your focus will encompass crucial aspects of IT security, including Identity and Access Management (IAM), Zero Trust architecture, and endpoint security, all while ensuring the protection of critical business applications and sensitive data.Your deep expertise in security principles and comprehensive understanding of the entire infrastructure stack will be invaluable in crafting robust security solutions that are future-ready. You will help enhance the resilience and integrity of our systems against the ever-evolving cyber threat landscape.This position is critical for shaping our security strategy across various modern platforms including AWS, Azure, GCP, and SaaS solutions. You will establish a strong least privilege (PoLP) model, provide specialized IAM expertise, and securely support SaaS with sensitive information. Additionally, you will contribute to our internal strategy for secure AI development.Collaboration with cross-functional teams is essential to embed security throughout the technology stack, ensuring that our internal strategies align with enterprise security standards.

At Databricks, we are dedicated to empowering data teams to tackle the world's most challenging problems—from revolutionizing transportation to accelerating medical breakthroughs. Our mission is realized by providing the world’s premier data and AI infrastructure platform, enabling customers to harness deep data insights for business enhancement. Our Search technologies are integral to this mission, facilitating keyword retrieval, semantic similarity through vector embeddings, and hybrid solutions that blend both methods. These capabilities enable users to discover and comprehend information across vast, intricate datasets, powering applications such as Retrieval Augmented Generation (RAG), AI assistants, recommendation systems, enterprise knowledge management, in-product search, and data exploration. As a Senior Engineering Manager for Search, you will be pivotal in designing and scaling our next-gen Search product, which includes Vector Search, Keyword Search, and Search Quality. You will lead the development of a robust, high-performance, cost-effective, and developer-friendly Search stack, equipping customers to create AI applications with precision, speed, and scalability. In addition to hands-on contributions, you will help define the long-term vision, mentor senior engineers, collaborate across teams, and spearhead strategic initiatives that enhance the technical and business impact of Search at Databricks.

P-57 At Databricks, we are passionate about empowering data teams to tackle the world's most challenging problems. Our mission is to create and operate the premier data and AI infrastructure platform, allowing our clients to concentrate on the high-value challenges central to their objectives. Founded in 2013 by the original creators of Apache Spark, Databricks has expanded from a small office in Berkeley, California, to a global powerhouse with over 1,000 dedicated employees. Thousands of organizations, from small startups to Fortune 100 companies, rely on Databricks for their critical workloads, establishing us as one of the fastest-growing SaaS companies globally. Our engineering teams are committed to developing highly technical products that meet significant needs in the industry. We continuously push the limits of data and AI technology while ensuring resilience, security, and scalability—key factors in ensuring our customers' success on our platform. We manage one of the largest software platforms in existence, comprising millions of virtual machines that generate terabytes of logs and process exabytes of data daily. At this scale, we frequently encounter cloud hardware, network, and operating system issues, necessitating that our software effectively shields our customers from such challenges. As a Data Scientist on the Data Team, you will play a crucial role in fostering a data-driven culture at Databricks by working on the company's top priorities. The Data Team also acts as an internal customer, utilizing Databricks to shape the future direction of our products. Your Impact: Analysis at the speed of thought: Enhance decision-making capabilities by developing robust data science tools for business leaders, analysts, and fellow data scientists. Extend capabilities of Databricks: Collaborate closely with the Data Platform and Product Engineering teams to integrate data science tools with existing offerings and the core product. Strategic business insights: Spearhead insight generation for key company priorities and essential engineering initiatives focusing on reliability and efficiency. Gather evolving requirements, define project OKRs and milestones, and communicate progress and results to both technical and non-technical stakeholders. Mentor and guide junior data scientists on the team regarding project planning, technical decisions, and reviews of code and documentation. Represent the data science discipline across the organization, advocating for a more data-driven approach. Act as a representative for Databricks at academic and industry events.

Join our innovative team at dev2 as a Backend Software Engineer. In this role, you will have the opportunity to work on cutting-edge technologies and develop scalable applications that make a difference. Your expertise will contribute to our mission of delivering high-quality software solutions.

About Matic RoboticsAt Matic Robotics, we are dedicated to transforming home life by introducing cutting-edge robotics technology. Each year, billions of hours are squandered on mundane household chores. Our mission is to reclaim that time, making home management effortless and efficient.Our flagship product, Matic, is an innovative floor-cleaning robot that combines advanced features akin to 'full self-driving' technology for homes. With real-time 3D mapping, adaptive path tracking, and an intelligent understanding of its environment, Matic operates seamlessly using only RGB cameras and on-device neural networks.We prioritize privacy; all data processing is conducted on the robot, ensuring that what happens in your home stays in your home.Our approach to home robotics mirrors the evolution of consumer electronics: starting with specialized devices and gradually evolving into more sophisticated solutions.At Matic, we foster a collaborative and hardworking culture focused on creating products that our customers truly value.Role OverviewBackend Software EngineerJoin our dynamic team as a Backend Software Engineer, where your contributions will have a significant and immediate impact. You will collaborate with cross-functional teams including software, mechanical, and electrical engineers to develop internal tools.Key ResponsibilitiesDesign and enhance internal tools that facilitate and optimize our production processes.Create robust systems in Rust to elevate manufacturing efficiency and reliability.Work collaboratively with Mechanical and Electrical Engineering teams to translate tooling requirements into functional solutions.Identify opportunities for tool creation and infrastructure improvements to enhance cross-team operations.Become a vital member of our compact software team.Qualifications2-5 years of experience in backend software development.Proficiency in Rust programming language.Strong problem-solving skills and attention to detail.A collaborative spirit and a passion for building innovative solutions.

About Us At Otter, we empower restaurateurs to thrive in the digital food delivery landscape. Our innovative platform is trusted by globally recognized brands like Chick-fil-A, Ben & Jerry’s, KFC, and Eataly. We are passionate about enhancing the dining experience and simplifying online ordering for restaurants of all sizes. Your Role As a Backend Engineer, you will play a pivotal role in designing, implementing, and optimizing our core systems that drive critical applications. Your responsibilities will adapt to your unique strengths and interests, allowing you to focus on: Infrastructure: Build and maintain secure, scalable, and high-performance systems to ensure operational excellence. Platform: Develop robust APIs and developer tools that enhance team efficiency and scalability. Product: Engineer backend services that support customer-facing applications for a seamless user experience. Key Responsibilities: Architect and implement scalable backend systems and APIs. Ensure high performance, reliability, and security of systems. Collaborate with product managers, designers, and fellow engineers to deliver comprehensive solutions. Engage in code reviews, testing, and documentation to uphold quality standards. Mentor junior engineers or take on leadership roles as per your expertise.

The OpportunityJoin our dynamic team at Otter.ai as a Backend Software Engineer focused on product development. In this pivotal role, you will contribute to the creation and support of innovative AI-powered features. You will be instrumental in developing, deploying, and managing our web services and infrastructure that underlie our advanced AI capabilities. Collaborating with a talented engineering team, you will optimize systems aimed at enhancing user productivity through state-of-the-art functionalities powered by AI.Your ImpactLeverage your expertise in Python to develop technology that integrates seamlessly with our AI-driven products.Construct robust business logic using frameworks like Django and manage database integrations with MySQL to ensure a scalable AI infrastructure.Work closely with technologies such as Nginx, Kafka, MySQL, Cassandra, Redis, and big data platforms to efficiently process and manage AI workloads.Produce reusable, testable, and efficient code that supports and scales our innovative AI solutions.

P-78 At Databricks, we are driven by a commitment to empower data teams in tackling some of the most significant challenges facing the world today—from revolutionizing transportation to speeding up the discovery of medical innovations. By constructing and managing the premier data and AI infrastructure platform, we enable our clients to harness profound data insights to enhance their operations. Founded by engineers and intensely focused on customer satisfaction, we eagerly embrace every chance to address technical challenges, whether it’s crafting next-generation UI/UX for data interaction or scaling our services and infrastructure across millions of virtual machines. And we are just getting started. As a Senior Software Engineer with a focus on backend development, you will collaborate with your team to create scalable infrastructure and products for the Databricks platform.

Job Title: Senior Staff Software Engineer - Enzyme Location: Mountain View, California At Databricks, we are driven by a passion for empowering data teams to tackle some of the world’s most intricate challenges—from revolutionizing transportation to accelerating medical advancements. We achieve this by developing and managing the leading data and AI infrastructure platform, enabling our customers to harness deep data insights to elevate their businesses. The Lakeflow Engineering team at Databricks oversees our comprehensive ETL product line, including innovative solutions like Materialized Views, Structured Streaming, and Delta Live Tables. We operate one of the largest data engineering platforms globally, processing exabytes of data daily for our extensive client base. We are in search of a committed technical leader to guide the Materialized Views engineering team. This team is tasked with creating next-generation Materialized View features that enhance both ETL workloads and query acceleration. In this role, you will engage in various areas to design and implement advanced systems that surpass current state-of-the-art: Incrementally maintaining materialized views Optimizing query performance Managing resources efficiently Developing effective storage structures Automating physical data optimization Your Impact: Address significant business needs at a large scale through your software engineering expertise. Deliver a robust, scalable, and fault-tolerant architecture for Materialized Views. Conduct low-level systems debugging, performance measurement, and optimization on large production clusters. Shape architecture design, influence product roadmaps, and assume ownership of new projects. Introduce tools that enhance automation and operational efficiency of services. Utilize your extensive experience to help preempt and resolve production issues. Plan and lead complex technical projects collaborating with multiple teams across the organization.

The OpportunityAre you a seasoned Staff Software Engineer with a strong background in product development, eager to make a profound impact at the convergence of scalable web services and groundbreaking AI technologies? We are on the lookout for a skilled engineer to spearhead the development, deployment, and management of web services and infrastructure that empower AI-driven products and initiatives. As a vital part of our dynamic engineering team, you will collaborate closely with AI researchers, product managers, and technologists to deliver innovative solutions that revolutionize the experiences of end-user professionals and enterprise clients across a wide array of industries.Your ImpactUtilize your extensive experience in Python to develop and enhance backend systems that integrate seamlessly with machine learning and AI-driven functionalities.Design and implement robust business logic using web frameworks such as Django, ensuring compatibility with AI components and integration with MySQL.Architect scalable solutions employing technologies such as Nginx, Kafka, MySQL, Cassandra, Redis, and big data platforms to support sophisticated AI models and workflows.Craft reusable, efficient, and maintainable code that underpins our high-performance web services and bolsters real-time AI analytics.Work closely with AI teams to optimize data pipelines, enhance model deployment, and ensure backend systems effectively support AI-based product features.Communicate proficiently with stakeholders across multiple teams, articulating technical strategies related to AI-enabled infrastructure.

The OpportunityAre you a seasoned Senior Software Engineer specializing in Backend Infrastructure eager to drive significant advancements at the confluence of scalable web services and innovative AI technologies? Otter.ai is on the lookout for a skilled engineer to spearhead the creation, deployment, and management of web services and infrastructure that empower our AI-driven products and initiatives. In our vibrant engineering team, you will work hand in hand with AI researchers, product managers, and technology experts to deliver transformative solutions that enhance the experiences of professional end-users and enterprise clients across various sectors.Your ImpactUtilize your extensive expertise in Python to develop and refine backend systems that integrate seamlessly with machine learning and AI-driven functionalities.Design and implement robust business logic using web frameworks such as Django, ensuring compatibility with AI components and integration with MySQL.Architect scalable solutions using technologies like Nginx, Kafka, MySQL, Cassandra, Redis, and big data platforms to facilitate sophisticated AI models and workflows.Write reusable, efficient, and maintainable code that forms the backbone of our high-performance web services and supports real-time AI analytics.Collaborate closely with AI teams to enhance data pipelines, optimize model deployment, and ensure backend systems effectively support AI-based product features.Effectively communicate with stakeholders across various teams, articulating technical strategies related to AI-enabled infrastructure.

At Databricks, we are dedicated to empowering data and AI teams to tackle some of the world's most significant challenges — from transforming transportation to expediting medical innovations. Our mission is to create and maintain the premier data and AI infrastructure platform, enabling our clients to leverage deep data insights to enhance their businesses. Founded by engineers with a customer-first mentality, we eagerly embrace every opportunity to address technical challenges, whether it’s designing next-gen UI/UX for data interaction or scaling our services across millions of virtual machines. And we are just getting started. As a member of the ML/AI Environments team, you will develop the systems that allow AI researchers and engineers to effortlessly establish their desired training and serving environments. You’ll join a high-agency, high-visibility team at the cutting edge of AI infrastructure, closely connected to research, product development, and real-world enterprise applications. Databricks Mosaic AI is one of our fastest-growing sectors, assisting thousands of customers in democratizing AI within their organizations. We are building the infrastructure that fuels the next generation of AI.

P-1126 Summary At Databricks, we are passionate about empowering data teams to tackle the world's most pressing challenges, ranging from security threat detection to groundbreaking cancer drug development. Our mission is to create and maintain the premier data and AI infrastructure platform that enables our customers to concentrate on high-value tasks central to their objectives. Our engineering teams are dedicated to developing innovative products that address significant real-world needs. We continually strive to advance the frontiers of data and AI technology while ensuring our operations are resilient, secure, and scalable to guarantee our customers' success on our platform. Customers place their trust in Databricks with their most critical data, and the Trust & Safety department's mission is to establish the most reliable data analytics and machine learning platform globally. Our Security Engineering team plays a pivotal role in safeguarding customer data from malicious threats. We seek seasoned leaders like you to craft the vision and strategy for this vital domain.

P-188 At Databricks, we are on a mission to fundamentally transform the data lifecycle, simplifying processes from data ingestion to ETL, Business Intelligence (BI), and extending to Machine Learning (ML) and Artificial Intelligence (AI) through a unified platform. We envision a future where traditional data warehouse architectures are replaced by innovative solutions like the Lakehouse architecture (CIDR 2021 paper), which integrate data warehousing and advanced analytics, effectively addressing critical challenges such as data staleness, reliability, cost of ownership, data lock-in, and limited use-case support. To turn this vision into reality, we are developing a next-generation decoupled query engine and structured storage system designed to surpass specialized data warehouses in relational query performance while maintaining the versatility of general-purpose systems like Apache Spark™. This system is intended to support a wide array of workloads, from ETL processes to data science applications. As a member of our team, you will engage in one or more of the following areas, contributing to the design and implementation of cutting-edge systems that redefine industry standards: Query compilation and optimization Distributed query execution and scheduling Vectorized execution engine Data security measures Resource management strategies Transaction coordination processes Efficient storage structures (encodings, indexes) Automatic physical data optimization techniques

Join Waymo as a Senior Staff Software Engineer specializing in TLM (Traffic Light Management). In this crucial role, you will lead the design and development of innovative software solutions that enhance our autonomous vehicle systems. Your expertise will contribute significantly to improving traffic management systems and ensuring seamless operations for our fleet.

P-1124 Summary At Databricks, we are driven by a passion for empowering data teams to tackle some of the most challenging problems in the world, spanning from security threat detection to advancements in cancer drug development. We achieve this by constructing and managing a top-tier data and AI infrastructure platform, enabling our customers to concentrate on their high-value missions. Our engineering teams are dedicated to creating highly technical products that address significant needs in the world. We continuously push the limits of data and AI technology while maintaining the resilience, security, and scalability essential for our customers’ success on our platform. Customers place their trust in Databricks with their most sensitive data, and our Trust & Safety team is committed to establishing the world’s most reliable data analytics and machine learning platform. Security Engineering plays a pivotal role within Trust & Safety, working diligently to safeguard customer data from malicious threats. We are seeking seasoned leaders like you to craft the vision and define the strategy for this critical area. The impact you will have: Enhance the safety of Databricks for our customers by identifying and addressing critical vulnerabilities in our infrastructure and services. Attract industry-leading talent to our team. Serve as a representative for the security engineering discipline throughout the organization, advocating for a data-driven approach. Represent Databricks at academic and industry conferences and events. What we look for: A minimum of 7 years of experience in Data Security or related fields, with expertise in at least two of the following areas: Cryptography, Kubernetes Security, Web Security, Governance, Privacy, Trust, Safety, Authentication, Identity Management, Access Control, Key Management, Inter-Service Authentication, Secure Application Frameworks, Detection & Response. Proven experience in building systems at large-scale internet companies is highly desirable. ...

At Databricks, we are dedicated to empowering data teams to tackle the world's most challenging problems—from transforming the future of transportation to expediting medical breakthroughs. We achieve this by creating and managing the most advanced data and AI infrastructure platform, enabling our customers to extract deep insights from data to drive business success. Founded by engineers with a strong customer focus, we eagerly embrace every opportunity to confront technical challenges, whether it’s designing next-generation UI/UX for data interfacing or scaling our services and infrastructure across millions of virtual machines. And we’re just getting started.The Logging Platform team is pivotal in developing scalable and efficient logging solutions that enhance observability across all Databricks services. As a Staff Software Engineer, you will spearhead the evolution of our logging infrastructure, empowering engineers throughout the company to gain profound insights into system behavior, troubleshoot issues efficiently, and optimize performance at scale.