Inference Software Engineer

About EtchedEtched is pioneering the world's first AI inference system specifically designed for transformers, achieving over 10x higher performance, significantly reduced costs, and minimal latency compared to B200 systems. Our custom ASICs enable the development of innovative products that were previously unattainable with GPUs, such as real-time video generation models and advanced chain-of-thought reasoning agents. With substantial backing from leading investors and a team of top engineers, Etched is revolutionizing the infrastructure of one of the fastest-growing industries in history.Key ResponsibilitiesAssist in porting cutting-edge models to our architecture and contribute to the development of programming abstractions and testing capabilities to streamline the model porting process.Develop, enhance, and scale Sohu’s runtime, focusing on multi-node inference, intra-node execution, state management, and effective error handling.Optimize routing and communication layers utilizing Sohu's collectives.Employ performance profiling and debugging tools to pinpoint bottlenecks and correctness challenges.Ideal Candidate ProfileStrong proficiency in C++ or Rust programming languages.Solid understanding of performance-critical and complex distributed software systems, including Linux internals, accelerator architectures (e.g., GPUs, TPUs), compilers, and high-speed interconnects (e.g., NVLink, InfiniBand).Familiarity with machine learning frameworks such as PyTorch or JAX.Experience in porting applications to non-standard accelerator hardware or platforms.Preferred QualificationsExperience in developing low-latency, high-performance applications using both kernel-level and user-space networking stacks.In-depth understanding of distributed systems concepts, algorithms, and challenges, including consensus protocols and communication patterns.Thorough knowledge of Transformer architectures, particularly Mixture-of-Experts (MoE).Experience building applications with substantial SIMD (Single Instruction, Multiple Data) optimizations for performance-critical paths.BenefitsHealth Insurance401k

About EtchedEtched is pioneering the world’s first AI inference system specifically designed for transformers, achieving over 10x the performance and substantially reduced cost and latency compared to traditional GPUs. Our innovative ASICs enable the creation of groundbreaking products, such as real-time video generation models and advanced reasoning agents with deep and parallel processing capabilities. Supported by substantial investments from leading venture firms and a team of top-tier engineers, Etched is transforming the infrastructure layer for one of the fastest-growing industries globally.Key ResponsibilitiesKernel-Mode Driver Development: Innovate, develop, and maintain kernel-mode drivers with an emphasis on reliability, comprehensive debugging, and peak performance.Performance Optimization: Conduct thorough analysis and optimization of driver efficiency for high-demand AI tasks, aiming to reduce latency and enhance throughput.Hardware Integration and Co-Design: Work in tandem with hardware engineers during the ASIC design lifecycle to ensure seamless integration.Virtualization Support: Develop driver capabilities for virtualization technologies including SR-IOV, VFIO, and para-virtualization.Memory Management: Design and implement effective memory management protocols, addressing kernel memory mapping, page tables, NUMA awareness for device data caching, and IOMMU configurations.Security: Create kernel drivers with a fundamental focus on security, safeguarding host processes, physical memory spaces, and device attestation.Debugging and Troubleshooting: Identify and resolve intricate driver issues utilizing standard kernel debugging tools and methodologies (ftrace, dmesg, etc.) for bug resolution.Synchronization and Concurrency: Develop synchronization strategies to manage concurrent access to multiple accelerators.System Validation and Testing: Formulate and execute comprehensive testing protocols to validate the functionality, stability, and performance of drivers in production and manufacturing settings.Collaboration and Troubleshooting: Work collaboratively with both software and hardware teams to diagnose and resolve complex system-level challenges.

About EtchedEtched is pioneering the development of the world's first AI inference system explicitly engineered for transformers, providing over 10x greater performance while significantly reducing costs and latency compared to traditional solutions. Our Etched ASICs enable the creation of products that were previously unattainable with GPUs, such as real-time video generation models and deeply parallel chain-of-thought reasoning agents. With substantial backing from leading investors and a team of top-tier engineers, Etched is reshaping the infrastructure for one of the fastest-growing industries in history.Job SummaryWe invite you to join our innovative team as a Software Engineer - Performance Tools. In this pivotal role, you will spearhead the development of an advanced performance analysis tool specifically designed for Sohu. Your expertise will be crucial in creating essential tools that empower our ML engineers and clients to comprehend workload behaviors, pinpoint performance limitations, and fully harness the potential of Sohu in accelerating the most demanding ML applications globally. This unique opportunity allows you to influence the performance analysis landscape for groundbreaking hardware from its inception.Key ResponsibilitiesTool Architecture & Design: Lead the architecture and design of a robust performance analysis suite, incorporating data collection mechanisms, processing pipelines, analysis engines, and user interfaces (CLI and/or GUI).Low-Level Data Collection: Create reliable methods to gather performance data directly from our custom ML accelerator hardware (e.g., hardware performance counters, execution unit status, memory access patterns) through driver interfaces or other means.Host & System Tracing: Establish tracing for host-side API interactions (runtime libraries, driver communications) and system-level events (CPU activity, PCIe traffic, memory usage, network contention) associated with Sohu workloads.Data Correlation & Synchronization: Develop and implement methodologies for accurately correlating performance events across host CPUs, device drivers, PCIe buses, multiple accelerators, and various hosts, ensuring precise time synchronization.Performance Analysis Engine: Construct analysis modules to automatically interpret the collected trace and counter data, identifying key performance limits (e.g., compute-bound, memory bandwidth-bound, latency-bound, PCIe-bound).

AI Architecture Intern - InferenceLocation: San Jose, CA Team: ArchitectureAbout EtchedAt Etched, we are pioneering the development of the world’s first AI inference system specifically designed for transformers, achieving over 10 times the performance and significantly reduced cost and latency compared to traditional systems. Our innovative ASICs empower the creation of groundbreaking products, enabling real-time video generation and advanced reasoning agents that are unattainable with conventional GPUs. Supported by substantial investments from leading venture capital firms and staffed by top-tier engineering talent, Etched is at the forefront of transforming the infrastructure of the fastest-growing industry.The RoleWe are in search of a motivated Architecture Intern to join our dynamic team, contributing to the design and optimization of next-generation AI accelerators. This role will involve developing and fine-tuning compute architectures that deliver outstanding performance and efficiency for transformer workloads. Throughout your internship, you will tackle cutting-edge architectural challenges and engage in performance modeling.Key ResponsibilitiesAssist in adapting state-of-the-art models to our architecture and develop programming abstractions and testing capabilities for rapid model iteration.Help enhance and scale Sohu’s runtime, including multi-node inference, intra-node execution, state management, and robust error handling.Contribute to the optimization of routing and communication layers utilizing Sohu’s collectives.Employ performance profiling and debugging tools to pinpoint bottlenecks and correctness issues.Gain a deep understanding of Sohu to collaboratively design hardware instructions and model architecture operations to maximize performance.Implement high-performance software components for the Model Toolkit.QualificationsCurrently pursuing a Bachelor’s, Master’s, or PhD in Computer Science, Computer Engineering, Applied Mathematics, or a related discipline.Strong programming skills in Python and C++.Familiarity with performance-sensitive or complex distributed software systems, such as Linux internals, accelerator architectures (e.g., GPUs, TPUs), and compilers.

About EtchedEtched is pioneering the development of the world’s first AI inference system specifically designed for transformers, achieving performance metrics that exceed standard models by over 10x while significantly decreasing costs and latency compared to traditional GPUs. With our cutting-edge ASIC technology, we empower the creation of groundbreaking products, such as real-time video generation models and advanced reasoning agents that feature deep and parallel processing capabilities. Supported by substantial investments from leading venture capitalists and a team of top-tier engineers, Etched is at the forefront of transforming the infrastructure landscape in the rapidly evolving AI sector.Job SummaryWe are looking for enthusiastic and talented Supercomputing Engineers (Network) to enhance our dynamic team. This pivotal role involves the development, qualification, and optimization of high-performance networking solutions tailored for extensive inference workloads. As a Pod Software Engineer, your focus will be on creating and validating software that facilitates communication between Sohu inference nodes across multi-rack clusters. You will work in close collaboration with kernel, platform, and telemetry teams to maximize the efficiency of peer-to-peer RDMA communications.Key ResponsibilitiesHigh Performance Peer to Peer Networking: Conceptualize, develop, and implement RDMA-based networking solutions that enable high bandwidth and low latency communication across PCIe nodes, both within and between racks. This role encompasses work across operating systems, kernel drivers, embedded software, and system software.Test Development: Create and implement tests to validate host processors (x86), NICs, TORs, and device network interfaces for optimal performance.Burn-in Integration: Provide burn-in teams with testing frameworks that simulate real-world use cases and workloads for device-to-device networking, including extreme-load stress testing.Performance/Health Telemetry Design: Establish key metrics that system software should gather to ensure high availability and performance under demanding communication workloads.Representative ProjectsEvaluate performance deviations, refine network stack configurations, and suggest kernel tuning parameters for low-latency, high-bandwidth inference workloads.Design and execute automated qualification tests for RDMA NICs and interconnects across a variety of server configurations.Identify and troubleshoot network-related issues to enhance overall system performance.

Join Couchbase Inc. as a Principal Software Engineer and play a pivotal role in designing and implementing innovative software solutions. You will collaborate with cross-functional teams to drive the technical direction of our products, ensuring high performance and scalability. Your expertise will guide the development of robust systems that meet customer needs and industry standards.

About EtchedEtched is pioneering the construction of the world's first AI inference system specifically designed for transformers. Our innovative technology delivers more than 10 times the performance while significantly lowering costs and latency compared to traditional systems like the B200. With Etched's ASICs, you can create groundbreaking products that were previously unattainable with GPUs, such as real-time video generation models and advanced deep reasoning agents capable of complex thought processes. Supported by substantial investments from premier investors and a team of top engineers, Etched is on a mission to transform the infrastructure layer of the fastest growing industry in history.Job SummaryWe are on the lookout for a highly driven, intelligent, and versatile Component Engineer to guarantee that the components central to our systems—from voltage regulator modules (VRMs) to high-speed connectors—adhere to our standards for performance, manufacturability, long-term supply stability, and reliability at scale. This role will report directly to the Head of Quality and Reliability, with a strong collaboration with the Electrical Engineering R&D team.This position is crucial for scaling our data center infrastructure design and production, strategically positioned at the nexus of electrical engineering, quality assurance, supply chain management, and manufacturing. The Component Engineer will be responsible for maintaining the technical and lifecycle integrity of electronic components utilized in Etched's compute trays, IO boards, mezzanine cards, and associated subsystems. By spearheading evaluation and qualification initiatives and resolving supply chain challenges, this role is key to accelerating design and production timelines.

Join archer56 as a Staff Software Engineer specializing in Engineering Applications. In this pivotal role, you will leverage your extensive software engineering expertise to develop innovative solutions that enhance our engineering capabilities. Collaborate with cross-functional teams to design, implement, and optimize applications that streamline engineering processes and improve productivity.

About EtchedAt Etched, we are pioneering the world's first AI inference system uniquely designed for transformers, achieving over 10x greater performance, along with significantly reduced costs and latency compared to conventional solutions like the B200. Our innovative ASICs enable the creation of groundbreaking products, such as real-time video generation models and highly advanced deep reasoning agents. With substantial backing from premier investors and a team of top engineers, Etched is transforming the infrastructure layer for the fastest growing industry in history.Key ResponsibilitiesDevelop detailed performance models and forecasts for Etched's transformer-centric architecture across various workloads and configurations.Profile and assess deep learning workloads on Etched to detect micro-architectural bottlenecks and potential optimization areas.Create analytical and simulation-driven models to anticipate performance across different architectural setups and design trade-offs.Collaborate with hardware architects to influence micro-architectural decisions based on workload characteristics and performance insights.Facilitate hardware/software co-optimization by pinpointing opportunities where architectural features can substantially enhance performance.Analyze and optimize memory hierarchy efficiency, interconnect utilization, and computational resource effectiveness.Establish performance benchmarking frameworks and methodologies tailored specifically for transformer inference workloads.Performance CharacterizationConstruct detailed roofline models and performance forecasts for Etched across various transformer architectures (e.g., Llama, Mixtral).Profile production inference workloads to identify and mitigate micro-architectural bottlenecks.Evaluate memory bandwidth, compute utilization, and interconnect performance to inform next-gen architecture decisions.Develop performance modeling tools that forecast chip behavior based on different batch sizes, sequence lengths, and model configurations.Characterize the performance implications of architectural features such as specialized datapaths, memory hierarchies, and on-chip interconnects.Benchmark Etched's architectural efficiency against competitive solutions to ensure industry-leading performance.

About EtchedEtched is at the forefront of innovation, creating the world’s first AI inference system specifically designed for transformers. Our technology delivers over 10x the performance and significantly reduces cost and latency compared to traditional systems like the B200. With our advanced ASICs, we empower the development of groundbreaking products including real-time video generation models and highly sophisticated chain-of-thought reasoning agents. Supported by substantial investment from top-tier VCs and staffed by a team of elite engineers, Etched is reshaping the infrastructure for the fastest-growing industry in history.Job SummaryWe are on the lookout for a driven and detail-oriented Supercomputing Engineer (Test) to join our dynamic team. This integral position is crucial for maintaining the reliability and stability of our high-performance inference server hardware and software. In this role, you will design, develop, and execute comprehensive burn-in test suites, analyze test results, and collaborate closely with both hardware and software engineering teams at Etched and our ODM partners to swiftly identify and rectify potential issues. You will play a vital role in ensuring that our server products uphold the highest quality standards before reaching our valued customers.Key ResponsibilitiesTest Development: Craft, develop, and implement automated burn-in test suites utilizing common scripting languages (Python, Go, Bash) and testing frameworks, covering all facets of System Operation including boot sequences, root-of-trust, system management, workload deployment, and performance.Test Execution: Conduct burn-in tests on server hardware, monitor system performance and health, and interpret test results.Failure Analysis: Delve into and troubleshoot hardware and software failures uncovered during testing, delivering detailed reports and mitigation strategies.Collaboration: Engage with both internal and external hardware and software engineering teams to pinpoint root causes of failures and implement corrective measures.Test Infrastructure: Aid in the creation and upkeep of the burn-in testing infrastructure, encompassing portable test environments and automation tools operable in any setting.Documentation: Generate and maintain thorough documentation for test plans, test cases, and results.Performance Analysis: Evaluate system performance metrics to identify areas for enhancement.

Join Extreme Networks, a global leader in cloud-driven networking solutions trusted by over 50,000 customers worldwide. Our innovative services empower organizations to enhance their digital transformation journeys, yielding remarkable progress and scalability. With a consistent double-digit growth year over year, we stand out as a premier provider committed to delivering exceptional outcomes.At Extreme, inclusion is woven into our core values. We are dedicated to cultivating a diverse and inclusive workplace where every employee thrives, and their unique perspectives contribute to our collective success.Be a part of something significant with Extreme! Discover why now is the perfect time to join our dynamic team.

About EtchedAt Etched, we are at the forefront of innovation, developing the world’s first AI inference system that is meticulously designed for transformers. Our technology offers over 10x the performance of traditional systems like the B200, while significantly reducing costs and latency. With our advanced ASICs, we empower the creation of groundbreaking products, enabling real-time video generation and complex reasoning models that surpass the capabilities of GPU-based systems. Supported by substantial investments from prominent industry leaders and a team of top-tier engineers, Etched is reshaping the infrastructure of the fastest-growing sector in history.Job SummaryWe are seeking highly skilled Physical Design Engineers to join our dynamic team. In this pivotal role, you will take charge of block-level implementation and verification, spearhead timing closure and PPA optimization, oversee third-party design collaborations, and enhance our design workflows and iteration speed.Key ResponsibilitiesGain a comprehensive understanding of the physical design processExecute PD flows to achieve block closure, support ASIC infrastructure, automate PD processes, and refine CAD toolsCollaborate with RTL Designers to provide insightful Physical Design feedback aimed at enhancing PPADevelop and manage dashboards that track project convergence related to PDOptimize tool workflows by partnering with EDA vendors to adopt the latest featuresTake responsibility for achieving block-level closureOversee the outsourcing of physical design tasks to third-party service providersYou may be a great fit if you possess the following qualifications:5-10+ years of relevant experience in Physical DesignProficiency with tools, workflows, and design methodologies spanning RTL synthesis to GDSII sign-offExperience in back-end design and timing closure on advanced process nodes (5nm and below)Familiarity with Cadence (Innovus, Genus) or Synopsys (ICC2, Fusion Compiler) automated RTL-to-GDSII flowsKnowledge of sign-off tools (PrimeTime, Tempus, Voltus, etc.)Expertise in UPF-based low power design methodologies, power verification, synthesis, scan insertion/ATPG, formal verification, floorplanning, placement, CTS, routing, IR drop, and EM/antenna analysisCreative problem-solving skills with a first-principles approach

About Tessera Labs: At Tessera Labs, we are pioneering the integration of Artificial Intelligence into enterprise operations. Supported by Foundation Capital and led by an exceptional founding team, we create advanced multi-agent AI systems designed to automate intricate business workflows across various platforms including SAP, Salesforce, Workday, Snowflake, and MuleSoft. Our mission is to deliver genuine AI automation to enterprises with unmatched speed, precision, and measurable outcomes. We thrive on rapid execution, embrace accountability, and innovate at the cutting edge of applied AI.Job Overview:We are on the hunt for a dynamic Frontend Engineer who flourishes in a fast-moving, high-responsibility startup atmosphere. This hands-on position is ideal for a proactive individual who is ready to take charge of the user experience from ideation through to deployment. You will be expected to iterate swiftly based on user feedback while optimizing for performance, usability, and speed. Your role will involve designing and developing intricate UI components, interactive dashboards, and data visualizations that have a direct influence on our end users.If you are passionate about tackling challenging UI problems, delivering polished interfaces at a rapid pace, and collaborating in a supportive, low-ego environment, we invite you to apply for this exciting opportunity.

Join a thriving community of over 50,000 customers worldwide who trust our innovative, cloud-driven networking solutions. At Extreme Networks, we empower organizations to accelerate their digital transformation and achieve remarkable progress through our award-winning services and unwavering support. With consistent double-digit growth each year, we are uniquely positioned to deliver scalable outcomes that transform businesses.At our core, we value inclusion and diversity, creating a workplace where every employee can flourish because of their unique perspectives and experiences. We believe that our differences are our strength, and we are dedicated to fostering an environment that supports everyone’s success.Now is the perfect time to become part of something significant with Extreme! As a global leader in networking, discover the exceptional opportunities that await you on the Extreme team.

San Jose or LA CA (Hybrid)At FloQast, we are revolutionizing the way accounting teams function with our AI-driven workflows. Our platform is trusted by thousands of companies worldwide, serving as a comprehensive operating system for accounting that streamlines intricate financial processes into efficient, dependable, and scalable systems.We are in search of a Staff Software Engineer who not only develops software but also contributes to shaping platforms, enhances team efficiency, and drives substantial impact. In this pivotal role, your architectural choices will significantly affect the entire FloQast application ecosystem, directly fostering innovation across our product offerings.

At Efficient, we are on a mission to create the world's most energy-efficient general-purpose computer processor. Our patented technology utilizes an astonishing 100 times less energy than the leading ultra-low-power processors available today. This groundbreaking efficiency allows for continuous AI/ML operations on just a AA battery for a remarkable 5-10 years. Our platform's unparalleled efficiency empowers IoT devices to intelligently gather and curate first-party data, paving the way for the next computing revolution.We invite a Senior Software Optimization Engineer to join our dynamic team at Efficient. In this pivotal role, you will design, optimize, and deploy applications for our state-of-the-art programmable processor known as the “Fabric.” You will take ownership of complex, performance-critical applications, collaborating closely with our compiler team to implement customer specifications, validate software quality, and offer deep technical insights into compiler interactions. This position presents a unique opportunity to engage in cutting-edge hardware/software co-design, while making an immediate impact in advancing the next generation of embedded applications.

Archer is an innovative aerospace firm located in San Jose, California, dedicated to creating an all-electric vertical takeoff and landing aircraft. Our goal is to enhance sustainable air mobility, designing, manufacturing, and operating an aircraft capable of transporting four passengers with minimal noise production.We aim high and tackle challenging problems, believing that a diverse workforce fosters smarter solutions, deeper insights, and collective success. Archer is committed to nurturing an inclusive and equitable environment that values differences and supports every member of our team.What you will do:Manage Archer's integrated lab environments to support validation and verification processes, including quality assessments, human factors evaluations, and test pilot training.Work collaboratively across internal teams and with suppliers to integrate, instrument, and maintain the configuration of simulation models within a real-time hardware-in-the-loop environment.Develop simulation models of various systems and integrate them into the overarching framework.Create and implement graphical user interfaces to facilitate control of the real-time simulation environment.Support the execution of test plans and automation of testing procedures.

Role overview itD seeks an Embedded Diagnostic Software Engineer to design, develop, and validate embedded diagnostic solutions. This position plays a key part in system reliability, performance, and smooth integration between hardware and software. The work centers on hands-on embedded systems development and low-level programming. The engineer will address complex interactions between hardware and software, debug challenging issues, and deliver code ready for production use. Location This is a hybrid position based in San Jose, CA. Expect to be onsite four days per week, with flexibility for five days onsite as needed. Compensation The hourly pay range is $55 to $62. Requirements Extensive experience in embedded systems development Strong low-level programming skills Demonstrated ability to debug complex hardware and software issues Track record of delivering production-quality solutions Benefits Medical coverage 401k plan Paid holidays Additional benefits included Additional details Direct W2 candidates only No sponsorship available

Collaboration Fuels Innovation. Transforming the Viewing ExperienceRoku is the leading TV streaming platform in the U.S., Canada, and Mexico, with ambitions to power every television globally. As pioneers in streaming technology, our goal is to connect consumers with their favorite content, empower content creators to build and monetize large audiences, and offer advertisers unparalleled tools to engage users.From day one at Roku, your contributions will be recognized and valued. Join a rapidly growing public company where you can make a difference and delight millions of TV streamers worldwide while expanding your expertise across various disciplines. About the Team The Ad Engineering Team is essential for creating a scalable and reliable real-time advertising platform crucial for expanding Roku's advertising business. We are seeking a Senior Software Engineer to join our Advertising Platform team, playing a key role in this mission. If you have extensive experience in developing complex systems, a passion for technology at scale, and enjoy mentoring and problem-solving, this opportunity is for you! About the Role As a leader in the TV streaming industry, Roku continues to innovate and shape the future of streaming through The Roku Channel. Our success depends on our investment in Advertising Engineering. The ideal candidate will exhibit a strong curiosity, blending a global vision with locally relevant execution. Successful candidates will showcase a balance of technical and interpersonal skills, adapting quickly to changing business needs.Building a real-time, highly scalable, and available advertising platform is vital for our success. Our team's mission is to develop a top-tier advertising platform that supports and enhances Roku's rapidly growing ad business, ensuring high performance and user privacy.For California Only - The estimated annual salary for this position ranges from $195,000 to $408,000. Compensation packages consider unique factors for each candidate, including experience and qualifications.

About the RoleJoin our team as a Senior Robotics Software Engineer, responsible for developing and managing the essential platform infrastructure for our cutting-edge autonomous robots. We are looking for a versatile robotics engineer with a strong background in software development across middleware, device integration, embedded systems, build and deployment pipelines, and cloud infrastructure.ResponsibilitiesTake ownership of the robotics platform stack from middleware to embedded software across various robot configurations.Develop infrastructure for sensor integration, data capture, system observability, and comprehensive diagnostics for our fleet.Enhance developer workflows by creating tools that increase team productivity and efficiency.Design and manage CI/CD processes, build, and deployment pipelines, along with fleet management systems for embedded targets.Lead architectural discussions on system design, messaging protocols, and cross-team integrations.