Perplexity logo

Senior Backend Software Engineer - Crawler Team

PerplexityBelgrade
On-site Full-time

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Experience Level

Mid to Senior

Qualifications

Qualifications5+ years of software development experience. Proficient in Python, C++, Rust, or Go. Experience with large-scale web crawlers. Expertise in building high-load distributed services. Hands-on experience with Kubernetes and AWS.

About the job

Join our dynamic Crawler team at Perplexity as a Senior Backend Software Engineer. In this pivotal role, you will be responsible for designing, developing, and managing systems that handle the ingestion, processing, and management of vast web-scale data to support our cutting-edge search technologies. This is a high-impact engineering position that requires a blend of backend and frontend expertise within our data acquisition stack.

Key Responsibilities

  • Lead the development of large-scale web crawlers, ingestion pipelines, and data processing systems.

  • Construct, maintain, and optimize vital backend and frontend components for crawler services, including storage solutions, data retrieval functions, and UI dashboards for effective data management.

  • Work closely with Search and Infrastructure teams to ensure efficient and high-quality ingestion and processing of billions of web pages.

  • Architect and implement full-stack features and scalable distributed systems capable of handling high loads and real-time data operations.

  • Experiment with innovative approaches to rapidly enhance system performance, usability, and reliability.

Qualifications

  • At least 5 years of software development experience with a solid understanding of data structures and algorithms in languages such as Python, C++, Rust, or Go.

  • Experience in developing large-scale web crawlers is highly preferred.

  • Demonstrated expertise in building, deploying, and optimizing high-load, distributed services.

  • Strong knowledge of cloud infrastructure, particularly hands-on experience with Kubernetes (K8s) and AWS.

  • A passion for writing clean, efficient, and scalable code.

About Perplexity

Perplexity is a forward-thinking technology company dedicated to revolutionizing search technologies through innovative data solutions. Our team is passionate about building systems that harness the power of information for a smarter, more connected world.

Similar jobs

Browse all companies, explore by city & role, or SEO search pages.

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.