About the job
At Replit, we are revolutionizing the world of software development with our innovative platform that empowers individuals to create applications using natural language. With a thriving community of millions of users and over 500,000 businesses leveraging our technology, we are committed to breaking down traditional barriers and democratizing coding for everyone.
About the Position
Join our dedicated Anti-Abuse team as we protect Replit’s platform from exploitation and misuse. In this role, you will be at the forefront of defending against phishing attacks, preventing cryptomining on our free-tier infrastructure, and stopping LLM token farming. You will be tasked with developing advanced detection systems and automated responses that keep pace with ever-evolving threats. This position is unique in that you will tackle challenges that are often unprecedented, such as establishing safety measures for AI-generated code and implementing defenses against prompt injection attacks. If you are eager to gain hands-on experience applying AI in real-world security challenges, this is the ideal opportunity for you.
Your Responsibilities
Design and implement protective measures for AI-generated code and agent interactions to detect abuse scenarios.
Develop AI-driven detection systems utilizing LLMs to identify malicious activities, classify threats, and automate responses.
Create and maintain abuse detection systems for phishing, cryptomining, account takeovers, and financial fraud across millions of user actions daily.
Design automated response mechanisms that enforce platform policies seamlessly.
Oversee the entire abuse response lifecycle, including detection, investigation, enforcement, and appeal management in collaboration with Support and Legal teams.
Analyze attack patterns using BigQuery and Hex to translate findings into new detection rules.
Enhance and maintain internal detection tools (Slurper, Netwatch) that consistently monitor user activity.
Integrate and fine-tune security scanners (SAST, SCA) within CI pipelines, ensuring adherence to stringent performance SLAs.
Monitor abuse trends, assess detection effectiveness, and adapt defenses as attack patterns evolve.
