Join Credicorp Capital and transform challenges into opportunities as our next AI Quality Engineer on the GEN AI & Innovation team in Lima, Peru.Mission:To ensure comprehensive quality (functional, technical, security, and behavioral) of AI solutions (GenAI/LLM, RAG, agents, and ML components) developed by the AI Squad. This involves enabling production deployments with measurable quality metrics, controlled risks, and sufficient traceability for audits; reducing post-production defects and accelerating the time to production through Shift-Left practices and automation.Key Responsibilities:Define the QA strategy for AI, including exit criteria (DoD), criticality thresholds (Tier), and minimal suites; maintain baselines for each solution.Participate from discovery/design; review user stories and acceptance criteria; design test plans (happy path, edge cases, and fallbacks) before construction.Build “golden sets”; execute grounding/consistency/hallucination tests; validate retrieval and evidence.Conduct prompt injection/jailbreak, data leakage, and misuse validation; confirm guardrails and permissions defined with Architecture and Risk teams.Maintain automation for APIs, end-to-end, and regression testing of prompts/RAG; integrate into CI/CD and version control.Log/prioritize defects; perform root cause analysis with the squad; propose enhancements to prompts/knowledge bases/retrieval flows.Define metrics (success, fallback, scalability, latency); review logs/telemetry; alert on degradations and initiate fixes.Document plans/results/evidence/versions (prompts/datasets/config); maintain an artifact repository and sign-offs.Leverage generative AI tools to assist in creating test plans, generating test cases, analyzing results, and code reviews.Validate RAG pipelines by verifying retrieval quality, document relevance, and traceability of evidence used by the model.Qualifications:University degree in Systems Engineering, Computer Science, Information Technology, or related fields.Preferred to have QA training or certification (ISTQB Foundation or equivalent) and courses related to AI/GenAI Testing (evaluation of LLMs, RAG, or agents).Understanding of ML/GenAI fundamentals (evaluation metrics, embeddings/retrieval) and knowledge of security and privacy in regulated environments (DLP, data classification) is desirable.Experience:Minimum 4 years of experience in QA (functional and automated) for digital products; corporate/regulatory experience is ideal.At least 2-3 years of relevant experience in AI/GenAI environments.
Apr 30, 2026