Multiple locations
·
Fully Remote
QA Engineer – RAG-Enabled Test Automation
We’re looking for a hands-on QA Engineer to own functional quality as we integrate a Retrieval-Augmented Generation (RAG) assistant into our existing Java/Selenium/RestAssured automation stack. You will create high-value manual test cases, validate and refine autoscripted outputs, and help improve the feedback loop that makes our RAG engine smarter over time.
What You’ll Do
- Author and curate requirement tickets and manual test cases using structured formats (GIVEN/WHEN/THEN + data, edge cases, and links).
- Validate autoscripted tests generated by the RAG pipeline: compile, run, triage, and refactor them following Screenplay best practices; log precision/recall metrics into LangChain evaluators with guidance from the Automation Agent Engineer.
- Collaborate with SDETs to refactor duplicated Java/Cucumber steps, increase API-layer test coverage, and adopt code snippets surfaced by the retriever.
- Perform exploratory and regression testing on the new onboarding UI MVP; file defects, verify fixes, and maintain traceable acceptance criteria in Qtest.
What You Bring
- 3+ years hands-on QA automation experience with Java, Cucumber (BDD), Selenium/WebDriver, and RestAssured.
- Comfortable writing structured requirement tickets and decomposing stories into atomic acceptance criteria.
- Intermediate SQL skills for multi-database validation.
- Familiarity with Jenkins pipelines and artifact triage processes.
- Clear and precise communication skills, with the ability to translate fuzzy requirements into crisp GIVEN/WHEN/THEN steps.
It is an asset if you have
- Exposure to the Screenplay pattern (Serenity BDD) and Serenity reporting.
- Experience reviewing or prompting AI-generated code (e.g., GitHub Copilot, Cursor).
- Knowledge of LangChain evaluators or RAGAS for retrieval-quality scoring.
- Locations
- Multiple locations
- Remote status
- Fully Remote
About Perform
Since 2005, Perform's engineers have been helping companies scale their apps and their teams. We were near-shoring before it was even a term and have worked with 100s of clients along the way.
Founded in
2005
Already working at Perform?
Let’s recruit together and find your next colleague.