Assignment Details
Our client, a leading global specialist in energy management and automation is looking to engage with a Consultant QA Automation Engineer Requirements for RAG Applications
Key responsibilities:
Quality Evaluation
i) Evaluate Model Outputs:
-Develop automated tests to assess the quality of RAG-generated responses.
-Verify correctness, coherence, and relevance of answers.
ii) Test Automation:
-Create test scripts for end-to-end testing.
-Implement regression, functional, and performance tests.
Continuous Integration and Deployment (CI/CD)
i) CI/CD Pipelines:
-Set up and maintain CI/CD pipelines for RAG applications.
-Automate deployment, testing, and monitoring.
ii) Version Control:
-Proficiency with Git or other version control systems.
-Collaborate with development teams to manage code changes.
Monitoring and Reporting
i) System Health Monitoring:
-Implement monitoring tools to track application performance.
-Detect anomalies, bottlenecks, and errors.
ii) Reporting and Documentation:
-Document test results, issues, and improvements.
-Communicate findings to stakeholders.
Domain-Specific Knowledge
i) Generative AI and RAG:
-Understand how RAG models work.
-Adapt QA strategies to the unique challenges of generative content.
ii) Compliance (if applicable):
-Ensure compliance with industry-specific guidelines.
-Validate RAG responses against regulatory requirements.
Duration: 6 months (extendable to 9 months)
Capacity: Full time
Location: Bengaluru
Skills Required
- Relevant experience in QA Automation Engineer in Generative AI Space.
- AI and ML Concepts: Understand fundamental concepts related to artificial intelligence and machine learning. Familiarity with large language models (LLMs) and their applications.
- Semantic Search Techniques: Knowledge of semantic search algorithms, Ability to optimize search queries for efficient retrieval.
- Database Management: Experience with databases (e.g., SQL, NoSQL), Understand data ingestion, storage, and retrieval.
Share
3 month(s)
5+ years 
Remote
