Test Automation Engineer Python+genAI

4 dni temu

Warszawa, Mazovia, Polska Transition Technologies MS Pełny etat

We're searching for a Test Automation Engineer.

Your responsibilities:

  • Generative AI Testing Strategy: Design and implement comprehensive testing strategies specifically tailored for LLM-based applications, including evaluation of model responses, RAG pipelines accuracy, and overall system reliability
  • Quality Assurance Framework Development: Utilize Client's testing frameworks that address both traditional software quality aspects and AI-specific concerns such as output consistency, contextual accuracy, and ethical compliance; co-create and maintain such frameworks when required
  • Test Automation Development: Design and implement automated testing solutions for continuous evaluation of LLM applications, including integration tests, performance tests, and specialized AI behavior tests
  • Edge Case Analysis: Identify and develop test scenarios for edge cases in LLM behavior, including handling of ambiguous inputs, potential biases, and unexpected response patterns
  • Bias and Fairness Testing: Design and execute tests to identify potential biases in model outputs and ensure fair treatment across different user groups and use cases
  • Security Testing: Collaborate within development teams to test for potential vulnerabilities specific to LLM applications, including prompt injection, data leakage, and other AI-specific security concerns
  • Test Documentation: Create and maintain comprehensive test documentation testing strategy, test cases, and testing guidelines specific to AI applications and compliant with Client practices; document the analysis of requirements and risks and develop tests accordingly
  • Performance Testing: Collaborate with other engineers to conduct thorough performance testing of GenAI applications, including response time analysis, load testing, and resource utilization monitoring

We are looking for you, if you have:

  • Experience: 3+ years of experience in software testing, with at least 1 year focused on genAI applications
  • Technical skills: Strong proficiency in Python and testing frameworks (e.g., pytest, unittest); experience with AI testing tools and frameworks
  • AI/ML knowledge: Understanding of LLM architectures, RAG systems, and common failure modes in AI applications; familiarity with evaluation metrics for generative AI
  • Test Automation: Experience with test automation frameworks and CI/CD pipelines; ability to design and implement automated testing solutions for AI applications
  • Ability to analyze requirements, identify test scenarios, and prioritize testing activities based on risk and impact
  • Cloud Platforms: Practical experience with testing applications on cloud platforms (AWS, Azure) and working with cloud-based AI services
  • Monitoring tools: Experience with monitoring and observability tools, log analysis, and performance metrics tracking (e.g. DataDog, Grafana)
  • ALM tools: Experienced in ALM tools (e.g. Jira, GitLab, ServiceNow)
  • Security Testing: Understanding of security testing methodologies, particularly in the context of AI applications
  • Very good practical and theoretical knowledge of testing processes (ISTQB Advanced, etc.)
  • The successful candidate should also hold B.Sc., B.Eng., M.Sc., M.Eng. in Computer Science, Software Engineering, or equivalent degree with strong background in testing methodologies
  • Be able to communicate in English at the level of C1+

We offer:

  • Participation in interesting and demanding projects
  • Flexible working hours
  • A great, non-corporate atmosphere
  • Stable employment conditions (contract of employment or B2B contract)
  • Opportunity to develop your career

We reserve the right to contact the selected candidates.

  • Test Automation Engineer

    2 tygodni temu

    Warszawa, Mazovia, Polska Transition Technologies MS Pełny etat

    We're searching for a Test Automation Engineer. Your responsibilities:Generative AI Testing Strategy: Design and implement comprehensive testing strategies specifically tailored for LLM-based applications, including evaluation of model responses, RAG pipelines accuracy, and overall system reliabilityQuality Assurance Framework Development: Utilize Client's...

  • QA Engineer with GenAI

    4 dni temu

    Warszawa, Mazovia, Polska DCG Pełny etat

    As a recruitment company, DCG understands that every business is powered by experienced professionals. Our management style and partnership approach enable us to meet your needs and provide continuous support. Due to our ongoing growth and the large number of recruitment projects we undertake for our partners, we are currently looking for:QA Engineer with...

  • Automation Test Engineer

    4 dni temu

    Warszawa, Mazovia, Polska Link Group Pełny etat

    We are looking for a highly skilled Senior Automation Test Engineer with over 10 years of experience in software testing, including 4+ years in automation testing using Python or Java/Selenium. The ideal candidate will have a strong foundation in Core Python and expertise in data reconciliation, API automation, and backend automation testing.This role...

  • Senior Test Automation Engineer

    3 dni temu

    Warszawa, Mazovia, Polska EndySoft Pełny etat

    Senior Test Automation EngineerWarszawaWarsaw, Masovian Voivodeship, PolskaWe are seeking a highly skilled Test Automation Engineer to join our team. The ideal candidate will have strong experience in Python programming and test automation across various domains, including web applications, APIs, and virtualization environments. This role involves leveraging...

  • Senior Test Automation Engineer

    2 dni temu

    Warszawa, Mazovia, Polska EndySoft Pełny etat

    technologies-expected : Python Selenium VMware OpenStack GitLab technologies-optional : Docker Kubernetes about-project : We are seeking a highly skilled Test Automation Engineer to join our team. The ideal candidate will have strong experience in Python programming and test automation across various domains, including web applications, APIs, and...

  • Test Automation Developer

    3 dni temu

    Warszawa, Mazovia, Polska EndySoft Pełny etat

    Senior Test Automation Engineer OverviewWe are looking for a seasoned professional to take our test automation capabilities to the next level. As a Senior Test Automation Engineer, you will be responsible for developing and maintaining automated test scripts for web applications using tools such as Selenium or Playwright.Main Responsibilities:Develop...

  • Test Automation Engineer

    4 dni temu

    Warszawa, Mazovia, Polska Inetum Polska Pełny etat

    Drive Digital Transformation with Inetum Polska - Senior Software QA Engineer RoleInetum Polska, a leading provider of IT services, invites experienced professionals to apply for the position of Senior Software QA Engineer. As a key player in driving the digital transformation of businesses and public institutions, we offer a unique opportunity for growth...

  • Test Automation Engineer

    1 tydzień temu

    Warszawa, Mazovia, Polska Nord Security Pełny etat

    The Infrastructure department is responsible for influencing and tracking change, providing frontline support, and delivering software-defined solutions.We are seeking a skilled Test Automation Engineer (Python) / Python Developer – Automation & Testing to join our team. The ideal candidate will have strong experience in Python development, automation, and...

  • Software Test Automation Engineer

    3 tygodni temu

    Warszawa, Mazovia, Polska HAYS POLAND Sp. z o.o. Pełny etat

    Software Test Automation Engineerlokalizacja: Warszawa (mazowieckie)numer referencyjny: 1191728/jobs.plforma zatrudnienia: Pełny etatYour new company Join global leader of heavy machinery industry and work in their newly established technological centre. You will become part of a division that is responsible for delivering innovative solutions to help...

  • Python Automation Tester

    2 dni temu

    Warszawa, Mazovia, Polska TN Poland Pełny etat

    We are looking for a candidate who wants to join the largest Test Services Center team as a Test Automation Engineer with Python and Robot Framework in Sii to support our Client from the financial industry.The Team is working on a project related to data harmonization. Join us and take on new challengesYour RoleDesigning, maintaining and executing test...

  • Automation Test Engineer

    5 dni temu

    Warszawa, Mazovia, Polska Altexsoft Pełny etat

    Senior Automation QA EngineerWe are seeking a skilled Senior Automation QA Engineer to join our team at Altexsoft. The ideal candidate will have strong expertise in test automation, quality assurance, and software testing.The successful Senior Automation QA Engineer will be responsible for designing, developing, and maintaining automated test frameworks and...

  • Test Automation Engineer

    2 tygodni temu

    Warszawa, Mazovia, Polska Nord Security Pełny etat

    The Infrastructure department is responsible for influencing and tracking change, providing frontline support, and delivering software-defined solutions.We are seeking a skilled Test Automation Engineer (Python)/Python Developer – Automation & Testing to join our team. The ideal candidate will have strong experience in Python development, automation, and...

  • Test Automation Engineer

    1 tydzień temu

    Warszawa, Mazovia, Polska Nord Security Pełny etat

    The Infrastructure department is responsible for influencing and tracking change, providing frontline support, and delivering software-defined solutions.We are seeking a skilled Test Automation Engineer (Python) / Python Developer – Automation & Testing to join our team. The ideal candidate will have strong experience in Python development,...

  • Senior Test Automation Engineer with Python, Nordea Payments

    7 dni temu

    Warszawa, Mazovia, Polska TN Poland Pełny etat

    Social network you want to login/join with:Senior Test Automation Engineer with Python, Nordea Payments, WarsawClient:Nordea BankLocation:Warsaw, PolandJob Category:OtherEU work permit required:YesJob Reference:bc9c8f7c62e3Job Views:4Posted:24.03.2025Expiry Date:08.05.2025Job Description:Job ID: 26839Looking for an exciting career opportunity? Come join our...

  • Test Automation Engineer

    1 tydzień temu

    Warszawa, Mazovia, Polska TeamQuest Sp. z o.o. Pełny etat

    Our client is an innovative company operating in the ICT industry since 2023, specializing in NMS/OSS systems, system platforms, and as a Software House. It offers a wide range of products, from custom IT solutions, commercial solutions, to those based on open-source software, always tailored to the specific needs of the client. The company stands out in the...

  • Test Automation Engineer

    7 dni temu

    Warszawa, Mazovia, Polska ipracujzdalnie Pełny etat

    Praca na stanowisku: Test Automation EngineerOur client is an innovative company operating in the ICT industry since 2023, specializing in NMS/OSS systems, system platforms, and as a Software House. It offers a wide range of products, from custom IT solutions, commercial solutions, to those based on open-source software, always tailored to the specific needs...

  • Senior Automation Tester with Python @

    3 godzin temu

    Warszawa, Mazovia, Polska Capgemini Polska Sp. z o.o. Pełny etat

    Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and...

  • Agile Test Automation Engineer

    23 godzin temu

    Warszawa, Mazovia, Polska Lingaro Pełny etat

    Test automation framework development : designing, implementing, and maintaining robust test automation frameworks to support efficient and scalable testing. Evaluating and selecting appropriate tools and technologies for test automation. Test planning : collaborating with QA engineers, developers, and product managers to plan and prioritize automated...

  • Senior Performance and Test Automation Engineer

    1 tydzień temu

    Warszawa, Mazovia, Polska Link Group Pełny etat

    We are looking for a Test Automation Engineer to join our team and help us drive test automation and ensure the highest quality of our software solutions. You will collaborate with the team on testing and implementation activities while having a real impact on the processes and quality of delivered projects.Key Responsibilities:Collaborate with the team on...

  • Lead Test Automation Engineer

    4 tygodni temu

    Warszawa, Mazovia, Polska Square One Resources Sp. z o.o. Pełny etat

    About Us:We are Square One. For 30 years, we've specialized in IT recruitment, and we know exactly what IT experts like you need. If you're looking for new challenges, aren't afraid of skill-building projects, and want to stay on the technological cutting edge—then you've come to the right placeAbout the Project:Our client, who specializes is web and...