QA Automation Engineer

4 tygodni temu

Kraków, małopolskie, Polska Patronus AI Pełny etat
About Patronus AI

Patronus AI is the leading automated AI evaluation and security company. We are on a mission to boost enterprise confidence in generative AI. Our world-class platform enables enterprise development teams to score LLM performance, generate adversarial test cases, benchmark LLMs, and more. Customers use Patronus AI to detect LLM mistakes at scale and deploy AI products safely and confidently.


Our founding team comes from top applied ML and research backgrounds, including Facebook AI Research (FAIR), Airbnb, Meta Reality Labs, and quant finance. As a team, we have published research papers at top ML conferences (NeurIPS, EMNLP, ACL), designed and launched Airbnb’s first conversational AI assistant, pioneered causal inference at Meta Reality Labs, exited a quant hedge fund backed by Mark Cuban, and scaled 0→1 products at high growth startups. We are backed by Lightspeed Venture Partners and high profile operators like Amjad Masad, Gokul Rajaram, and Fortune 500 executives and board members. We are advised by Douwe Kiela, Adjunct Professor at Stanford University and former Head of Research at HuggingFace.


Responsibilities

As the QA Engineer for Patronus AI, you are responsible for ensuring the quality and reliability of our AI products and systems. You will work closely with our engineering, AI and design teams to identify and resolve any potential issues or bugs, create testing plans and datasets, ensuring that our products meet the highest standards of performance and functionality. Successful applicants are skilled, have prior QA experience and are detail-oriented.


In this role, you will:

  • Design, execute, develop and lead QA processes. Taking responsibility for cross-team collaboration, development, planning and extending the team.
  • Develop and execute test plans to ensure the quality of software features. Create high quality, specific test cases that cover diverse scenarios and edge cases to catch unexpected behavior.
  • Ensure that production releases are well tested. Typically releases happen on a weekly cadence; QA will ensure reliability of new features in the web platform, collaborating closely with frontend engineers
  • Regularly report bugs and issues from testing results and file tasks; Triage to engineers accordingly
  • Provide feedback to product owner on areas for improvement
  • Collaborate with the engineering and AI teams to understand requirements for testing; Participate in technical design meetings as needed
  • Construct datasets for AI research, working with the AI team, including datasets for model training, finetuning and evaluation
  • Develop and maintain automated testing scripts for our API
  • Help customer success team investigate and troubleshoot customer-reported issues to identify root causes and recommend solutions
  • Executing load tests
  • Run benchmarking scripts to assess evaluator performance from configuration changes, eg. a change in the model selection, underlying dataset or other parameters


Qualifications

“The number one qualification to succeed in this machine learning course is gumption” - John Lafferty, CS Professor at Yale


Above all, we look for a proactive mindset, willingness to learn, relentless drive, and passion for engineering and product. You are a great fit if you have a background in the following:


  • Minimum of 3 years of experience as a QA engineer working on web platforms in a production environment
  • Minimum of at least 2 year of experience working on test automation with Python or Javascript
  • Experience with REST APIs
  • Experience performing manual application tests
  • Experience writing end-to-end tests
  • Experience with Selenium, Cypress or Playwright
  • Experience with tools like Postman, Swagger, OpenAPI
  • Have good character, integrity, and respect for others


Nice to haves:
  • Advanced proficiency in English
  • Experience with SQL
  • Experience with CI/CD tools like Jenkins, GitLab, GitHub
  • Certification or experience with advance ISTQB or other testing qualifications


Benefits
  • Competitive salary and equity packages
  • Unlimited PTO
  • Fun global offsites

  • QA Automation Engineer

    5 godzin temu

    Kraków, małopolskie, Polska ClearCourse Dev Pełny etat

    Job OverviewJoin a rapidly growing collaborative of disruptive technology companies working together to build a brilliant fintech payments business with exceptional people and the best products and services.Since our inception in 2018, we’ve scaled our portfolio to encompass nearly 40 leading brands and now number in excess of 1,000 employees globally....

  • Lead QA Automation Engineer

    5 godzin temu

    Kraków, małopolskie, Polska ClearCourse Dev Pełny etat

    Role description:Join a rapidly growing collaborative of disruptive technology companies working together to build a brilliant fintech payments business with exceptional people and the best products and services.Since our inception in 2018, we’ve scaled our portfolio to encompass nearly 40 leading brands and now number in excess of 1,000 employees...

  • Automation QA Engineer

    3 tygodni temu

    Kraków, małopolskie, Polska NeoGames Pełny etat

    Project description:We are NeoGames, a technology-driven innovator and a global leader of iLottery solutions and services to the national and state-regulated lotteries.We empower our customers to create the most successful iLottery programs.We are looking for an experienced Automation QA Engineer to develop an enterprise-scale web-based platform for online...

  • QA Automation Engineer

    5 godzin temu

    Kraków, małopolskie, Polska TransferRoom Pełny etat

    TransferRoom is a profitable B2B SaaS Marketplace on a mission to change the football transfer market for the better. We do this by empowering football clubs, agents and players to be successful in the transfer market by giving them real-time market intelligence and direct access to a global network of decision-makers.TransferRoom is market leading and has...

  • Automation QA Engineer

    4 tygodni temu

    Kraków, małopolskie, Polska NeoGames Pełny etat

    Project description:We are NeoGames, a technology-driven innovator and a global leader of iLottery solutions and services to the national and state-regulated lotteries.We empower our customers to create the most successful iLottery programs.We are looking for an experienced Automation QA Engineer to develop an enterprise-scale web-based platform for online...

  • Automation QA Engineer

    1 miesiąc temu

    Kraków, małopolskie, Polska NeoGames Pełny etat

    Project description:We are NeoGames, a technology-driven innovator and a global leader of iLottery solutions and services to the national and state-regulated lotteries.We empower our customers to create the most successful iLottery programs.We are looking for an experienced Automation QA Engineer to develop an enterprise-scale web-based platform for online...

  • QA Manager

    5 godzin temu

    Kraków, małopolskie, Polska FUN CRAFTERS SP. Z O O. Pełny etat

    As a QA Manager, you will play a key role in overseeing the quality assurance process for our games, from initial concept to final release. You will lead a team of QA specialists, establishing strategies and best practices and collaborate closely with the Tech Department Leads to build a sustainable and quality oriented workflow. Responsibilities:Build and...

  • Test Automation Engineer

    4 tygodni temu

    Kraków, małopolskie, Polska Brown Brothers Harriman Pełny etat

    At BBH we value diverse backgrounds, so if your experience looks a little different from what we've outlined and you think you can bring value to the role, we will still welcome your application!What You Can Expect At BBH:If you join BBH you will find a collaborative environment that enables you to step outside your role to add value wherever you can....

  • Test Automation Engineer

    1 miesiąc temu

    Kraków, małopolskie, Polska Brown Brothers Harriman Pełny etat

    At BBH we value diverse backgrounds, so if your experience looks a little different from what we've outlined and you think you can bring value to the role, we will still welcome your application!What You Can Expect At BBH:If you join BBH you will find a collaborative environment that enables you to step outside your role to add value wherever you can....

  • Senior QA Engineer

    4 tygodni temu

    Kraków, małopolskie, Polska TimeCamp S.A. Pełny etat

    Kilka słów o nas: Jesteśmy firmą technologiczną z Wrocławia, posiadającą własny produkt o globalnym zasięgu. TimeCamp od lat jest wykorzystywany przez tysiące klientów z całego świata. Pracujemy w oparciu o relacje z użytkownikami, słuchamy i obserwujemy.Jesteśmy blisko każdego ich problemu, a to co robimy pozwala zarządzać zadaniami i...

  • Senior QA Engineer

    1 miesiąc temu

    Kraków, małopolskie, Polska TimeCamp S.A. Pełny etat

    Kilka słów o nas: Jesteśmy firmą technologiczną z Wrocławia, posiadającą własny produkt o globalnym zasięgu. TimeCamp od lat jest wykorzystywany przez tysiące klientów z całego świata. Pracujemy w oparciu o relacje z użytkownikami, słuchamy i obserwujemy.Jesteśmy blisko każdego ich problemu, a to co robimy pozwala zarządzać zadaniami i...

  • DevOps Automation Engineer

    4 tygodni temu

    Kraków, małopolskie, Polska HSBC Service Delivery Pełny etat

    Some careers shine brighter than others.If you’re looking for a career that will help you stand out, join HSBC, and fulfil your potential. Whether you want acareer that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities,support and rewards that will take you further.Your career opportunityWell HSBC offers...

  • Java Automation Engineer

    5 godzin temu

    Kraków, małopolskie, Polska HSBC Service Delivery Pełny etat

    Some careers shine brighter than others.If you’re looking for a career that will help you stand out, join HSBC, and fulfil your potential. Whether you want acareer that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities,support and rewards that will take you further.Your career opportunityWell HSBC offers...

  • Senior Mobile QA Developer

    1 miesiąc temu

    Kraków, małopolskie, Polska IG Group Pełny etat

    As a Senior Mobile QA Developer, you will be part of a team of mobile engineers that builds out the technology that powers our retail brokerage business and supports our other business initiatives. From day one you will be testing our, iPhone, iPad, Android and applications. We care about the speed and stability of our customer-facing applications we build....

  • Senior Mobile QA Developer

    4 tygodni temu

    Kraków, małopolskie, Polska IG Group Pełny etat

    As a Senior Mobile QA Developer, you will be part of a team of mobile engineers that builds out the technology that powers our retail brokerage business and supports our other business initiatives. From day one you will be testing our, iPhone, iPad, Android and applications. We care about the speed and stability of our customer-facing applications we build....

  • Data Quality Engineer

    5 godzin temu

    Kraków, małopolskie, Polska NeoGames Pełny etat

    /Only for candidates based in Warsaw or Cracow who are comfortable working on Umowa o Pracę - Employment Contract (no possibility to work on B2B)/Data Quality EngineerJoin Aristocrat in Bringing Joy to Life Through the Power of Play. Be part of our growing global team where people come first because they fuel our success. Here, it’s All About the Player...

  • Site Reliability Engineer

    2 miesięcy temu

    Kraków, małopolskie, Polska TransferRoom Pełny etat

    TransferRoom is a profitable B2B SaaS Marketplace on a mission to change the football transfer market for the better. We do this by empowering football clubs, agents and players to be successful in the transfer market by giving them real-time market intelligence and direct access to a global network of decision-makers.TransferRoom is market leading and has...

  • Platform Engineer

    3 tygodni temu

    Kraków, małopolskie, Polska emagine Polska Pełny etat

    PROJECT INFORMATION:Work model: 100% remoteWorkload: Full-timeContract length: 6 months option for extensionAssignment type: B2BStart: ASAP / 1 month notice periodTrips: occasional to Denmark Job descripitionThe platform team at SAS responsible for providing baseline & building blocks of cloud environments using Microsoft enterprise landing zone concept...

  • Senior Cloud Platform Engineer

    5 godzin temu

    Kraków, małopolskie, Polska HSBC Service Delivery Pełny etat

    Some careers shine brighter than others.If you’re looking for a career that will help you stand out, join HSBC, and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.Your career opportunityThe most...

  • Principal Backend Engineer

    5 godzin temu

    Kraków, małopolskie, Polska Backbase Pełny etat

    To develop and deliver a production-ready banking platform across all verticals, therefore, being full stack is a big plus.What you'll doThis role is part of a Backbase product team in R&D. It is a multidisciplinary team of experienced and highly skilled UX designers, software developers, QA engineers, and technical writers. As part of the team, you will...