Senior LLM Engineer

We are looking for a Senior LLM Engineer to join the development of the Noa.ai product. Our mission is to revolutionize access to quality healthcare by alleviating the administrative burden on doctors, clinics, and medical personnel, enabling them to focus on patient care.

Currently, the team is dedicated to Noa Notes, an innovative solution that automates medical documentation creation freeing doctors from time-consuming administrative tasks.Technically, it blends Automatic Speech Recognition models with the power the LLMs power. Apart from the online inference system, we’ve built a comprehensive quality assurance and validation stack.

Our mission includes improving transcription quality by training ASR for medical terminology and low-quality audio, enhancing note quality by refining prompts to capture essential details while maintaining structure, and enabling personalized, structured medical notes.

But that’s not all - we are actively planning and exploring additional groundbreaking AI initiatives aimed at transforming healthcare through technology. As part of this team, you’ll play a key role in shaping the future of AI-driven healthcare solutions.

Responsibilities:

Prompt engineering

- Optimize prompts to improve recall and veracity, especially for medical terminology and concepts.

- Continuously refine input-output workflows to enhance LLM accuracy in complex medical scenarios.

- Identify common failure points in LLM outputs, particularly in medical contexts, and implement strategies to mitigate inaccuracies.

- Enable system-level personalization for healthcare providers by tailoring AI outputs to align with individual doctors' preferences and needs.

- Design methods for customized output formatting, ensuring compatibility with various medical systems (e.g., EHRs, diagnostic tools).

QA & validation

- Integrate quality assurance mechanisms to reduce misinformation in generated content.

- Build and maintain validation datasets to validate the improvements and run regression tests.

- Review reported cases to monitor potential errors and implement improvements to mitigate them.

Collaboration

- Collaborate with a Product Manager and Software Engineers to integrate structured AI outputs seamlessly into existing healthcare workflows.

- Work closely with other members of the ML/DS team to improve the overall performance of Noa (not only your silo).

- Collaborate with legal and security teams to ensure compliance with data processing policies.

What will help you thrive?

- Proven experience working with large language models (e.g., OpenAI, Anthropic, Llama).

- Track record in prompt engineering and fine-tuning language models.

- Experience in improving AI models for accuracy and customization in domain-specific applications.

- Proficiency in programming languages (eg Python, .Net).

- Ability to analyze and interpret user feedback for continuous system improvement.

- Problem-solving mindset and adaptability in dynamic environments with a focus on delivering business value to end customers.

- Proven ability to manage timelines, prioritize tasks, and deliver results under tight deadlines.

- Experience collaborating with cross-functional teams (e.g., product, marketing, engineering).

Bonus points

- Experience working with Speech Recognition / Transcription systems.

- Understanding of medical terminology and healthcare processes.

Let’s talk money

- A salary adequate to your experience and skills.

- Share options plan after 6 months of working with us.

True flexibility and work-life balance

- Remote or hybrid work model with or hub in Warsaw;

- Flexible working hours (fully flexible, as in most cases you only have to be on a couple of meetings weekly);

- 20/26 days of paid time off (depending on your contract);

- Additional paid day off on your birthday or work anniversary (you choose what you want to celebrate).

Health comes first

- Private healthcare plan with Signal Iduna for you and subsidized for your family.

- Multisport card co-financing for you to have access to sports facilities across Poland.

- Access to iFeel, a technological platform for mental wellness offering online psychological support and counseling.

Keep growing with us

- Free English classes.

Are you from abroad?

- Relocation support to Poland (we handle the bureaucracy, so you don’t have to).

We promote and embrace equal opportunities in our hiring process, and also every day at work. When you apply for our roles you receive equal treatment regardless of age, disabilities, gender reassignment, marital or civil partner status, pregnancy or parental status, race, colour, nationality, ethnic or national origin, religion or belief, sex, sexual orientation or any other dimension of human difference. If you require additional support in your recruitment process, we kindly encourage you to let us know. Behind those words you’re reading, there’s a person (hi) who already helped a candidate by adapting the interviews, and now we’re lucky to have this person with us. So, even if you’ve never asked for it before, may this serve as a sign that, now, you can do so. We can only truly be equal if we adapt to each other.

“We believe all humans, in all their beautiful diversity, should have equal rights, dignity and respect. Period.” Mariusz Gralewski, CEO

Welcome to the good side of tech ?

You might have heard about us, but with a different name: Doctoralia. It all started 12 years ago when we asked ourselves: is anyone in healthcare thinking about patients? We jumped in and we empowered patients by giving them access to leave and read reviews about their visit. We then provided doctors with the technology to manage bookings easily and save time, so they could devote themselves to what they always wanted: treating patients. And today is the day in which we ask you: wanna join us in the next step of making the healthcare experience more human?

Docplanner at scale

We are leaders in 13 countries so far, and more than 90 million patients trust us every month. almost 300.000 specialists believe in us and our product, and so do leading venture capital funds such as Point Nine Capital, Goldman Sachs Asset Management, and One Peak Partners. And yet, employing over 3.000people all over the globe, we managed to keep the startup mindset we started with over 12 years ago.

How does Docplanner Tech fit here?

At Docplanner Tech we are a diverse group of almost 400 people working in Engineering, Data, and Product teams. We are responsible for building the product for all locations. Many of us have been here for over 5 years, yet we still welcome each new person with great joy and excitement.

We could tell you about us, but we will let our reviews on Glassdoor speak for themselves. In case you’d like to see how it feels to be 100% yourself at work, here’s a video of us.

And why should you join us?

Because it feels good to tell your family and your friends how you made the world a little bit better. You go to bed knowing that what you do matters, and that your talents align with your beliefs.

We want to make the healthcare experience more human, and that starts with you being you. We believe that taking the diversity of human experience into account makes a better healthcare experience for all . We’re not just different: we embrace diversity. We will encourage you to come to work your whole self, and that includes not coming to the office at all if you prefer not to, as we're 100% remote-friendly.