Senior Data Engineer – AI Healthcare Analytics

Senior Data Engineer – AI Healthcare Analytics Miejsce pracy: Warszawa Technologies we use Expected Python AWS SQL Optional PostgreSQL About the project As a Senior Data Engineer – AI Healthcare Analytics, you will be working for our client, a leader in healthcare data solutions dedicated to transforming complex health data into actionable insights. You'll build scalable data pipelines and infrastructure that fuel AI and analytics projects, empowering smarter patient care and operational excellence. Unleash the Power of Data — Shape the Future of Healthcare Intelligence Poland or Portugal-based opportunity with a fully remote work model (5 days per week). Only candidates with an existing legal right to work in Europe will be considered for this role. Your responsibilities Design and develop production ETL/ELT pipelines from diverse healthcare data sources, including claims, patient records, and referrals, into S3 data lakes and data warehouses. Implement and manage batch workflows with orchestration, error handling, retries, and lineage tracking. Write modern Python-based data processing jobs using the latest libraries to transform and merge healthcare datasets. Build and optimize AWS data infrastructure, including S3, Athena, IAM, and Glue, focusing on cost efficiency and query performance. Integrate healthcare-specific data such as ICD/CPT codes, NPI data, and demographic details, normalizing and processing them for analytical use. Develop and optimize complex SQL queries for healthcare metrics, cohort analysis, patient segmentation, and performance dashboards. Ensure data quality through validation, schema enforcement, anomaly detection, and automated validation pipelines. Profile data for completeness, accuracy, and duplication, preparing comprehensive data quality reports. Monitor and optimize data pipelines for throughput, memory efficiency, and cost — implementing caching, partitioning, and cost-tracking strategies. Collaborate using Git, maintain detailed documentation, and develop clear technical specifications and data dictionaries. Our requirements At least 5 years of professional experience building production-grade ETL/ELT pipelines processing large volumes of healthcare data. Strong proficiency in Python (3.12) and experience with modern data processing libraries. Hands-on AWS experience with S3, Athena, IAM, and Glue, including cost management and schema design. Expertise in writing and optimizing advanced SQL queries on datasets with millions or billions of rows, utilizing CTEs and window functions. Experience working with large-scale datasets and performance trade-offs. Good understanding of development practices such as Git workflows, code reviews, and documentation. Fluent English Optional Healthcare data experience — claims processing, patient journey analysis, ICD/CPT coding, PHI/HIPAA compliance. Familiarity with healthcare provider data (NPI, facility types). Multi-cloud experience with Azure Data Factory, Synapse, or ADLS. Knowledge of LLM data workflows, prompt engineering, and structured output formats like JSON or YAML. Experience with data modeling for analytics (star/snowflake schemas, data warehouses). Exposure to metadata and data catalog tools. Skills in orchestration tools like Airflow, Dagster, or Step Functions. Experience with PostgreSQL, Supabase, or serverless Postgres solutions. This is how we organize our work This is how we work at the client's site agile scrum What we offer Stable and long-term cooperation with very good conditions Enhance your skills and develop your expertise in the financial industry Work on the most strategic projects available in the market Define your career roadmap and develop yourself in the best and fastest possible way by delivering strategic projects for different clients of ITDS over several years Participate in Social Events, training, and work in an international environment Access to attractive Medical Package Access to Multisport Program Access to Pluralsight Flexible hours Benefits sharing the costs of sports activities private medical care remote work opportunities flexible working time fruits integration events corporate gym saving & investment scheme no dress code coffee / tea drinks christmas gifts birthday celebration sharing the costs of a streaming platform subscription access to 100 projects access to Pluralsight Recruitment stages first online interview second online interview GETREADY to meet with us ITDS Business Consultants is involved in many various, innovative and professional IT projects for international companies in the financial industry in Europe. We offer an environment for professional, ambitious, and driven people. We would like to meet you. If you are interested please apply and attach your CV in English or Polish, including a statement that you agree to our processing and storing of your personal data. ITDS's Whistleblower Procedure You can report violations in accordance with ITDS's Whistleblower Procedure available here: https://itds.pl/wp-content/uploads/2025/03/INFORMATION-ON-PROCEDURE-FOR-REPORTING-BREACHES-AND-PROTECTION-OF-WHISTLEBLOWERS-AT-ITDS-POLSKA.pdf ITDS Polska Sp. z o.o. ITDS supports financial service providers to take the next steps. We identify what's possible, every day. Opportunities in the areas of technology, organization, and digitization. We see where banks, insurers, payment companies, or fintech can go and how they can get there. That's why we want to stimulate you to ramp up your ambition. Forget what you perceive as restraints and step towards the new reality. ITDS in Poland - Pure player in new Technologies & Financial Industry 300 IT implementation professionals 20 clients in the Banking, Insurance, Payment & Fintech Industry ITDS excels in digital strategy delivery and implementation of best-of-breed lending solutions. ITDS has delivered successful strategic projects throughout Europe since 1998. We combine the experience we've accumulated with in-depth knowledge of technologies, business processes, and EU legislation to unlock new business opportunities. Informujemy, że administratorem danych jest ITDS z siedzibą w Warszawie, ul. Złota 59 (dalej jako "administrator"). Masz prawo do żądania dostępu do swoich danych osobowych, ich sprostowania, usunięcia lub ograniczenia przetwarzania, prawo do wniesienia sprzeciwu wobec przetwarzania, a także prawo do przenoszenia danych oraz wniesienia skargi do organu nadzorczego. Dane osobowe przetwarzane będą w celu realizacji procesu rekrutacji. Podanie danych w zakresie wynikającym z ustawy z dnia 26 czerwca 1974 r. Kodeks pracy jest obowiązkowe. W pozostałym zakresie podanie danych jest dobrowolne. Odmowa podania danych obowiązkowych może skutkować brakiem możliwości przeprowadzenia procesu rekrutacji. Administrator przetwarza dane obowiązkowe na podstawie ciążącego na nim obowiązku prawnego, zaś w zakresie danych dodatkowych podstawą przetwarzania jest zgoda. Dane osobowe będą przetwarzane do czasu zakończenia postępowania rekrutacyjnego i przez okres możliwości dochodzenia ewentualnych roszczeń, a w przypadku wyrażenia zgody na udział w przyszłych postępowaniach rekrutacyjnych - do czasu wycofania tej zgody. Zgoda na przetwarzanie danych osobowych może zostać wycofana w dowolnym momencie. Odbiorcą danych jest serwis Hello HR oraz inne podmioty, którym powierzyliśmy przetwarzanie danych w związku z rekrutacją. Naruszenia można zgłaszać zgodnie z Procedurą zgłaszania nieprawidłowości ITDS dostępną tutaj:https://itds.pl/wp-content/uploads/2025/03/INFORMATION-ON-PROCEDURE-FOR-REPORTING-BREACHES-AND-PROTECTION-OF-WHISTLEBLOWERS-AT-ITDS-POLSKA.pdf