Senior Data Engineer

2 tygodni temu

Poznan, Polska Datumo Pełny etat

Datumo specializes in providing Big Data and Cloud consulting services to clients from all over the world, primarily in Western Europe, Poland and the USA.

Core industries we support include e-commerce, telecommunications and life science. Our team consists of exceptional people whose commitment allows us to conduct highly demanding projects .


Our team members tend to stick around for more than 3 years, and when a project wraps up, we don't let them go - we embark on a journey to discover exciting new challenges for them.

It's not just a workplace; it's a community that grows together

What we expect:

Must-have:

  • 4 years of commercial experience in Big Data
  • proven record with a selected cloud provider: GCP preferred, Azure or AWS
  • senior level knowledge of:
programming language - Scala/Java/JVM
distributed data processing - Spark or similar
pipeline orchestration - Airflow or similar
distributed datastore - BigQuery, Snowflake, Hive or similar

  • designing and implementing Big Data systems following best practices
  • good knowledge of Python
  • ensuring solution quality through automatic tests, CI / CD and code review
  • proven collaboration with businesses
  • English proficiency at B2 level, communicative in Polish

Nice to have:
experience in Snowflake/Databricks platform
knowledge of Apache Kafka, Docker and Kubernetes technologies
experience in Machine Learning projects
experience in Flink
willingness to share knowledge (conferences, articles, open-source projects)
experience in leading a team or willingness to gain it - for the position of Team Leader

What's on offer:
100% remote work, with workation opportunity
20 free days
onboarding with a dedicated mentor
project switching possible after a certain period
individual budget for training and conferences

benefits:
Medicover private medical care, co-financing of the Medicover Sport card
opportunity to learn English with a native speaker
regular company trips and informal get-togethers

Development opportunities in Datumo:
participation in industry conferences
establishing Datumo's online brand presence
support in obtaining certifications (e.g. GCP, Azure, Snowflake)
involvement in internal initiatives, like building technological roadmaps
training budget
access to internal technological training repositories

Discover our exemplary projects:
IoT data ingestion to cloud
The project integrates data from edge devices into the cloud using Azure services.

The platform supports data streaming via either the IoT Edge environment with Java or Python modules, or direct connection using Kafka protocol to Event Hubs.

It also facilitates batch data transmission to ADLS.

Data transformation from raw telemetry to structured tables is done through Spark jobs in Databricks or data connections and update policies in Azure Data Explorer.

Petabyte-scale data platform migration to Google Cloud

The goal of the project is to improve scalability and performance of the data platform by transitioning over a thousand active pipelines to GCP.

The main focus is on rearchitecting existing Spark applications to either Cloud Dataproc or Cloud BigQuery SQL, depending on the Client's requirements and automate it using Cloud Composer.

Data analytics platform for investing company
The project centers on developing and overseeing a data platform for an asset management company focused on ESG investing. Databricks is the central component. The platform, built on Azure cloud, integrates various Azure services for diverse functionalities. The primary task involves implementing and extending complex ETL processes that enrich investment data, using Spark jobs in Scala.

Integrations with external data providers, as well as solutions for improving data quality and optimizing cloud resources, have been implemented.

Realtime Consumer Data Platform
The initiative involves constructing a consumer data platform (CDP) for a major Polish retail company. Datumo actively participates from the project's start, contributing to planning the platform's architecture. The CDP is built on Google Cloud Platform (GCP), utilizing services like Pub/Sub, Dataflow and BigQuery.

Open-source tools, including a Kubernetes cluster with Apache Kafka, Apache Airflow and Apache Flink, are used to meet specific requirements.

This combination offers significant possibilities for the platform.

Recruitment process:
Quiz - 15 minutes
Soft skills interview - 30 minutes
Technical interview - 60 minutes

Find out more by visiting our website -

If you like what we do and you dream about creating this world with us - don't wait, apply now

  • Senior Data Engineer

    4 tygodni temu

    Poznan, Polska Limango Polska Pełny etat

    Senior Data Engineer @ Limango Remote/Wroclaw B2B contract – daily rate 900-1250pln Your role and main tasks Development of Central Lakehouse Data Platform based on AWS / Databricks ecosystem Managing data infrastructure (Terraform AWS / Databricks) and ETL code (PySpark, Delta Live Tables, SparkSQL) Implementing monitoring, data quality tests, unit...

  • Senior Data Engineer

    4 tygodni temu

    Poznan, Polska Rite NRG Pełny etat

    Dołącz do naszej ekipy  Rite NRG  jako  Senior Data Engineer  i stwórz z nami w Polsce centrum dostaw dla lidera branży iGaming. Będziesz miał możliwość rozwijania się w obszarze analizy danych i zarządzania danymi. Jakimi umiejętnościami możesz się pochwalić? Jesteś ekspertem w korzystaniu z narzędzi  AWS, Bardzo dobrze znasz...

  • Senior Data Engineer

    1 miesiąc temu

    Poznan, Polska Shaped Thoughts Pełny etat

    We are looking for a Senior Data Engineer who wants to join an outstanding engineering team that will develop a cutting-edge data platform in the insurance sector. Who are we ? We are a remote-first boutique software house company formed by a group of seasoned software experts (the average experience in the team is 12 years). What connected all of us,...

  • Senior Data Engineer

    4 tygodni temu

    Poznan, Polska Shaped Thoughts Pełny etat

    We are looking for a Senior Data Engineer who wants to join an outstanding engineering team that will develop a cutting-edge data platform in the insurance sector. Who are we ? We are a remote-first boutique software house company formed by a group of seasoned software experts (the average experience in the team is 12 years). What connected all of us,...

  • Senior Data Engineer Python Pyspark Cloud

    1 miesiąc temu

    Poznan, Polska Crestt Pełny etat

    Cześć! Jako Senior/Lead Data Engineer będziesz miał możliwość pracy nad kluczowym projektem naszego klienta, który obejmuje budowanie mechanizmu składowania danych decyzyjnych. Projekt ten oferuje unikalną szansę na zaangażowanie w tworzenie i implementację zaawansowanych systemów bazodanowych przy użyciu dużych zbiorów danych i...

  • Senior Data Engineer MS SQL

    1 miesiąc temu

    Poznan, Polska Opensignal Pełny etat

    Job title:  Senior Data Engineer (MS SQL) Department:  Engineering Reporting to:  Engineering Manager Location:  Poland Purpose of Role We’re looking for a Senior Data Engineer to join our Market Performance Group in transforming our real-world raw data into valuable and credible industry leading metrics that provide insights to our analysts and our...

  • Senior Data Engineer

    4 tygodni temu

    Poznan, Polska Opensignal Pełny etat

    Job title:  Senior Data Engineer (MS SQL) Department:  Engineering Reporting to:  Engineering Manager Location:  Poland Purpose of Role We’re looking for a Senior Data Engineer to join our Market Performance Group in transforming our real-world raw data into valuable and credible industry leading metrics that provide insights to our analysts and our...

  • Senior Software Engineer

    1 miesiąc temu

    Poznan, Polska Snowflake Pełny etat

    There is only one Data Cloud. Snowflake’s founders started from scratch and designed a data platform built for the cloud that is effective, affordable, and accessible to all data users. But it didn’t stop there. They engineered Snowflake to power the Data Cloud, where thousands of organizations unlock the value of their data with near-unlimited scale,...

  • Senior Data Engineer

    4 tygodni temu

    Poznan, Polska Amer Sports Pełny etat

    ❄️ Mid/Senior Data Engineer (Snowflake) ❄️ Hybrid /Remote work, Cracow You will be part of the Enterprise Data & Analytics team, building strong ties with our international teams in North America and Asia to ensure seamless database operations. Your role is crucial in safeguarding data integrity, optimizing processes, and supporting the diverse...

  • Senior Big Data Engineer

    6 dni temu

    Poznan, Polska Capgemini Polska Sp. z o.o. Pełny etat

    YOUR ROLE As a Senior Engineer, you will build distributed data processing solution and highly loaded database solutions for various businesses cases including reporting, product analytics, marketing optimization and financial reporting. Contribute as part of self-organized team of experienced data engineers working in a challenging, innovative environment...

  • Data Engineer

    6 dni temu

    Poznan, Polska MindPal Pełny etat

    Data Engineer Miejsce pracy: Poznań Technologies we use Expected Python SQL AWS About the project We are looking for a Data Engineer to join the team at MindPal, working for our key client in the technology sector. We offer the opportunity to work with modern technologies and tools, close collaboration with experienced professionals, participation in an...

  • Senior Backend Developer

    6 dni temu

    Poznan, Polska Data System Pełny etat

    Hej! Tutaj Data System. Przez ponad 18 lat świadczymy dla naszych klientów usługi monitoringu i lokalizacji pojazdów. Jest w tym sporo zabawy. Nasze małe lokalizatory, nie tylko pomagają znaleźć samochód w gąszczu innych na parkingu, ale też umożliwiają łatwe i precyzyjne rozliczanie przejazdów po płatnych drogach dzięki systemowi e-Toll....

  • Senior PHP Developer

    1 miesiąc temu

    Poznan, Polska Transparent Data Pełny etat

    Transparent Data to unikalna kombinacja mocnej technologii i doświadczenia w pracy z danymi. Wykorzystujemy dane z otwartych rejestrów dotyczących danych gospodarczych by dostarczać usługi i produkty dla klientów biznesowych. Tworzymy rozwiązania m.in dla. największych instytucji finansowych oraz działów zakupów, AML i controlingu liderów...

  • Data Engineer

    4 tygodni temu

    Poznan, Polska MindPal Pełny etat

    technologies-expected : SQL Python about-project : We are looking for a Data Engineer (Snowflake) to join the team at MindPal, working for our key client in the technology sector. We offer the opportunity to work with modern technologies and tools, close collaboration with experienced professionals, participation in an exciting project requiring innovative...

  • Data Engineer

    4 tygodni temu

    Poznan, Polska Pragmile Pełny etat

    Pragmile is running an innovative research & development project for a large Polish institution which should result in a unique solution to monitor media market. We're looking for an experienced engineer who could support us in the successful execution and delivery. Requirements: Strong understanding of data structures and algorithms Strong understanding of...

  • Senior DataOps Engineer @ Nordcloud

    1 miesiąc temu

    Poznan, Polska Nordcloud Pełny etat

    Join Nordcloud and make your mark on the European IT industry. Help our clients thrive on their cloud journey in solution areas such as infrastructure, migration, data, and security. Currently, we are looking for a Senior DataOps Engineer for our team in Poland. About Nordcloud Nordcloud, an IBM company, is a European leader in cloud advisory,...

  • Data Engineer

    6 dni temu

    Poznan, Polska Ework Group Pełny etat

    For our client - a company from pharmaceutical area, we are looking for Data Engineer. What you will be doing You will be close to the heart of our client’s clinical operations where you will play a key role in shaping the future of clinical trials and patient care, by building scalable solutions in the cloud. Your key responsibilities include the...

  • Senior QA Automation Engineer

    4 tygodni temu

    Poznan, Polska Egnyte Poland Pełny etat

    Senior QA Automation Engineer Egnyte is the secure multi-cloud platform for content security and governance that enables organizations to better protect and collaborate on their most valuable content. Established in 2008, Egnyte has democratized cloud content security for more than 22,000 organizations, helping customers improve data security, maintain...

  • Senior QA Automation Engineer

    4 tygodni temu

    Poznan, Polska Egnyte Poland Pełny etat

    Senior QA Automation Engineer Egnyte is the secure multi-cloud platform for content security and governance that enables organizations to better protect and collaborate on their most valuable content. Established in 2008, Egnyte has democratized cloud content security for more than 22,000 organizations, helping customers improve data security, maintain...

  • Hyperscaler Engineer @ NTT DATA Business Solutions

    4 tygodni temu

    Poznan, Polska NTT DATA Business Solutions Pełny etat

    That is what makes the NTT DATA Business Solutions spirit special: Team-oriented corporate culture, collaboration as equals and steady knowledge transfer Sustainable career development in a growth-oriented technology environment with a business focus on SAP full service (financing certificates and trainings) Remote work possibilities Benefits f.e. private...