Senior Data Engineer

2 tygodni temu

Poznań, Greater Poland Datumo Pełny etat

Datumo specializes in providing Big Data and Cloud consulting services to clients from all over the world, primarily in Western Europe, Poland and the USA. Core industries we support include e-commerce, telecommunications and life science. Our team consists of exceptional people whose commitment allows us to conduct highly demanding projects .

Our team members tend to stick around for more than 3 years, and when a project wraps up, we don't let them go - we embark on a journey to discover exciting new challenges for them. It's not just a workplace; it's a community that grows together

What we expect:

Must-have:

● 4 years of commercial experience in Big Data

● proven record with a selected cloud provider: GCP preferred, Azure or AWS

● senior level knowledge of:

  • programming language - Scala/Java/JVM
  • distributed data processing - Spark or similar
  • pipeline orchestration - Airflow or similar
  • distributed datastore - BigQuery, Snowflake, Hive or similar

● designing and implementing Big Data systems following best practices

● good knowledge of Python

● ensuring solution quality through automatic tests, CI / CD and code review

● proven collaboration with businesses

● English proficiency at B2 level, communicative in Polish

Nice to have:

  • experience in Snowflake/Databricks platform
  • knowledge of Apache Kafka, Docker and Kubernetes technologies
  • experience in Machine Learning projects
  • experience in Flink
  • willingness to share knowledge (conferences, articles, open-source projects)
  • experience in leading a team or willingness to gain it - for the position of Team Leader

What's on offer:

  • 100% remote work, with workation opportunity
  • 20 free days
  • onboarding with a dedicated mentor
  • project switching possible after a certain period
  • individual budget for training and conferences
  • benefits: Medicover private medical care, co-financing of the Medicover Sport card
  • opportunity to learn English with a native speaker
  • regular company trips and informal get-togethers

Development opportunities in Datumo:

  • participation in industry conferences
  • establishing Datumo's online brand presence
  • support in obtaining certifications (e. g. GCP, Azure, Snowflake)
  • involvement in internal initiatives, like building technological roadmaps
  • access to internal technological training repositories

Discover our exemplary projects:

IoT data ingestion to cloud

The project integrates data from edge devices into the cloud using Azure services. The platform supports data streaming via either the IoT Edge environment with Java or Python modules, or direct connection using Kafka protocol to Event Hubs. It also facilitates batch data transmission to ADLS. Data transformation from raw telemetry to structured tables is done through Spark jobs in Databricks or data connections and update policies in Azure Data Explorer.

Petabyte-scale data platform migration to Google Cloud

The goal of the project is to improve scalability and performance of the data platform by transitioning over a thousand active pipelines to GCP. The main focus is on rearchitecting existing Spark applications to either Cloud Dataproc or Cloud BigQuery SQL, depending on the Client's requirements and automate it using Cloud Composer.

Data analytics platform for investing company

The project centers on developing and overseeing a data platform for an asset management company focused on ESG investing. Databricks is the central component. The platform, built on Azure cloud, integrates various Azure services for diverse functionalities. The primary task involves implementing and extending complex ETL processes that enrich investment data, using Spark jobs in Scala. Integrations with external data providers, as well as solutions for improving data quality and optimizing cloud resources, have been implemented.

Realtime Consumer Data Platform

The initiative involves constructing a consumer data platform (CDP) for a major Polish retail company. Datumo actively participates from the project's start, contributing to planning the platform's architecture. The CDP is built on Google Cloud Platform (GCP), utilizing services like Pub/Sub, Dataflow and BigQuery. Open-source tools, including a Kubernetes cluster with Apache Kafka, Apache Airflow and Apache Flink, are used to meet specific requirements. This combination offers significant possibilities for the platform.

Recruitment process:

  • Soft skills interview - 30 minutes
  • Technical interview - 60 minutes

Find out more by visiting our website -

If you like what we do and you dream about creating this world with us - don't wait, apply now

Firma: Datumo

Dodano:
Praca na stanowisku - aktualna

Otrzymuj nowe oferty e-mailem

#J-18808-Ljbffr

  • Senior Data Engineer

    2 tygodni temu

    Poznań, Greater Poland Lite e-Commerce Pełny etat

    Senior Data Engineer Poznań, Województwo wielkopolskie Senior Data Engineer Poznań Poznań, Greater Poland Voivodeship, Polska Informacje: Lokalizacja: Poznań Poznań, Greater Poland Voivodeship, Polska Dodano: Praca na stanowisku - aktualna Praca Poznań - Ciekawe oferty pracy w okolicy:PoznańOferty pracy w okolicznych lokalizacjachPoznań: Inne...

  • Senior Data Engineer Flink

    2 tygodni temu

    Poznań, Greater Poland GetInData | Part of Xebia Pełny etat

    Senior Data Engineer Flink Poznań, Województwo wielkopolskie Senior Data Engineer Flink Poznań Poznań, Greater Poland Voivodeship, Polska Informacje: Lokalizacja: Poznań Poznań, Greater Poland Voivodeship, Polska Dodano: Praca na stanowisku - aktualna Praca Poznań - Ciekawe oferty pracy w okolicy:PoznańOferty pracy w okolicznych...

  • Senior Data Engineer

    2 tygodni temu

    Poznań, Greater Poland j-labs Pełny etat

    Senior Data Engineer Poznań, Województwo wielkopolskie Senior Data Engineer Poznań Poznań, Greater Poland Voivodeship, Polska Dołącz do zespołu odpowiedzialnego za przetwarzanie danych potrzebnych do generowania raportów.Masz okazję zostać częścią projektu, który ma na celu zrewolucjonizowanie branży zarządzania inwestycjami.Platforma pozwala...

  • Senior Data Engineer

    2 tygodni temu

    Poznań, Greater Poland Rite NRG Pełny etat

    Senior Data Engineer Poznań, Województwo wielkopolskie Senior Data Engineer Poznań Poznań, Greater Poland Voivodeship, Polska Dołącz do naszej ekipy Rite NRG jako Senior Data Engineer i stwórz z nami w Polsce centrum dostaw dla lidera branży iGaming. Będziesz miał możliwość rozwijania się w obszarze analizy danych i zarządzania danymi.Jakimi...

  • Senior Data Engineer

    2 tygodni temu

    Poznań, Greater Poland Sigma Software Pełny etat

    Seeking Senior Data Engineer to lead a small team, modernize a simple data platform into a cloud-based self-service analytics project.CUSTOMER Sigma Software provides top-quality software development, graphic design, testing, and support services.Systems developed to deliver customer benefits in automotive, telecommunications, aviation, advertising, gaming,...

  • Senior Data Engineer z

    2 tygodni temu

    Poznań, Greater Poland dmTECH Polska Pełny etat

    Senior Data Engineer z j.niemieckim Poznań, Województwo wielkopolskie Senior Data Engineer z j.niemieckim Poznań Poznań, Województwo wielkopolskie, Polska dmTECH to międzynarodowy zespół ponad tysiąca ekspertek i ekspertów z różnych obszarów IT i technologii wspierających e-commerce pracujących dla dm – jednej z największych sieci drogerii...

  • Senior Data Engineer @

    2 tygodni temu

    Poznań, Greater Poland Capgemini Polska Sp. z o.o. Pełny etat

    Capgemini Engineering is a world leader in engineering and R&D services. We combine our broad industry knowledge and cutting-edge technologies in digital and software to support the convergence of the physical and digital worlds. Every day, we help our clients to accelerate their journey towards Intelligent Industry.At Capgemini, we are a responsible and...

  • Senior Data Engineer Python Pyspark Cloud

    2 tygodni temu

    Poznań, Greater Poland Crestt Pełny etat

    Senior Data Engineer Python Pyspark Cloud Poznań, Województwo wielkopolskie Senior Data Engineer Python Pyspark Cloud Poznań Poznań, Greater Poland Voivodeship, Polska CześćJako Senior/Lead Data Engineer będziesz miał możliwość pracy nad kluczowym projektem naszego klienta, który obejmuje budowanie mechanizmu składowania danych decyzyjnych....

  • Senior Data Engineer z jniemieckim

    2 tygodni temu

    Poznań, Greater Poland dmTECH Polska Pełny etat

    Senior Data Engineer z jniemieckim Poznań, Województwo wielkopolskie Senior Data Engineer z jniemieckim Poznań Poznań, Greater Poland Voivodeship, Polska dmTECH to międzynarodowy zespół ponad tysiąca ekspertek i ekspertów z różnych obszarów IT i technologii wspierających e-commerce pracujących dla dm – jednej z największych sieci drogerii w...

  • Senior Data Engineer

    2 tygodni temu

    Poznań, Greater Poland Shaped Thoughts Pełny etat

    We are looking for a Senior Data Engineer who wants to join an outstanding engineering team that will develop a cutting-edge data platform in the insurance sector.Who are we?We are a remote-first boutique software house company formed by a group of seasoned software experts (the average experience in the team is 12 years). What connected all of us, despite a...

  • Senior Data Engineer with Databricks

    2 tygodni temu

    Poznań, Greater Poland INSIGHT Pełny etat

    We assist our customers in developing IoT services, portals, and mobile apps and, above all, analyze data using AI and advanced analytics to reduce costs, streamline and find new opportunities for the business. Our goal is to grow strong, but not in numbers, but rather in competencies. We get things done and, on the way, we build a dynamic company culture to...

  • Senior Data Engineer

    2 tygodni temu

    Poznań, Greater Poland Ework Group Pełny etat

    We are seeking a skilled and experienced Data Engineer to join our client (pharmaceutical area) team. The ideal candidate will have a strong background in data architecture, data modeling, data warehousing and ETL processes. The Data Engineer will be responsible for designing, building, and maintaining the data infrastructure that supports our organization's...

  • Senior Cloud Data Engineer Azure and Snowflake

    2 tygodni temu

    Poznań, Greater Poland Future Processing Pełny etat

    Senior Cloud Data Engineer Azure and Snowflake Poznań, Województwo wielkopolskie Senior Cloud Data Engineer Azure and Snowflake Poznań Poznań, Greater Poland Voivodeship, Polska Szukamy Ciebie, jeśli:masz min. 5 lat doświadczenia w IT, w tym min. 3, 5 roku w pracy z danymi w chmurze (potwierdzone projektami komercyjnymi wdrożonymi na...

  • Data Engineer

    2 tygodni temu

    Poznań, Greater Poland Experis ManpowerGroup Sp. z o.o. Pełny etat

    Data Engineer We are seeking a talented Data Engineer with 4+ years of experience to join our dynamic team. The ideal candidate will be passionate about transforming raw data into valuable insights and possess a strong proficiency in SQL, T-SQL, and various data manipulation and visualization tools. If you're looking for an opportunity to work with...

  • Data Engineer

    2 tygodni temu

    Poznań, Greater Poland Mindpal Pełny etat

    Technologies-expected : SQL Python about-project : We are looking for a Data Engineer (Snowflake) to join the team at Mind Pal, working for our key client in the technology sector.We offer the opportunity to work with modern technologies and tools, close collaboration with experienced professionals, participation in an exciting project requiring innovative...

  • Senior PHP Developer

    2 tygodni temu

    Poznań, Greater Poland Transparent Data Pełny etat

    Senior PHP Developer Poznań Poznań, Greater Poland Voivodeship, Polska Transparent Data to unikalna kombinacja mocnej technologii i doświadczenia w pracy z danymi. Wykorzystujemy dane z otwartych rejestrów dotyczących danych gospodarczych by dostarczać usługi i produkty dla klientów biznesowych. Tworzymy rozwiązania m. in dla. największych...

  • Data Engineer

    2 tygodni temu

    Poznań, Greater Poland MindPal Pełny etat

    Technologies-expected : Python SQL AWS about-project : We are looking for a Data Engineer to join the team at Mind Pal, working for our key client in the technology sector.We offer the opportunity to work with modern technologies and tools, close collaboration with experienced professionals, participation in an exciting project requiring innovative...

  • Senior MLOps Engineer

    2 tygodni temu

    Poznań, Greater Poland hubQuest Pełny etat

    Senior MLOps Engineer Poznań, Greater Poland Voivodeship We are a team of experts, bringing together the best talents in IT and analytics. Our mission is to provide solutions through our flagship service, which includes forming tech teams from scratch, and growing existing units, all tailored to help our partners to become real data driven...

  • Senior PHP Engineer

    2 tygodni temu

    Poznań, Greater Poland Supermetrics Pełny etat

    Senior PHP Engineer Poznań, Greater Poland Voivodeship We're seeking a Senior PHP Engineer to join our Connectors team at SupermetricsThe team:The Connectors group works with different group-wide initiatives spanning connectors, aiming to improve our and our customers' abilities to get data from different sources and improve usage experience. In other...

  • Senior QA Automation Engineer

    2 tygodni temu

    Poznań, Greater Poland Egnyte Poland Pełny etat

    Senior QA Automation Engineer Poznań, Greater Poland Voivodeship Senior QA Automation EngineerEgnyte is the secure multi-cloud platform for content security and governance that enables organizations to better protect and collaborate on their most valuable content. Established in 2008, Egnyte has democratized cloud content security for more than 22, 000...