Senior Data Engineer

2 tygodni temu

Warsaw, Polska DATUMO sp. z o.o. Pełny etat

Senior Data Engineer Miejsce pracy: Warszawa Technologies we use Expected Google Cloud Platform
Azure DevOps
AWS
Big Data
Apache Airflow
Apache Spark
Snowflake Data Cloud
Python
Scala
Optional Docker
Kafka
Databricks
Your responsibilities Our team members tend to stick around for more than 3 years, and when a project wraps up, we don't let them go - we embark on a journey to discover exciting new challenges for them. It's not just a workplace; it's a community that grows together
Our requirements 4 years of commercial experience in Big Data
proven record with a selected cloud provider: GCP preferred, Azure or AWS
senior level knowledge of: programming language - Scala/Java/JVM, distributed data processing - Spark or similar , pipeline orchestration - Airflow or similar, distributed datastore - BigQuery, Snowflake, Hive or similar, designing and implementing Big Data systems following best practices
good knowledge of Python
ensuring solution quality through automatic tests, CI / CD and code review
proven collaboration with businesses
English proficiency at B2 level, communicative in Polish
Optional experience in Snowflake/Databricks platform
knowledge of Apache Kafka, Docker and Kubernetes technologies
experience in Machine Learning projects
experience in Flink
willingness to share knowledge (conferences, articles, open-source projects)
experience in leading a team or willingness to gain it - for the position of Team Leader
Division of working time 65% Development
10% Meetings
10% Fixing bugs
10% Your own ideas
5% Documentation
This is how we organize our work Team size 4 - 7
This is how we work at the client's site
you focus on a single project at a time
you can change the project
you have influence on the choice of tools and technologies
you focus on product development
agile
Development opportunities we offer conferences in Poland
development budget
intracompany training
mentoring
technical knowledge exchange within the company
time for development of your ideas
What we offer 100% remote work, with workation opportunity
20 free days
onboarding with a dedicated mentor
project switching possible after a certain period
individual budget for training and conferences
benefits: Medicover private medical care, co-financing of the Medicover Sport card
opportunity to learn English with a native speaker
regular company trips and informal get-togethers
Benefits sharing the costs of sports activities
private medical care
sharing the costs of professional training & courses
remote work opportunities
flexible working time
integration events
Recruitment stages Quiz - 15 minutes
Soft skills interview - 30 minutes
Technical interview - 60 minutes
Development opportunities in Datumo: participation in industry conferences
establishing Datumo's online brand presence
support in obtaining certifications (e.g. GCP, Azure, Snowflake)
involvement in internal initiatives, like building technological roadmaps
training budget
access to internal technological training repositories
Discover our exemplary projects: IoT data ingestion to cloud

The project integrates data from edge devices into the cloud using Azure services. The platform supports data streaming via either the IoT Edge environment with Java or Python modules, or direct connection using Kafka protocol to Event Hubs. It also facilitates batch data transmission to ADLS. Data transformation from raw telemetry to structured tables is done through Spark jobs in Databricks or data connections and update policies in Azure Data Explorer.

Petabyte-scale data platform migration to Google Cloud

The goal of the project is to improve scalability and performance of the data platform by transitioning over a thousand active pipelines to GCP. The main focus is on rearchitecting existing Spark applications to either Cloud Dataproc or Cloud BigQuery SQL, depending on the Client’s requirements and automate it using Cloud Composer.

Data analytics platform for investing company

The project centers on developing and overseeing a data platform for an asset management company focused on ESG investing. Databricks is the central component. The platform, built on Azure cloud, integrates various Azure services for diverse functionalities. The primary task involves implementing and extending complex ETL processes that enrich investment data, using Spark jobs in Scala. Integrations with external data providers, as well as solutions for improving data quality and optimizing cloud resources, have been implemented.

Realtime Consumer Data Platform

The initiative involves constructing a consumer data platform (CDP) for a major Polish retail company. Datumo actively participates from the project’s start, contributing to planning the platform’s architecture. The CDP is built on Google Cloud Platform (GCP), utilizing services like Pub/Sub, Dataflow and BigQuery. Open-source tools, including a Kubernetes cluster with Apache Kafka, Apache Airflow and Apache Flink, are used to meet specific requirements. This combination offers significant possibilities for the platform.
DATUMO sp. z o.o. Datumo specializes in providing Big Data and Cloud consulting services to clients from all over the world, primarily in Western Europe, Poland and the USA. Core industries we support include e-commerce, telecommunications and life science. Our team consists of exceptional people whose commitment allows us to conduct highly demanding projects.
Klikając w przycisk „Aplikuj”, „Aplikuj teraz” lub w inny sposób wysyłając zgłoszenie rekrutacyjne do DATUMO sp. z o.o. z siedzibą w Warszawie, ul. Dziekońskiego 1, 00-728 (administrator danych), zgadzasz się na przetwarzanie przez DATUMO sp. z o.o. Twoich danych osobowych zawartych w zgłoszeniu rekrutacyjnym, w celu prowadzenia rekrutacji na stanowisko wskazane w ogłoszeniu. W każdym czasie możesz cofnąć zgodę kontaktując się z nami pod adresem e-mail: contact@datumo.pl.Twoje dane osobowe możemy przetwarzać także w celu prowadzenia przyszłych rekrutacji na różne stanowiska (w oparciu o Twoją zgodę). Podanie danych osobowych wskazanych w przepisach prawa, niezbędnych do podjęcia działań na Twoje żądanie przed zawarciem umowy lub niezbędnych w naszym prawnie uzasadnionym interesie, jest obowiązkowe, żeby wziąć udział w rekrutacji. Podanie pozostałych danych jest dobrowolne.Masz prawo dostępu do swoich danych, w tym uzyskania ich kopii, sprostowania danych, żądania ich usunięcia, ograniczenia przetwarzania, wniesienia sprzeciwu wobec przetwarzania.Pełną informację odnośnie przetwarzania Twoich danych osobowych znajdziesz na www.datumo.io/privacy-policy

  • Senior Data Engineer

    3 tygodni temu

    Warsaw, Polska Digital Turbine Pełny etat

    Senior Data Engineer Warsaw, PolandEMEA – AGP Tech /Regular /HybridAt Digital Turbine , we make mobile advertising experiences more meaningful and rewarding for users, app publishers, and advertisers — intelligently connecting people in more ways, across more devices. We provide app publishers and advertisers with powerful ads and experiences that...

  • Senior Data Engineer

    2 tygodni temu

    Warsaw, Polska Data Science UA Pełny etat

    About us: Data Science UA is a service company with strong data science and AI expertise. Our journey began in 2016 with the organization of the first Data Science UA conference, setting the foundation for our growth. Over the past 7 years, we have diligently fostered the largest Data Science Community in Eastern Europe, boasting a network of over 30,000...

  • Senior Data Engineer

    1 miesiąc temu

    Warsaw, Polska Appcast Inc Pełny etat

    Job Description The Senior Data Engineer will serve as a liaison between the Appcast core development team and downstream analysts on the Data and Insights team. The senior data engineer sets up the data engineering infrastructure, using the tools of your choice. From the architecture you establish, you will then code automated processes that extract...

  • Senior Structural Engineer, EMEA

    7 dni temu

    Warsaw, Polska Vantage Data Centers Pełny etat

    About Vantage Data Centers Vantage is committed to being a workplace of inclusion, equity, respect and acceptance. We celebrate diversity and intentionally seek out opportunities to learn from one another’s experience. Vantage Data Centers powers, cools, protects and connects the technology of the world’s well-known hyperscalers, cloud...

  • Senior Data Engineer

    3 tygodni temu

    Warsaw, Polska VGW Pełny etat

    Senior Data Engineer VGW is an interactive entertainment company, harnessing technology and creativity to deliver world-class, free-to-play games.   We have an exciting opportunity to join our Engineering team in Poland, and we are currently looking for a Senior Data Engineer to join the team. As a Senior Data Engineer in our Growth team, you will...

  • Senior Data Engineer

    1 miesiąc temu

    Warsaw, Polska Appcast Pełny etat

    Job DescriptionThe Senior Data Engineer will serve as a liaison between the Appcast core development team and downstream analysts on the Data and Insights team. The senior data engineer sets up the data engineering infrastructure, using the tools of your choice. From the architecture you establish, you will then code automated processes that extract data...

  • Senior Data Engineer

    1 miesiąc temu

    Warsaw, Polska Reckitt Pełny etat

    Senior Data Engineer City: Warszawa ​​​About Us We’re a diverse, global family of over 40,000 people united by one passionate purpose: a cleaner, healthier world for all. We are Reckitt. Home to the world's best loved and trusted hygiene, health and nutrition brands. Our IT&D Digital transformation is well under way, and we need innovators and...

  • Senior Data Engineer

    7 dni temu

    Warsaw, Polska AI Clearing Pełny etat

    At AI Clearing, we disrupt the construction industry with AI! We are a Warsaw-based SaaS startup that applies deep learning models to crunch drone-based data and project designs to deliver the new quality of construction site analytics. Our platform is universally acclaimed by the biggest players in the industry and we already monitor dozens of construction...

  • Senior Data Engineer

    2 tygodni temu

    Warsaw, Polska LITE E-COMMERCE sp. z o.o. Pełny etat

    technologies-expected : Python SQL technologies-optional : PubSub Kafka Terraform Docker Kubernetes about-project : We are currently looking for a Senior Data Engineer, who will become Lite’s hero or heroine in the mission of technological development of the world! responsibilities : designing and building a platform for ETL/ELT; designing of ELT/ETL...

  • Senior Data Engineer

    1 miesiąc temu

    Warsaw, Polska Digital Turbine Pełny etat

    Senior Data Engineer Warsaw, PolandEMEA – Tech - Data /Full-time /HybridAt Digital Turbine , we make mobile advertising experiences more meaningful and rewarding for users, app publishers, and advertisers — intelligently connecting people in more ways, across more devices. We provide app publishers and advertisers with powerful ads and experiences...

  • Senior Data Engineer

    3 dni temu

    Warsaw, Polska SwingDev Pełny etat

    We are a Polish branch of a fast-growing InsurTech product company from Silicon Valley - Hippo Insurance. Our mission is to revolutionize home insurance in the US, starting from IoT monitoring devices to our industry-leading software. How do we want to get there? We need a top-notch talent, just like you! Putting a lot of effort into hiring top-tier...

  • Senior Data Engineer

    1 miesiąc temu

    Warsaw, Polska Ecovadis Pełny etat

    Description We are looking for a Senior Data Engineer to join our growing data platform team. We need a skilled engineer who will help us develop and maintain data architectures and processing solutions that power our data scientists, BI analytics and many other internal data consumers. One major role of the team is to prepare and maintain self-serve...

  • Senior Data Engineer

    3 tygodni temu

    Warsaw, Polska Algoteque Pełny etat

    ALGOTEQUE is an IT consultancy firm that helps startups, mid-sized and large corporations to create and deliver innovative technologies.Our team has a successful track record in designing, developing, implementing, and integrating software solutions (AI, ML, BI, Web, Automation) for Telecom, Energy, Bank, Insurance, Pharma, Automotive, Industry, e-commerce....

  • Data Engineer

    3 tygodni temu

    Warsaw, Polska Digital Turbine Pełny etat

    Data Engineer Warsaw, PolandEMEA – AGP Tech /Regular /HybridAt Digital Turbine , we make mobile advertising experiences more meaningful and rewarding for users, app publishers, and advertisers — intelligently connecting people in more ways, across more devices. We provide app publishers and advertisers with powerful ads and experiences that...

  • Senior Data Engineer

    2 tygodni temu

    Warsaw, Polska Displate Pełny etat

    Imagine a group of fascinating people… who spend time at work creating the product they absolutely love. Imagine empowered tech freaks for whom the team equals comrades. Imagine tremendous professionals and geeks who like, support, and respect each other. See yourself in that picture? Well, hello there! We’re Displate, the XXI-century metal posters...

  • Senior Data Engineer

    3 dni temu

    Warsaw, Polska AI Clearing Pełny etat

    We are a mature Warsaw-based startup with global ambitions, that applies AI to supervise huge construction sites. Our SaaS platform is universally acclaimed by the biggest companies in the construction industry like Skanska, Eurovia, PCL, Neom, and many more, and we already monitor dozens of mega construction sites across 6 continents. Now, we are entering...

  • Senior Data Engineer

    2 tygodni temu

    Warsaw, Polska emagine Consulting Pełny etat

    Industry: Banking Work model: hybrid in Warsaw office (2 days a week working from home) Assignment Type: B2B Project Length: 12 months (extendable) Start Date: ASAP Project Language: English About a role: As a Senior Data Engineer, you will join a team that is part of the data platforms and services department. The domain is responsible...

  • Senior Data Engineer

    1 miesiąc temu

    Warsaw, Polska Alter Solutions Pełny etat

    Job DescriptionThe position of a Senior Data Engineer offers an opportunity to work with a team of senior developers who focus on regulatory reporting. The team deals in the first place with development but also with maintenance and support of Data Warehousing/ ETL solutions.QualificationsMinimum 4 years (senior) of experience working with ETL, data...

  • Senior Data Engineer

    1 miesiąc temu

    Warsaw, Polska ALTER SOLUTIONS Pełny etat

    Job Description The position of a Senior Data Engineer offers an opportunity to work with a team of senior developers who focus on regulatory reporting. The team deals in the first place with development but also with maintenance and support of Data Warehousing/ ETL solutions. Qualifications Minimum 4 years (senior) of experience working with ...

  • Senior Data Engineer

    2 tygodni temu

    Warsaw, Polska SwingDev Pełny etat

    We are a Polish branch of a fast-growing InsurTech product company from Silicon Valley - Hippo Insurance. Our mission is to revolutionize home insurance in the US, starting from IoT monitoring devices to our industry-leading software. How do we want to get there? We need a top-notch talent, just like you! Putting a lot of effort into hiring top-tier...