Data Engineer for Cloud-Based Data Warehouse

1 miesiąc temu


Bernarda Chrzanowskiego Gdańsk, Polska 7N Pełny etat
Cloud-Based Data Warehouse Engineer Opportunity

We are seeking a highly skilled and experienced DataOps Engineer to join our team working in a global container liner shipping company. Our team specializes in building a Data Lake and leveraging the Databricks platform.

About the Role

This exciting opportunity allows you to lead the development and optimization of data orchestration platforms, leveraging Astronomer and Airflow. You will enable CI/CD workflows for a cutting-edge, cloud-based Data Warehouse built on Databricks and AWS.

Responsibilities
  • Developing and maintaining a scalable Astronomer-based scheduling environment for our Data Lakehouse platform.
  • Designing and implementing CI/CD pipelines to automate deployments, validate data models, and monitor workflows.
  • Configuring and managing Astronomer workspaces, deployments, and access control.
  • Integrating dbt Core with Cosmos for orchestrating dbt models in Airflow.
  • Building and optimizing Cosmos Airflow DAGs for data ingestion and transformation.
  • Documenting and implementing deployment processes, including ephemeral environments for development.
  • Collaborating with stakeholders to ensure best practices in pipeline design and Airflow job scheduling.
  • Leveraging AWS services (e.g., S3) and Databricks for performance, scalability, and security.
  • Troubleshooting and supporting data scheduling, CI/CD processes, and model deployment pipelines.
  • Ensuring comprehensive documentation of workflows, CI/CD processes, and operational standards.
  • Finalizing the migration of the existing Data Lake scheduling from Airflow MWAA to Astronomer.
Requirements
  • At least 5+ years of experience as a Data Engineer or DataOps Engineer, focusing on CI/CD pipelines and workflow orchestration.
  • Expertise in configuring and deploying Astronomer or Apache Airflow in cloud environments.
  • Hands-on experience with Databricks, dbt Core, and AWS services.
  • Proficiency in Python and Spark for data transformation and pipeline development.
  • Strong SQL skills and knowledge of relational databases.
  • Familiarity with GitFlow and CI/CD tools like GitLab or Jenkins.
  • Knowledge of dimensional modeling, data access patterns, and data storage optimization.
  • Experience with Terraform for Infrastructure as Code and Airflow DAGs with Cosmos.
  • Excellent collaboration and communication skills, with a proactive approach to sharing knowledge.
  • Proficiency in English at the C1 level.
What We Offer
  • A competitive salary ranging from 26,880 to 31,920 PLN (160 - 190 PLN/h) + VAT.
  • A clear wage model where our consultants know exactly what's our fee for our agent role on the project.
  • Professional development support, including trainings, technical certificates, conference participation, and foreign language classes.
  • An opportunity to join our Career Development Program, offering advisory support in career planning.
  • The possibility of 100% remote work.


  • Bernarda Chrzanowskiego , Gdańsk, Polska 7N Pełny etat

    **Job Overview:**We are seeking a highly skilled and experienced DataOps Engineer to join our team working in a global container liner shipping company. Our team specializes in building a Data Lake and leveraging the Databricks platform.This role is an exciting opportunity to lead the development and optimization of data orchestration platforms, leveraging...

  • Cloud Data Architect

    2 tygodni temu


    Gdańsk, Polska DCG Pełny etat

    As a leading recruitment company, DCG thrives on partnering with businesses to empower their growth through experienced professionals. Our approach is centered around meeting the unique needs of each client and providing continuous support. Given our rapid expansion and numerous recruitment projects for partners, we are currently seeking an exceptional Cloud...


  • Gdańsk, Polska emagine Polska Pełny etat

    Industry: BankingLocation: GdańskWork model: hybrid (1 day per week is a must)Estimated salary: 165 PLN/h netto + VATType of assignment: B2BProject length: long-termLanguage: English & PolishWe are seeking an experienced Data Warehouse Automation Specialist to join our team at emagine Polska. The successful candidate will be responsible for designing and...


  • Gdańsk, Polska RITS Professional Services Pełny etat

    RITS Professional Services is seeking an experienced Cloud Data Engineer to join our team and work closely with our business partners from the USA.Job Description:The successful candidate will be responsible for designing, building, and maintaining large-scale data platforms using public cloud infrastructure (AWS and GCP), Kafka, databases, and containers....

  • Data Engineer Specialist

    2 tygodni temu


    Gdańsk, Polska TechTorch Pełny etat

    About TechTorch">TechTorch is a pioneer in digital transformation, leveraging AI-powered platforms and global expertise to drive business benefits for private equity sector companies.Job Overview">We are seeking an experienced Data Engineer Specialist to join our growing team. The ideal candidate will have a strong background in data engineering,...

  • Data Engineer

    1 miesiąc temu


    Gdańsk, Trójmiasto, Polska HAYS Poland Pełny etat

    Hays IT Contracting is a cooperation based on B2B principles. Our company matches IT specialists to the most interesting technological projects on the market. Join the group of 500 satisfied Contractors and work for Hays Clients!We are currently looking for candidates for the position:Data Engineer - Snowflake SpecialistWork model: Hybrid, at least 1 day...


  • Gdańsk, Pomerania, Polska Grid Dynamics Poland Pełny etat

    We are seeking a skilled Senior Machine Learning Engineer to join the Consumer and Commercial Data Engineering & ML Engineering team at Grid Dynamics Poland.About the Role:This is an exciting opportunity to execute data infrastructure requests from Data Science teams focused on customer and store modeling, leveraging expertise in data engineering and/or ML...

  • Data Engineer

    5 miesięcy temu


    Gdańsk, Polska Epam Pełny etat

    Description We are seeking a Data Engineer who would like to join EPAM Data Competency Center and grow professionally by participating in a project for a technological giant. Our work involves close collaboration with clients, understanding business needs, providing top-tier engineering experiences, and applying best practices to consistently create...

  • Cloud Data Engineer

    4 tygodni temu


    Gdańsk, Trójmiasto, Polska Antal Sp. z o.o. Pełny etat

    Nasz Klient to globalny lider w transformacji i zarządzaniu biznesem, który wykorzystuje moc technologii do budowania zrównoważonej i inkluzywnej przyszłości. Poszukujemy doświadczonego Cloud Data Engineer, który dołączy do naszego zespołu w lokalizacjach: Wrocław, Poznań, Gdańsk, Kraków lub Warszawa.Zadania:Projektowanie, rozwijanie i...


  • Gdańsk, Polska emagine Polska Pełny etat

    We are seeking a highly skilled Data Quality Assurance Engineer to join our team in Gdańsk, Poland. As a Senior Test Automation Engineer with expertise in ETL/DW environments, you will play a crucial role in ensuring the quality and reliability of our data processing systems.Industry: BankingRate: up to 165 PLN/h netto + VATThe successful candidate will...

  • Data Engineer

    7 miesięcy temu


    Gdańsk, Polska Lyreco Group Pełny etat

    Responsible for building systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. The data engineer aims to make data accessible so that the organization can use it to evaluate and optimize its performance Are you excited for your new career adventure? At Lyreco, we offer more than...


  • Gdańsk, Polska Endava Pełny etat

    Job DescriptionOur data team has expertise across engineering, analysis, architecture, modeling, machine learning, artificial intelligence, and data science. This discipline is responsible for transforming raw data into actionable insights, building robust data infrastructures, and enabling data-driven decision-making and innovation through advanced...


  • Gdańsk, Polska TechTorch Pełny etat

    About Us">TechTorch is a pioneer in digital transformation projects, utilizing an AI-powered platform and a global team of experts to accelerate business benefits for private equity sector companies.As a leading service company, we're seeking an experienced Data Engineer to join our growing team. This role requires strong background knowledge in data...

  • Data Integration Consultant

    1 tydzień temu


    Gdańsk, Polska DACl Pełny etat

    At DAC.digital, we are constantly expanding our business and seeking a skilled Data Integration Consultant to join our team in Gdańsk.We offer a competitive salary of 19,000 PLN net/month (125-140 PLN net/hour) for this position.Job DescriptionThis is an exciting opportunity to support our team in various aspects of data integration, including:Data...


  • Gdańsk, Pomerania, Polska VIATOR Pełny etat

    Unlock the Future of Travel with ViatorWe are seeking a skilled Data Engineer to join our team at Viator, a leading marketplace for travel experiences. As a key member of our MarTech Engineering & Operations team, you will play a crucial role in developing and maintaining the tooling used to drive millions of visitors to viator.com every month.The ideal...


  • Gdańsk, Pomerania, Polska Holisticon Insight Pełny etat

    Unlock the Power of Big Data with Holisticon InsightHolisticon Insight, a leading provider of IoT, AI, and advanced analytics solutions, is seeking a highly skilled Senior Cloud Data Architect to join their data product team. This exciting opportunity will enable you to leverage your expertise in Google Cloud Platform (GCP) to drive business growth and...


  • Gdańsk, Trójmiasto, Polska Arla Pełny etat

    We are looking for a passionate Cloud Operations Engineer to enhance our cutting-edge Data Foundation. If you have expertise in cloud technologies, especially within Microsoft Azure, and thrive in a progressive work environment, this opportunity is for you. About the jobAs a Cloud Operations Engineer you will be a part of our support engineering team in...

  • Middle Data Engineer

    7 miesięcy temu


    Gdańsk, Polska Ciklum Pełny etat

    Description Ciklum is looking for a Middle Data Engineer to join our team full-time in Poland. We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and...

  • Data engineer

    6 miesięcy temu


    Gdańsk, Polska b2bnetwork Pełny etat

    Detailed description of work task to be carried out We are seeking a skilled and motivated Data Engineer with extensive experience in Hadoop and HDFS as well as with deep understanding of relational databases to our team. In this role you will be responsible for designing, building and maintaining our data infrastructure to ensure efficient, reliable and...


  • Gdańsk, Polska Cornerstone OnDemand Pełny etat

    We are seeking a seasoned Principal BI Engineer to drive data-driven decision-making across our organization. This role requires strong expertise in data analysis, visualization, and warehousing.The ideal candidate will have 10+ years of experience in data analysis using complex SQL & Tableau, with a focus on UI/UX of reports and dashboards. They should also...