High-End Data Engineer for Cloud-Based Data Warehouse

5 dni temu


Bernarda Chrzanowskiego Gdańsk, Polska 7N Pełny etat

**Job Overview:**

We are seeking a highly skilled and experienced DataOps Engineer to join our team working in a global container liner shipping company. Our team specializes in building a Data Lake and leveraging the Databricks platform.

This role is an exciting opportunity to lead the development and optimization of data orchestration platforms, leveraging Astronomer and Airflow, and enabling CI/CD workflows for a cutting-edge, cloud-based Data Warehouse built on Databricks and AWS.

**About the Role:**

The ideal candidate will have at least 5+ years of experience as a Data Engineer or DataOps Engineer, focusing on CI/CD pipelines and workflow orchestration. They should have expertise in configuring and deploying Astronomer or Apache Airflow in cloud environments.

Additionally, they should have hands-on experience with Databricks, dbt Core, and AWS services, as well as proficiency in Python and Spark for data transformation and pipeline development.

**Responsibilities:**

  • Developing and maintaining a scalable Astronomer-based scheduling environment for our Data Lakehouse platform
  • Designing and implementing CI/CD pipelines to automate deployments, validate data models, and monitor workflows
  • Configuring and managing Astronomer workspaces, deployments, and access control
  • Integrating dbt Core with Cosmos for orchestrating dbt models in Airflow
  • Building and optimizing Cosmos Airflow DAGs for data ingestion and transformation
  • Documenting and implementing deployment processes, including ephemeral environments for development
  • Collaborating with stakeholders to ensure best practices in pipeline design and Airflow job scheduling
  • Leveraging AWS services (e.g., S3) and Databricks for performance, scalability, and security
  • Troubleshooting and supporting data scheduling, CI/CD processes, and model deployment pipelines
  • Ensuring comprehensive documentation of workflows, CI/CD processes, and operational standards
  • Finalizing the migration of the existing Data Lake scheduling from Airflow MWAA to Astronomer

**Requirements:**

  • At least 5+ years of experience as a Data Engineer or DataOps Engineer, focusing on CI/CD pipelines and workflow orchestration
  • Expertise in configuring and deploying Astronomer or Apache Airflow in cloud environments
  • Hands-on experience with Databricks, dbt Core, and AWS services
  • Proficiency in Python and Spark for data transformation and pipeline development
  • Strong SQL skills and knowledge of relational databases
  • Familiarity with GitFlow and CI/CD tools like GitLab or Jenkins
  • Knowledge of dimensional modeling, data access patterns, and data storage optimization
  • Experience with Terraform for Infrastructure as Code and Airflow DAGs with Cosmos
  • Excellent collaboration and communication skills, with a proactive approach to sharing knowledge
  • Proficiency in English at the C1 level

**What We Offer:**

  • Financial transparency: the salary is 26,880 - 31,920 PLN (160 - 190 PLN/h) + VAT
  • Clear wage model. 7N Consultants know exactly what's 7N fee for its agent role on the project.
  • Professional development support. We subsidize trainings, technical certificates, conference participation, and foreign language classes.
  • 7N Inspiration Team's support and an opportunity to join their ranks.
  • Cooperation with real experts – the average professional experience of a 7N consultant is 10 years.
  • Comprehensive and personal project support from one of a 7N Agent.
  • Numerous (10-12 each year) high-end events – both online and offline.
  • Healthcare, Benefit Multisport and life insurance subsidies regardless of cooperation model.
  • Access to Mindgram, a psychological support and professional development platform.
  • Professional and quality-oriented recruitment process led by the most experienced recruiters in the industry.


  • Bernarda Chrzanowskiego , Gdańsk, Polska 7N Pełny etat

    Cloud-Based Data Warehouse Engineer OpportunityWe are seeking a highly skilled and experienced DataOps Engineer to join our team working in a global container liner shipping company. Our team specializes in building a Data Lake and leveraging the Databricks platform.About the RoleThis exciting opportunity allows you to lead the development and optimization...


  • Gdańsk, Pomerania, Polska VIATOR Pełny etat

    About Viator: Revolutionizing TravelViator, a leading travel experiences marketplace and part of the Tripadvisor family, is shaping the future of travel. With over 300,000+ experiences to explore, we empower travelers to create unforgettable memories.Competitive SalaryWe offer a competitive compensation package, including a base salary of $120,000 - $180,000...


  • Gdańsk, Pomerania, Polska VIATOR Pełny etat

    Unlock the Future of Travel with ViatorWe are seeking a skilled Data Engineer to join our team at Viator, a leading marketplace for travel experiences. As a key member of our MarTech Engineering & Operations team, you will play a crucial role in developing and maintaining the tooling used to drive millions of visitors to viator.com every month.The ideal...


  • Gdańsk, Pomerania, Polska CodiLime Pełny etat

    As a Data Engineer in the Networking Domain at CodiLime, you will be responsible for designing and implementing ETL systems in cloud environments.The ideal candidate has hands-on experience with BigQuery and Cloud Functions, as well as expertise in network telemetry domains. You should also have excellent knowledge of networks, security, and platforms.We...


  • Gdańsk, Polska Devopsbay Pełny etat

    About DevopsbayWe are a digital solutions company helping organizations around the world achieve new thresholds of business performance. Our clients come from diverse industries including banking and financial services, hi-tech and manufacturing, insurance, consumer services, and more.Our teams specialize in cutting-edge big data solutions, leveraging...


  • Gdańsk, Polska Endava Pełny etat

    Job DescriptionOur data team has expertise across engineering, analysis, architecture, modeling, machine learning, artificial intelligence, and data science. This discipline is responsible for transforming raw data into actionable insights, building robust data infrastructures, and enabling data-driven decision-making and innovation through advanced...

  • Data Engineer

    3 tygodni temu


    Gdańsk, Trójmiasto, Polska HAYS Poland Pełny etat

    Hays IT Contracting is a cooperation based on B2B principles. Our company matches IT specialists to the most interesting technological projects on the market. Join the group of 500 satisfied Contractors and work for Hays Clients!We are currently looking for candidates for the position:Data Engineer - Snowflake SpecialistWork model: Hybrid, at least 1 day...

  • Middle Data Engineer

    6 miesięcy temu


    Gdańsk, Polska Ciklum Pełny etat

    Description Ciklum is looking for a Middle Data Engineer to join our team full-time in Poland. We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and...

  • Data Engineer

    5 miesięcy temu


    Gdańsk, Polska Epam Pełny etat

    Description We are seeking a Data Engineer who would like to join EPAM Data Competency Center and grow professionally by participating in a project for a technological giant. Our work involves close collaboration with clients, understanding business needs, providing top-tier engineering experiences, and applying best practices to consistently create...


  • Lęborska, Gdańsk, Polska Devopsbay Pełny etat

    Job DescriptionAs a Senior Cloud Data Solutions Expert at Devopsbay, you will play a crucial role in designing and deploying cutting-edge big data solutions leveraging the Databricks platform.About the Role:10+ years of experience as a Data Architect with a strong background in cloud-based data solutions, especially with Azure and Databricks.Hands-on...


  • Gdańsk, Pomerania, Polska State Street Pełny etat

    Company OverviewState Street is a leading global financial services provider with a rich history of innovation and excellence. With a presence in over 100 markets worldwide, we offer a wide range of investment servicing, data & analytics, investment research & trading, and investment management solutions to institutional clients.We are committed to helping...


  • Gdańsk, Pomerania, Polska Grid Dynamics Poland Pełny etat

    We are seeking a skilled Senior Machine Learning Engineer to join the Consumer and Commercial Data Engineering & ML Engineering team at Grid Dynamics Poland.About the Role:This is an exciting opportunity to execute data infrastructure requests from Data Science teams focused on customer and store modeling, leveraging expertise in data engineering and/or ML...

  • Cloud Data Engineer

    1 tydzień temu


    Gdańsk, Trójmiasto, Polska Antal Sp. z o.o. Pełny etat

    Nasz Klient to globalny lider w transformacji i zarządzaniu biznesem, który wykorzystuje moc technologii do budowania zrównoważonej i inkluzywnej przyszłości. Poszukujemy doświadczonego Cloud Data Engineer, który dołączy do naszego zespołu w lokalizacjach: Wrocław, Poznań, Gdańsk, Kraków lub Warszawa.Zadania:Projektowanie, rozwijanie i...


  • Gdańsk, Pomerania, Polska Nordea Bank Pełny etat

    We are Nordea Bank, a 200-year-old institution supporting and growing the Nordic economies. Our values are deeply rooted in open, progressive, and collaborative societies.Our vision is to enable dreams and aspirations for a greater good by building relationships, learning, and leading change.About Our TeamWe are the Data Tooling team at Nordea, responsible...

  • Data Engineer

    7 miesięcy temu


    Gdańsk, Polska Lyreco Group Pełny etat

    Responsible for building systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. The data engineer aims to make data accessible so that the organization can use it to evaluate and optimize its performance Are you excited for your new career adventure? At Lyreco, we offer more than...


  • Gdańsk, Pomerania, Polska Emagine Sp. Z O.o. Pełny etat

    Emagine Sp. Z O.o. Seeks Experienced Senior Automation QAWe are looking for a skilled Senior Automation QA to work on a long-term project in the data/ETL/DW area with expertise in SQL and Python required. The ideal candidate will have at least 8-10 years of experience in Quality Assurance Engineer & testing process, and proficiency in ETL/Data Warehouse/Data...

  • Data Engineer Lead

    1 miesiąc temu


    Gdańsk, Pomerania, Polska Arla foods Pełny etat

    About the roleArla Foods, a leading global dairy company, is seeking an experienced Senior Data Engineer to join our Commercial IT organization. As a key member of our team, you will play a critical role in supporting the continuous rollout and optimization of our global Data Foundation footprint in Microsoft Azure and other platforms.The ideal candidate...

  • Senior Data Modeler

    7 miesięcy temu


    Gdańsk, Polska Dynatrace Pełny etat

    Job DescriptionYou will take active part in identifying ways to make our platform better, working with various data, and transforming that data into solid solutions that make business decisions easier. You will help to provide value by enabling better self-serve for our customers and improving their time to reliable insights. We guarantee plenty of...


  • Gdańsk, Pomerania, Polska State Street Pełny etat

    About the RoleWe are seeking a highly skilled Data Integration Senior Developer to join our team at State Street. As a Data Integration Senior Developer, you will be responsible for designing and developing cutting-edge data integration solutions for our clients.Responsibilities:Design and develop ETL data pipelines using Talend Cloud Data Integration...


  • Bernarda Chrzanowskiego , Gdańsk, Polska 7N Pełny etat

    About 7N ConsultantsWe are looking for a skilled Backend Software Engineer to join our dynamic team developing streaming platforms using Python and Node.js. The successful candidate will have experience in designing, developing, and maintaining scalable backend applications on AWS.Key Responsibilities:Design and implement efficient, reusable code using...