Data Engineer for Cloud-Based Data Warehouse
1 miesiąc temu
We are seeking a highly skilled and experienced DataOps Engineer to join our team working in a global container liner shipping company. Our team specializes in building a Data Lake and leveraging the Databricks platform.
About the RoleThis exciting opportunity allows you to lead the development and optimization of data orchestration platforms, leveraging Astronomer and Airflow. You will enable CI/CD workflows for a cutting-edge, cloud-based Data Warehouse built on Databricks and AWS.
Responsibilities- Developing and maintaining a scalable Astronomer-based scheduling environment for our Data Lakehouse platform.
- Designing and implementing CI/CD pipelines to automate deployments, validate data models, and monitor workflows.
- Configuring and managing Astronomer workspaces, deployments, and access control.
- Integrating dbt Core with Cosmos for orchestrating dbt models in Airflow.
- Building and optimizing Cosmos Airflow DAGs for data ingestion and transformation.
- Documenting and implementing deployment processes, including ephemeral environments for development.
- Collaborating with stakeholders to ensure best practices in pipeline design and Airflow job scheduling.
- Leveraging AWS services (e.g., S3) and Databricks for performance, scalability, and security.
- Troubleshooting and supporting data scheduling, CI/CD processes, and model deployment pipelines.
- Ensuring comprehensive documentation of workflows, CI/CD processes, and operational standards.
- Finalizing the migration of the existing Data Lake scheduling from Airflow MWAA to Astronomer.
- At least 5+ years of experience as a Data Engineer or DataOps Engineer, focusing on CI/CD pipelines and workflow orchestration.
- Expertise in configuring and deploying Astronomer or Apache Airflow in cloud environments.
- Hands-on experience with Databricks, dbt Core, and AWS services.
- Proficiency in Python and Spark for data transformation and pipeline development.
- Strong SQL skills and knowledge of relational databases.
- Familiarity with GitFlow and CI/CD tools like GitLab or Jenkins.
- Knowledge of dimensional modeling, data access patterns, and data storage optimization.
- Experience with Terraform for Infrastructure as Code and Airflow DAGs with Cosmos.
- Excellent collaboration and communication skills, with a proactive approach to sharing knowledge.
- Proficiency in English at the C1 level.
- A competitive salary ranging from 26,880 to 31,920 PLN (160 - 190 PLN/h) + VAT.
- A clear wage model where our consultants know exactly what's our fee for our agent role on the project.
- Professional development support, including trainings, technical certificates, conference participation, and foreign language classes.
- An opportunity to join our Career Development Program, offering advisory support in career planning.
- The possibility of 100% remote work.
-
High-End Data Engineer for Cloud-Based Data Warehouse
3 tygodni temu
Bernarda Chrzanowskiego , Gdańsk, Polska 7N Pełny etat**Job Overview:**We are seeking a highly skilled and experienced DataOps Engineer to join our team working in a global container liner shipping company. Our team specializes in building a Data Lake and leveraging the Databricks platform.This role is an exciting opportunity to lead the development and optimization of data orchestration platforms, leveraging...
-
Cloud Data Architect
2 tygodni temu
Gdańsk, Polska DCG Pełny etatAs a leading recruitment company, DCG thrives on partnering with businesses to empower their growth through experienced professionals. Our approach is centered around meeting the unique needs of each client and providing continuous support. Given our rapid expansion and numerous recruitment projects for partners, we are currently seeking an exceptional Cloud...
-
Data Warehouse Automation Specialist
2 tygodni temu
Gdańsk, Polska emagine Polska Pełny etatIndustry: BankingLocation: GdańskWork model: hybrid (1 day per week is a must)Estimated salary: 165 PLN/h netto + VATType of assignment: B2BProject length: long-termLanguage: English & PolishWe are seeking an experienced Data Warehouse Automation Specialist to join our team at emagine Polska. The successful candidate will be responsible for designing and...
-
Cloud Data Engineer for Global Business Partnerships
2 tygodni temu
Gdańsk, Polska RITS Professional Services Pełny etatRITS Professional Services is seeking an experienced Cloud Data Engineer to join our team and work closely with our business partners from the USA.Job Description:The successful candidate will be responsible for designing, building, and maintaining large-scale data platforms using public cloud infrastructure (AWS and GCP), Kafka, databases, and containers....
-
Data Engineer Specialist
2 tygodni temu
Gdańsk, Polska TechTorch Pełny etatAbout TechTorch">TechTorch is a pioneer in digital transformation, leveraging AI-powered platforms and global expertise to drive business benefits for private equity sector companies.Job Overview">We are seeking an experienced Data Engineer Specialist to join our growing team. The ideal candidate will have a strong background in data engineering,...
-
Data Engineer
1 miesiąc temu
Gdańsk, Trójmiasto, Polska HAYS Poland Pełny etatHays IT Contracting is a cooperation based on B2B principles. Our company matches IT specialists to the most interesting technological projects on the market. Join the group of 500 satisfied Contractors and work for Hays Clients!We are currently looking for candidates for the position:Data Engineer - Snowflake SpecialistWork model: Hybrid, at least 1 day...
-
Gdańsk, Pomerania, Polska Grid Dynamics Poland Pełny etatWe are seeking a skilled Senior Machine Learning Engineer to join the Consumer and Commercial Data Engineering & ML Engineering team at Grid Dynamics Poland.About the Role:This is an exciting opportunity to execute data infrastructure requests from Data Science teams focused on customer and store modeling, leveraging expertise in data engineering and/or ML...
-
Data Engineer
5 miesięcy temu
Gdańsk, Polska Epam Pełny etatDescription We are seeking a Data Engineer who would like to join EPAM Data Competency Center and grow professionally by participating in a project for a technological giant. Our work involves close collaboration with clients, understanding business needs, providing top-tier engineering experiences, and applying best practices to consistently create...
-
Cloud Data Engineer
4 tygodni temu
Gdańsk, Trójmiasto, Polska Antal Sp. z o.o. Pełny etatNasz Klient to globalny lider w transformacji i zarządzaniu biznesem, który wykorzystuje moc technologii do budowania zrównoważonej i inkluzywnej przyszłości. Poszukujemy doświadczonego Cloud Data Engineer, który dołączy do naszego zespołu w lokalizacjach: Wrocław, Poznań, Gdańsk, Kraków lub Warszawa.Zadania:Projektowanie, rozwijanie i...
-
Data Quality Assurance Engineer for ETL DW Environment
2 tygodni temu
Gdańsk, Polska emagine Polska Pełny etatWe are seeking a highly skilled Data Quality Assurance Engineer to join our team in Gdańsk, Poland. As a Senior Test Automation Engineer with expertise in ETL/DW environments, you will play a crucial role in ensuring the quality and reliability of our data processing systems.Industry: BankingRate: up to 165 PLN/h netto + VATThe successful candidate will...
-
Data Engineer
7 miesięcy temu
Gdańsk, Polska Lyreco Group Pełny etatResponsible for building systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. The data engineer aims to make data accessible so that the organization can use it to evaluate and optimize its performance Are you excited for your new career adventure? At Lyreco, we offer more than...
-
Senior Data Engineer with GCP
3 miesięcy temu
Gdańsk, Polska Endava Pełny etatJob DescriptionOur data team has expertise across engineering, analysis, architecture, modeling, machine learning, artificial intelligence, and data science. This discipline is responsible for transforming raw data into actionable insights, building robust data infrastructures, and enabling data-driven decision-making and innovation through advanced...
-
Big Data Systems Architect
4 dni temu
Gdańsk, Polska TechTorch Pełny etatAbout Us">TechTorch is a pioneer in digital transformation projects, utilizing an AI-powered platform and a global team of experts to accelerate business benefits for private equity sector companies.As a leading service company, we're seeking an experienced Data Engineer to join our growing team. This role requires strong background knowledge in data...
-
Data Integration Consultant
1 tydzień temu
Gdańsk, Polska DACl Pełny etatAt DAC.digital, we are constantly expanding our business and seeking a skilled Data Integration Consultant to join our team in Gdańsk.We offer a competitive salary of 19,000 PLN net/month (125-140 PLN net/hour) for this position.Job DescriptionThis is an exciting opportunity to support our team in various aspects of data integration, including:Data...
-
Data Engineer for Travel Innovation
1 miesiąc temu
Gdańsk, Pomerania, Polska VIATOR Pełny etatUnlock the Future of Travel with ViatorWe are seeking a skilled Data Engineer to join our team at Viator, a leading marketplace for travel experiences. As a key member of our MarTech Engineering & Operations team, you will play a crucial role in developing and maintaining the tooling used to drive millions of visitors to viator.com every month.The ideal...
-
Senior Cloud Data Architect with GCP Expertise
2 tygodni temu
Gdańsk, Pomerania, Polska Holisticon Insight Pełny etatUnlock the Power of Big Data with Holisticon InsightHolisticon Insight, a leading provider of IoT, AI, and advanced analytics solutions, is seeking a highly skilled Senior Cloud Data Architect to join their data product team. This exciting opportunity will enable you to leverage your expertise in Google Cloud Platform (GCP) to drive business growth and...
-
Cloud Operations Engineer
4 dni temu
Gdańsk, Trójmiasto, Polska Arla Pełny etatWe are looking for a passionate Cloud Operations Engineer to enhance our cutting-edge Data Foundation. If you have expertise in cloud technologies, especially within Microsoft Azure, and thrive in a progressive work environment, this opportunity is for you. About the jobAs a Cloud Operations Engineer you will be a part of our support engineering team in...
-
Middle Data Engineer
7 miesięcy temu
Gdańsk, Polska Ciklum Pełny etatDescription Ciklum is looking for a Middle Data Engineer to join our team full-time in Poland. We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and...
-
Data engineer
6 miesięcy temu
Gdańsk, Polska b2bnetwork Pełny etatDetailed description of work task to be carried out We are seeking a skilled and motivated Data Engineer with extensive experience in Hadoop and HDFS as well as with deep understanding of relational databases to our team. In this role you will be responsible for designing, building and maintaining our data infrastructure to ensure efficient, reliable and...
-
Strategic Data Architect
4 dni temu
Gdańsk, Polska Cornerstone OnDemand Pełny etatWe are seeking a seasoned Principal BI Engineer to drive data-driven decision-making across our organization. This role requires strong expertise in data analysis, visualization, and warehousing.The ideal candidate will have 10+ years of experience in data analysis using complex SQL & Tableau, with a focus on UI/UX of reports and dashboards. They should also...