High-End Data Engineer for Cloud-Based Data Warehouse
5 dni temu
**Job Overview:**
We are seeking a highly skilled and experienced DataOps Engineer to join our team working in a global container liner shipping company. Our team specializes in building a Data Lake and leveraging the Databricks platform.
This role is an exciting opportunity to lead the development and optimization of data orchestration platforms, leveraging Astronomer and Airflow, and enabling CI/CD workflows for a cutting-edge, cloud-based Data Warehouse built on Databricks and AWS.
**About the Role:**
The ideal candidate will have at least 5+ years of experience as a Data Engineer or DataOps Engineer, focusing on CI/CD pipelines and workflow orchestration. They should have expertise in configuring and deploying Astronomer or Apache Airflow in cloud environments.
Additionally, they should have hands-on experience with Databricks, dbt Core, and AWS services, as well as proficiency in Python and Spark for data transformation and pipeline development.
**Responsibilities:**
- Developing and maintaining a scalable Astronomer-based scheduling environment for our Data Lakehouse platform
- Designing and implementing CI/CD pipelines to automate deployments, validate data models, and monitor workflows
- Configuring and managing Astronomer workspaces, deployments, and access control
- Integrating dbt Core with Cosmos for orchestrating dbt models in Airflow
- Building and optimizing Cosmos Airflow DAGs for data ingestion and transformation
- Documenting and implementing deployment processes, including ephemeral environments for development
- Collaborating with stakeholders to ensure best practices in pipeline design and Airflow job scheduling
- Leveraging AWS services (e.g., S3) and Databricks for performance, scalability, and security
- Troubleshooting and supporting data scheduling, CI/CD processes, and model deployment pipelines
- Ensuring comprehensive documentation of workflows, CI/CD processes, and operational standards
- Finalizing the migration of the existing Data Lake scheduling from Airflow MWAA to Astronomer
**Requirements:**
- At least 5+ years of experience as a Data Engineer or DataOps Engineer, focusing on CI/CD pipelines and workflow orchestration
- Expertise in configuring and deploying Astronomer or Apache Airflow in cloud environments
- Hands-on experience with Databricks, dbt Core, and AWS services
- Proficiency in Python and Spark for data transformation and pipeline development
- Strong SQL skills and knowledge of relational databases
- Familiarity with GitFlow and CI/CD tools like GitLab or Jenkins
- Knowledge of dimensional modeling, data access patterns, and data storage optimization
- Experience with Terraform for Infrastructure as Code and Airflow DAGs with Cosmos
- Excellent collaboration and communication skills, with a proactive approach to sharing knowledge
- Proficiency in English at the C1 level
**What We Offer:**
- Financial transparency: the salary is 26,880 - 31,920 PLN (160 - 190 PLN/h) + VAT
- Clear wage model. 7N Consultants know exactly what's 7N fee for its agent role on the project.
- Professional development support. We subsidize trainings, technical certificates, conference participation, and foreign language classes.
- 7N Inspiration Team's support and an opportunity to join their ranks.
- Cooperation with real experts – the average professional experience of a 7N consultant is 10 years.
- Comprehensive and personal project support from one of a 7N Agent.
- Numerous (10-12 each year) high-end events – both online and offline.
- Healthcare, Benefit Multisport and life insurance subsidies regardless of cooperation model.
- Access to Mindgram, a psychological support and professional development platform.
- Professional and quality-oriented recruitment process led by the most experienced recruiters in the industry.
-
Data Engineer for Cloud-Based Data Warehouse
3 tygodni temu
Bernarda Chrzanowskiego , Gdańsk, Polska 7N Pełny etatCloud-Based Data Warehouse Engineer OpportunityWe are seeking a highly skilled and experienced DataOps Engineer to join our team working in a global container liner shipping company. Our team specializes in building a Data Lake and leveraging the Databricks platform.About the RoleThis exciting opportunity allows you to lead the development and optimization...
-
Data Engineer II Specialist for Enterprise Data Pipelines
3 tygodni temu
Gdańsk, Pomerania, Polska VIATOR Pełny etatAbout Viator: Revolutionizing TravelViator, a leading travel experiences marketplace and part of the Tripadvisor family, is shaping the future of travel. With over 300,000+ experiences to explore, we empower travelers to create unforgettable memories.Competitive SalaryWe offer a competitive compensation package, including a base salary of $120,000 - $180,000...
-
Data Engineer for Travel Innovation
2 tygodni temu
Gdańsk, Pomerania, Polska VIATOR Pełny etatUnlock the Future of Travel with ViatorWe are seeking a skilled Data Engineer to join our team at Viator, a leading marketplace for travel experiences. As a key member of our MarTech Engineering & Operations team, you will play a crucial role in developing and maintaining the tooling used to drive millions of visitors to viator.com every month.The ideal...
-
Data Engineer for Cloud Infrastructure
1 miesiąc temu
Gdańsk, Pomerania, Polska CodiLime Pełny etatAs a Data Engineer in the Networking Domain at CodiLime, you will be responsible for designing and implementing ETL systems in cloud environments.The ideal candidate has hands-on experience with BigQuery and Cloud Functions, as well as expertise in network telemetry domains. You should also have excellent knowledge of networks, security, and platforms.We...
-
Senior Cloud Data Solutions Engineer
4 tygodni temu
Gdańsk, Polska Devopsbay Pełny etatAbout DevopsbayWe are a digital solutions company helping organizations around the world achieve new thresholds of business performance. Our clients come from diverse industries including banking and financial services, hi-tech and manufacturing, insurance, consumer services, and more.Our teams specialize in cutting-edge big data solutions, leveraging...
-
Senior Data Engineer with GCP
2 miesięcy temu
Gdańsk, Polska Endava Pełny etatJob DescriptionOur data team has expertise across engineering, analysis, architecture, modeling, machine learning, artificial intelligence, and data science. This discipline is responsible for transforming raw data into actionable insights, building robust data infrastructures, and enabling data-driven decision-making and innovation through advanced...
-
Data Engineer
3 tygodni temu
Gdańsk, Trójmiasto, Polska HAYS Poland Pełny etatHays IT Contracting is a cooperation based on B2B principles. Our company matches IT specialists to the most interesting technological projects on the market. Join the group of 500 satisfied Contractors and work for Hays Clients!We are currently looking for candidates for the position:Data Engineer - Snowflake SpecialistWork model: Hybrid, at least 1 day...
-
Middle Data Engineer
6 miesięcy temu
Gdańsk, Polska Ciklum Pełny etatDescription Ciklum is looking for a Middle Data Engineer to join our team full-time in Poland. We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and...
-
Data Engineer
5 miesięcy temu
Gdańsk, Polska Epam Pełny etatDescription We are seeking a Data Engineer who would like to join EPAM Data Competency Center and grow professionally by participating in a project for a technological giant. Our work involves close collaboration with clients, understanding business needs, providing top-tier engineering experiences, and applying best practices to consistently create...
-
Senior Cloud Data Solutions Expert
1 miesiąc temu
Lęborska, Gdańsk, Polska Devopsbay Pełny etatJob DescriptionAs a Senior Cloud Data Solutions Expert at Devopsbay, you will play a crucial role in designing and deploying cutting-edge big data solutions leveraging the Databricks platform.About the Role:10+ years of experience as a Data Architect with a strong background in cloud-based data solutions, especially with Azure and Databricks.Hands-on...
-
Cloud Data Integration Specialist for Financial Services
1 miesiąc temu
Gdańsk, Pomerania, Polska State Street Pełny etatCompany OverviewState Street is a leading global financial services provider with a rich history of innovation and excellence. With a presence in over 100 markets worldwide, we offer a wide range of investment servicing, data & analytics, investment research & trading, and investment management solutions to institutional clients.We are committed to helping...
-
Gdańsk, Pomerania, Polska Grid Dynamics Poland Pełny etatWe are seeking a skilled Senior Machine Learning Engineer to join the Consumer and Commercial Data Engineering & ML Engineering team at Grid Dynamics Poland.About the Role:This is an exciting opportunity to execute data infrastructure requests from Data Science teams focused on customer and store modeling, leveraging expertise in data engineering and/or ML...
-
Cloud Data Engineer
1 tydzień temu
Gdańsk, Trójmiasto, Polska Antal Sp. z o.o. Pełny etatNasz Klient to globalny lider w transformacji i zarządzaniu biznesem, który wykorzystuje moc technologii do budowania zrównoważonej i inkluzywnej przyszłości. Poszukujemy doświadczonego Cloud Data Engineer, który dołączy do naszego zespołu w lokalizacjach: Wrocław, Poznań, Gdańsk, Kraków lub Warszawa.Zadania:Projektowanie, rozwijanie i...
-
Big Data Engineer for Hybrid Platform Development
4 tygodni temu
Gdańsk, Pomerania, Polska Nordea Bank Pełny etatWe are Nordea Bank, a 200-year-old institution supporting and growing the Nordic economies. Our values are deeply rooted in open, progressive, and collaborative societies.Our vision is to enable dreams and aspirations for a greater good by building relationships, learning, and leading change.About Our TeamWe are the Data Tooling team at Nordea, responsible...
-
Data Engineer
7 miesięcy temu
Gdańsk, Polska Lyreco Group Pełny etatResponsible for building systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. The data engineer aims to make data accessible so that the organization can use it to evaluate and optimize its performance Are you excited for your new career adventure? At Lyreco, we offer more than...
-
Senior Automation QA Specialist for Data/ETL/DW Project
2 tygodni temu
Gdańsk, Pomerania, Polska Emagine Sp. Z O.o. Pełny etatEmagine Sp. Z O.o. Seeks Experienced Senior Automation QAWe are looking for a skilled Senior Automation QA to work on a long-term project in the data/ETL/DW area with expertise in SQL and Python required. The ideal candidate will have at least 8-10 years of experience in Quality Assurance Engineer & testing process, and proficiency in ETL/Data Warehouse/Data...
-
Data Engineer Lead
1 miesiąc temu
Gdańsk, Pomerania, Polska Arla foods Pełny etatAbout the roleArla Foods, a leading global dairy company, is seeking an experienced Senior Data Engineer to join our Commercial IT organization. As a key member of our team, you will play a critical role in supporting the continuous rollout and optimization of our global Data Foundation footprint in Microsoft Azure and other platforms.The ideal candidate...
-
Senior Data Modeler
7 miesięcy temu
Gdańsk, Polska Dynatrace Pełny etatJob DescriptionYou will take active part in identifying ways to make our platform better, working with various data, and transforming that data into solid solutions that make business decisions easier. You will help to provide value by enabling better self-serve for our customers and improving their time to reliable insights. We guarantee plenty of...
-
Gdańsk, Pomerania, Polska State Street Pełny etatAbout the RoleWe are seeking a highly skilled Data Integration Senior Developer to join our team at State Street. As a Data Integration Senior Developer, you will be responsible for designing and developing cutting-edge data integration solutions for our clients.Responsibilities:Design and develop ETL data pipelines using Talend Cloud Data Integration...
-
Backend Software Engineer with AWS Experience
2 tygodni temu
Bernarda Chrzanowskiego , Gdańsk, Polska 7N Pełny etatAbout 7N ConsultantsWe are looking for a skilled Backend Software Engineer to join our dynamic team developing streaming platforms using Python and Node.js. The successful candidate will have experience in designing, developing, and maintaining scalable backend applications on AWS.Key Responsibilities:Design and implement efficient, reusable code using...