Lead Data Engineer – Core Data Platform

6 dni temu

Gdańsk, Pomerania, Polska Hapag-Lloyd Pełny etat 80 000 zł - 120 000 zł rocznie
Description

We are seeking an experienced Lead Data Engineer– Core Data Platform to drive the development and stability of our cloud-native data infrastructure. This role is crucial in ensuring our data platform is secure, reliable, and scalable. You'll be responsible for the provisioning and automation of key components including Databricks, Airflow (Astronomer), GitLab CI/CD, Terraform, and monitoring tools such as Grafana. You will collaborate closely with DevOps, Security, and Architecture teams and provide platform support to other data teams, including those building the new Data Warehouse in Databricks. As a hands-on technical leader, you'll contribute to the platform codebase while defining standards, improving automation, and ensuring operational excellence across environments.

Responsibilities

Design and maintain the core data platform infrastructure including Databricks, Airflow (Astronomer), AWS, CI/CD pipelines, and monitoring. 

Manage and provision cloud environments using Terraform; maintain GitLab-based CI/CD workflows. 

Own the RBAC implementation and environment governance ensuring secure and maintainable access controls. 

Monitor cost and performance metrics using observability tools 

Troubleshoot infrastructure-level issues and support DataOps teams working on the Data Lakehouse. 

Collaborate closely with DevOps, Security, Architecture, and Product teams to align on platform standards. 

Support deployment processes from development to production for data workflows. 

Maintain documentation of architectural decisions, operational procedures, and tooling. 

Proactively identify and implement automation and improvements across platform components. 

Contribute directly to platform development, including coding, reviews, and mentoring. 

Maintain a modular and scalable Terraform repository structure for multi-environment deployments. 

Contribute to the development of internal tooling for platform automation and efficiency. 

Define tagging strategies and cost monitoring standards across environments and workspaces. 

Coordinate incident response and platform stability improvements. 

Partner with Security and Cloud Governance teams on policies, audits, and compliance initiatives. 

Track and address technical debt within platform infrastructure components. 

Qualifications

Minimum 6 years of experience in platform or DevOps engineering in a data context. 

Expertise in provisioning and automating cloud infrastructure using Terraform. 

Strong knowledge of AWS services (IAM, networking, S3, cost tracking) and Databricks. 

Handson experience with orchestrators like Airflow (Astronomer) and CI/CD pipelines (GitLab). 

Familiarity with Databricks workspace and admin configurations. 

Experience implementing RBAC, securing access and environments. 

Experience in monitoring performance and setting up alerting/observability (Grafana, Prometheus). 

Strong coding skills in Python and pySpark. 

Ability to work closely with both technical and business stakeholders. 

Mindset focused on scalability, reliability, and documentation. 

Deep understanding of CI/CD strategies for data platforms and workflow-driven applications. 

Familiarity with observability stacks (logs, metrics, traces) and SRE practices. 

Experience in supporting multi-account cloud environments and cross-region deployments. 

Proficiency in debugging infrastructure-related issues in cloud-native data pipelines. 

Strong documentation habits and ability to produce clear, actionable technical runbooks. 

Familiarity with platform cost optimization tools and practices (e.g., AWS Cost Explorer, native tagging, budgeting). 

  • Data Platform Lead

    1 tydzień temu

    Gdańsk, Pomerania, Polska TF Bank Pełny etat 90 000 zł - 120 000 zł rocznie

    We are seeking a highly skilled Data Platform Lead to drive the design and development of Avarda Bank's next-generation data platform. This is a pivotal role where you will own the architecture, governance, and build of a cloud-native, event-driven platform that powers both advanced analytics and the consumer journey across Europe. You will combine strategic...

  • Lead Data Engineer – Data Ingestion

    6 dni temu

    Gdańsk, Pomerania, Polska Hapag-Lloyd Pełny etat 60 000 zł - 120 000 zł rocznie

    DescriptionWe are looking for a hands-on Lead Data Engineer – Data Ingestion to guide the development and execution of our data ingestion processes into the Data Lakehouse. In this role, you will be responsible for designing scalable and reliable data pipelines, transforming source inputs into trusted Bronze-layer assets. You'll work closely with data...

  • Lead Data Engineer – Data Ingestion

    1 tydzień temu

    Gdańsk, Pomerania, Polska Hapag-Lloyd AG Pełny etat 80 000 zł - 120 000 zł rocznie

    We are looking for a hands-on Lead Data Engineer – Data Ingestion to guide the development and execution of our data ingestion processes into the Data Lakehouse. In this role, you will be responsible for designing scalable and reliable data pipelines, transforming source inputs into trusted Bronze-layer assets. You'll work closely with data owners, source...

  • Senior Data Engineer

    1 tydzień temu

    Gdańsk, Pomerania, Polska EPAM Systems Pełny etat 90 000 zł - 120 000 zł rocznie

    We are looking for a dynamic, hands-onSenior Data Engineerfor a full-stack AI native Data Eng team for building a cutting-edge quantitative investment platform.This role requires 5 days of work per week from the office in Krakow or Gdansk.ResponsibilitiesImplement data pipelines for structured, unstructured, and operational data sourcesEnsure efficient data...

  • Expert DevOps/SRE Engineer for a new strategic Data Platform

    2 dni temu

    Gdańsk, Pomerania, Polska Nordea Asset Management Portugal Pełny etat

    Expert DevOps/SRE Engineer for a new strategic Data PlatformGdańsk, PL, Helsinki, FI, 500 Łódź, PL, Gdynia, PL, Warszawa, PL, Stockholm, SE, 112 51Job ID: 30612Welcome to Group Technology, where we pride ourselves on engineering solutions and direct Nordea's transformation by providing a holistic technological view and structured understanding of the...

  • Data Engineer

    6 dni temu

    Gdańsk, Pomerania, Polska Capgemini Pełny etat 60 000 zł - 120 000 zł rocznie

    Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and...

  • Data Engineer

    6 dni temu

    Gdańsk, Pomerania, Polska Sanoma Learning Pełny etat 40 000 zł - 80 000 zł rocznie

    Important:This position isonly for candidates legally residing in Polandand operating under a registered sole proprietorship (JDG).As part of onboarding, we kindly invite new joiners to our Warsaw office on the first day for a short introduction and identity verification.Please note that applications from outside Poland cannot be considered.About UsSanoma...

  • Data Scientist

    4 dni temu

    Gdańsk, Pomerania, Polska ABAX UK Pełny etat 60 000 € - 120 000 € rocznie

    At ABAX, we're transforming telematics across Europe, helping businesses optimize operations, reduce costs, and improve efficiency, safety, and sustainability.As a data scientist, you'll be part of our ABAX central data science team, currently four members strong, develops and maintains core services for 45,000 customers.You'll develop, deploy, and maintain...

  • Data DevOps Engineer

    6 dni temu

    Gdańsk, Pomerania, Polska Hard Rock Digital Pełny etat 40 000 zł - 80 000 zł rocznie

    What are we building?Hard Rock Digital is a team focused on becoming the best online sportsbook, casino, and social casino company in the world. We're building a team that resonates passion for learning, operating, and building new products and technologies for millions of consumers. We care about each customer's interaction, experience, behaviour, and...

  • Lead Platform Engineer

    6 dni temu

    Gdańsk, Pomerania, Polska Kainos Pełny etat 60 000 zł - 120 000 zł rocznie

    Join Kainos and Shape the Future At Kainos, we're problem solvers, innovators, and collaborators - driven by a shared mission to create real impact. Whether we're transforming digital services for millions, delivering cutting-edge Workday solutions, or pushing the boundaries of technology, we do it together.We believe in a people-first culture, where your...