Senior Data Engineer – Scala

5 dni temu

Województwo pomorskie, Województwo pomorskie, Polska B2Bnetwork Pełny etat 140 000 zł - 190 000 zł Umowa

We are looking for an experienced Data Engineer to join an international team working on a strategic regulatory initiative within the financial sector. You will develop and maintain a data platform and reporting processes operating in a complex Big Data environment.

Responsibilities

  • Design, develop and maintain Data Engineering solutions using Hadoop and Apache Spark.
  • Build scalable solutions for processing large volumes of data.
  • Analyse complex business requirements and translate them into effective technical solutions.
  • Develop and optimise SQL queries and data-processing workflows.
  • Build and enhance data pipelines and reporting processes.
  • Create and maintain workflows using Apache Oozie and Apache Airflow.
  • Develop data solutions using Scala.
  • Work with event-driven architecture based on Apache Kafka.
  • Maintain and improve CI/CD processes using Git, pull requests and Jenkins.
  • Provide L3 support for production environments.
  • Investigate production incidents and technical problems in cooperation with relevant operational teams.
  • Collaborate with business stakeholders to define requirements and analyse issues.
  • Ensure the quality, performance, security and reliability of data solutions.

Requirements

  • Strong hands‑on experience with Scala.
  • Commercial experience with Apache Hadoop.
  • Practical knowledge of Apache Spark and large-scale data processing.
  • Experience building and maintaining Data Engineering pipelines.
  • Experience with Apache Kafka and event‑driven architecture.
  • Practical knowledge of CI/CD processes, particularly Jenkins.
  • Very good knowledge of Git, code review and pull request processes.
  • Strong SQL development and optimisation skills.
  • Experience supporting production environments.
  • Ability to independently investigate complex technical issues.
  • English proficiency sufficient for daily work in an international team.
  • Availability to work from the Warsaw or Gdańsk office twice per week.
  • Ability to perform the work from Poland.

Nice to have

  • Kubernetes experience.
  • Knowledge of Apache Flink.
  • Experience with Apache Airflow or Apache Oozie.
  • Previous involvement in regulatory or reporting projects.
  • Experience in the financial sector.

What we offer

  • Participation in a strategic regulatory initiative with a tangible impact on the financial sector.
  • Work within a large and advanced Big Data environment.
  • A modern technology stack including Scala, Hadoop, Spark, Kafka, Airflow and Jenkins.
  • A high level of independence and influence over technical solutions.
  • Cooperation with an experienced international team.
  • Stable, long‑term cooperation.
  • A professional environment focused on quality, security and expert development.
  • An opportunity to gain experience in financial regulations and regulatory reporting.