Senior Data Engineer – Scala
5 dni temu
Województwo pomorskie, Województwo pomorskie, Polska
B2Bnetwork
Pełny etat
140 000 zł - 190 000 zł Umowa
Bezpłatnie za pośrednictwem poczty elektronicznej lub Google
Zapisz tę ofertę pracy i uporządkuj wyszukiwanie
Utwórz bezpłatne konto, aby zapisywać oferty pracy, tworzyć alerty i wracać do tego ogłoszenia ze swojego pulpitu nawigacyjnego.
Bezpłatnie za pośrednictwem poczty elektronicznej lub Google
We are looking for an experienced Data Engineer to join an international team working on a strategic regulatory initiative within the financial sector. You will develop and maintain a data platform and reporting processes operating in a complex Big Data environment.
Responsibilities
- Design, develop and maintain Data Engineering solutions using Hadoop and Apache Spark.
- Build scalable solutions for processing large volumes of data.
- Analyse complex business requirements and translate them into effective technical solutions.
- Develop and optimise SQL queries and data-processing workflows.
- Build and enhance data pipelines and reporting processes.
- Create and maintain workflows using Apache Oozie and Apache Airflow.
- Develop data solutions using Scala.
- Work with event-driven architecture based on Apache Kafka.
- Maintain and improve CI/CD processes using Git, pull requests and Jenkins.
- Provide L3 support for production environments.
- Investigate production incidents and technical problems in cooperation with relevant operational teams.
- Collaborate with business stakeholders to define requirements and analyse issues.
- Ensure the quality, performance, security and reliability of data solutions.
Requirements
- Strong hands‑on experience with Scala.
- Commercial experience with Apache Hadoop.
- Practical knowledge of Apache Spark and large-scale data processing.
- Experience building and maintaining Data Engineering pipelines.
- Experience with Apache Kafka and event‑driven architecture.
- Practical knowledge of CI/CD processes, particularly Jenkins.
- Very good knowledge of Git, code review and pull request processes.
- Strong SQL development and optimisation skills.
- Experience supporting production environments.
- Ability to independently investigate complex technical issues.
- English proficiency sufficient for daily work in an international team.
- Availability to work from the Warsaw or Gdańsk office twice per week.
- Ability to perform the work from Poland.
Nice to have
- Kubernetes experience.
- Knowledge of Apache Flink.
- Experience with Apache Airflow or Apache Oozie.
- Previous involvement in regulatory or reporting projects.
- Experience in the financial sector.
What we offer
- Participation in a strategic regulatory initiative with a tangible impact on the financial sector.
- Work within a large and advanced Big Data environment.
- A modern technology stack including Scala, Hadoop, Spark, Kafka, Airflow and Jenkins.
- A high level of independence and influence over technical solutions.
- Cooperation with an experienced international team.
- Stable, long‑term cooperation.
- A professional environment focused on quality, security and expert development.
- An opportunity to gain experience in financial regulations and regulatory reporting.