Big Data Engineer Lead

6 dni temu


Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etat

We are looking for a highly skilled Big Data Engineer Lead to join our team at Luxoft. As a key member of our data engineering team, you will be responsible for leading the development of our modern cloud-based data processing platform.

The ideal candidate will have extensive experience in building data processing pipelines using various ETL/ELT design patterns and methodologies to Azure data solutions, as well as expertise in technologies such as ADLSv2, Azure Data Factory, Databricks, Python, and PySpark.

This role requires a strong background in software development craft, with focus on cloud-based event-driven solutions and architectures, including Apache Spark batch and streaming, Datalakehouses using medallion architecture. Knowledge of DataMesh principles is an added plus.

Key Skills

  • Expertise in building data processing pipelines using various ETL/ELT design patterns and methodologies to Azure data solutions.
  • Experience with technologies such as ADLSv2, Azure Data Factory, Databricks, Python, and PySpark.
  • Deep understanding of software development craft, with focus on cloud-based event-driven solutions and architectures.
  • Ability to debug using tools like Ganglia UI, expertise in Optimizing Spark Jobs.
  • The ability to work across structured, semi-structured, and unstructured data, extracting information and identifying linkages across disparate datasets.


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etat

    Lead Big Data Engineer (with Azure) - Wroclaw or KrakowLocation: Wroclaw or Krakow - Hybrid work (3 days from office)Project Description:The project focuses on building a modern cloud-based data processing platform, enabling efficient management and real-time analysis of large datasets. Key aspects include implementing ETL/ELT pipelines, optimizing data...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etat

    Are you ready to become one of the Luxofters?Who we are:Luxoft DXC Technology Company is an established company focusing on consulting and implementation of complex projects in the financial industry. At the interface between technology and business, we convince with our know-how, well-founded methodology and pleasure in success. As a reliable partner to our...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Poland Pełny etat

    The project focuses on building a modern cloud-based data processing platform, enabling efficient management and real-time analysis of large datasets. Key aspects include implementing ETL/ELT pipelines, optimizing data processing, and ensuring compliance with industry regulations. The solution leverages Azure, Apache Spark, and Data Lakehouse architecture to...

  • Data Engineer Lead

    2 dni temu


    Wrocław, Województwo dolnośląskie, Polska Addepto Pełny etat

    We are looking for a highly skilled Data Engineer Lead to join our team at Addepto. Our company specializes in AI and Big Data, helping clients deliver innovative data projects. As a Data Engineer Lead, you will have the opportunity to work with a team of technology experts on challenging projects across various industries, leveraging cutting-edge...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etat

    About the RoleWe are seeking an experienced Lead Big Data Engineer to join our team. In this role, you will lead the development of cloud-based data processing platforms, enabling efficient management and real-time analysis of large datasets.Responsibilities:Data Pipeline Development: Engineer reliable data pipelines for sourcing, processing, distributing,...


  • Wrocław, Województwo dolnośląskie, Polska K&K social resources and development GmbH Pełny etat

    We are seeking a highly skilled Big Data Engineer Specialist to join our team at K&K Social Resources and Development GmbH. As an international recruiting agency, we provide technical resources in the European region since 1993.The ideal candidate should have experience in building and optimizing data pipelines, ensuring efficient data processing and...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etat

    We are seeking an experienced Big Data Solutions Architect to lead the design and implementation of innovative big data solutions using Azure. The successful candidate will be responsible for developing and maintaining large-scale data processing systems, ensuring high performance, scalability, and reliability.The ideal candidate will have hands-on...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etat

    The project focuses on building a modern cloud-based data processing platform, enabling efficient management and real-time analysis of large datasets. Key aspects include implementing ETL/ELT pipelines, optimizing data processing, and ensuring compliance with industry regulations. The solution leverages Azure, Apache Spark, and Data Lakehouse architecture to...

  • Senior Big Data Engineer

    1 dzień temu


    Wrocław, Województwo dolnośląskie, Polska Correctcontext Pełny etat

    Correct Context is a global leader in media analytics, revolutionizing insights into consumer behavior, media consumption, and digital engagement.We are looking for a talented Java/Spark Big Data Developer to join our international team in Poland. As a trusted partner delivering accurate data to global businesses, we collaborate with industry leaders like...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etat

    About the JobThis is a mid-senior level position in our information technology department. The job involves working with Azure, Apache Spark, and Data Lakehouse architecture to support strategic business decisions.Responsibilities:Data Platform Development: Engineer reliable data pipelines for sourcing, processing, distributing, and storing data in different...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Poland Pełny etat

    About the RoleLuxoft Poland is looking for a highly skilled Big Data Engineering Manager to lead our data engineering team. As a manager, you will be responsible for ensuring the quality, security, reliability, and compliance of our data solutions.You should have experience in traditional data warehousing concepts (Kimball Methodology, Star Schema, SCD) /...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Poland Pełny etat

    We are seeking a highly skilled Lead Data Processing Specialist to join our data engineering team. As a Lead Data Processing Specialist, you will be responsible for designing and implementing modern data processing platforms using big data technologies.About the RoleStrong expertise in big data technologies such as Apache Spark, Hadoop, and NoSQL databases...


  • Wrocław, Województwo dolnośląskie, Polska Experis ManpowerGroup Sp. z o.o. Pełny etat

    About the TeamWe are looking for a highly skilled Big Data Infrastructure Manager to join our team. In this role, you will be responsible for designing, developing, and deploying big data infrastructure solutions that meet the needs of our clients.Key Responsibilities:Design and develop scalable big data infrastructure solutions using Hadoop, Spark, and...

  • Big Data Innovator

    2 dni temu


    Wrocław, Województwo dolnośląskie, Polska Addepto Pełny etat

    We are seeking a highly motivated Big Data Innovator to join our team at Addepto. As a Big Data Innovator, you will have the opportunity to work with a team of technology experts on challenging projects across various industries, leveraging cutting-edge technologies. Your main responsibilities will include designing, implementing, and optimizing scalable...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etat

    We are seeking a seasoned Azure Data Engineer Lead to lead the design and implementation of innovative data engineering solutions using Azure. The successful candidate will be responsible for developing and maintaining large-scale data processing systems, ensuring high performance, scalability, and reliability.The ideal candidate will have hands-on...


  • Wrocław, Województwo dolnośląskie, Polska DCV Technologies Pełny etat

    An exciting opportunity has arisen for an experienced Enterprise Data Hub (EDH) Architect to join our team at DCV Technologies. As a key member of our data engineering team, you will play a critical role in the design and implementation of a centralized data platform that drives business growth and innovation.About the RoleThe EDH Architect will be...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Poland Pełny etat

    As a Senior Big Data Solutions Architect at Luxoft Poland, you will play a crucial role in designing and developing innovative big data solutions for our clients. You will work closely with cross-functional teams to deliver high-quality solutions that meet the evolving needs of our customers.About the JobWe are looking for an experienced architect who can...


  • Wrocław, Województwo dolnośląskie, Polska Comscore (via CC) Pełny etat

    Correct Context is looking for a  Scala/Linux Big Data Developer for Comscore in Poland and around. Comscore is a global leader in media analytics, revolutionizing insights into consumer behavior, media consumption, and digital engagement. Comscore leads in measuring and analyzing audiences across diverse digital platforms. Thrive on using cutting-edge...


  • Wrocław, Województwo dolnośląskie, Polska Luxoft Poland Pełny etat

    The project focuses on building a modern cloud-based data processing platform, enabling efficient management and real-time analysis of large datasets. Key aspects include implementing ETL/ELT pipelines, optimizing data processing, and ensuring compliance with industry regulations. The solution leverages Azure, Apache Spark, and Data Lakehouse architecture to...


  • Wrocław, Województwo dolnośląskie, Polska Correctcontext Pełny etat

    Career Opportunities with Correct ContextWe are a leading company in media analytics, and we're looking for a talented DevOps Engineer to join our team. As a DevOps Engineer, you'll play a key role in maintaining and optimizing our high-volume, multi-tenant, and distributed data processing pipelines.Your responsibilities will include collaborating on cluster...