Big Data Lead Engineer

2 tygodni temu

Warszawa, Mazovia, Polska TN Poland Pełny etat

ALM Services Technology Group comprises creative, open-minded individuals who develop innovative solutions daily to help our clients expand their businesses.

Our mission is to create the best possible environment of work for our people, engage in innovative projects, and help to strengthen our competences.

Since 2020 we have been cooperating with our partner, a multinational company working in a med-tech and analytics space. A recognized global leader still willing to challenge the status quo to improve patient care.

We are currently seeking resources with experience in building and taking to production low latency, Massive Parallel Processing (MPP) data and analytic systems, ideally on Hadoop, Scala, and Spark.

Your responsibilities
  1. Data Engineer on the Data Lake/Hadoop application. Work on building data pipelines to load and manipulate data onto the Data Lake. Optimize data architecture for consumption, utilization, and analytics with data on Hadoop, including for data science, machine learning, and statistical use cases.
  2. Help lead the charge on a data lake store strategy, ensuring rapid delivery while taking responsibility for applying standards, principles, theories, and concepts.
  3. Responsible for design and delivery of data models, which power BI initiatives, dashboards, syndicated reporting, and ad-hoc data exploratory canvases for IQVIA solutions.
  4. Work with data architects on the logical data models and physical database designs optimized for performance, availability, and reliability.
  5. Tuning and optimization of backend and frontend data operations.
  6. Serve as a query tuning and optimization technical expert, providing feedback to the team.
  7. Design and develop ETL and master data management processes.
  8. Scripting and automation to support development, QA, and production database environments and deployments to production.
  9. Define and help enforce data governance and security policy.
  10. Mentor development team members.
  11. Proactively help to resolve difficult technical issues.
  12. Provide technical knowledge to teams during project discovery and architecture phases.
  13. Keep management informed of work activities and schedules.
  14. Assess new initiatives to determine the work effort and estimate the necessary time-to-completion.
  15. Document new development, procedures, or test plans as needed.
  16. Participate in data builds and deployment efforts. Help mature our Continuous Integration and Continuous Deployment methodologies.
  17. Participate in projects through various phases.
  18. Perform other related duties as assigned.
  19. Partner with the business units to develop effective solutions that solve business challenges.
  20. Should be able to lead a small team of Big Data engineers and manage them.
Years of experience
  1. 3+ years strong native SQL skills.
  2. 3+ years strong experience in database and data warehousing/data lake concepts and techniques. Understand relational and dimensional modeling, star/snowflake schema design, BI, Data Warehouse operating environments and related technologies, ETL, MDM, and data governance practices.
  3. 2+ years' experience working in Linux.
  4. 2+ years' experience with Hadoop, Hive, Impala, HBase, and related technologies.
  5. 3+ years strong experience with low latency (near real-time) systems and working with Tb data sets, loading and processing billions of records per day.
  6. 3+ years' experience with Spark, Scala.
  7. 3+ years' experience with MPP, shared nothing database systems, and NoSQL systems.
Required skills/experience
  1. Hadoop, Hive, and related technologies.
  2. Spark and Scala – Mid to Advanced level of proficiency.
  3. MPP, shared nothing database systems, NoSQL systems.
  4. Object Oriented and Functional Programming Experience.
  5. Modern DevOps experience.
  6. Linux.
  7. Data Warehousing design and concepts.
Benefits
  • Multisport.
  • Group Insurance.
  • Modern Office.
  • Fruit Buffet.
  • Health Care.
  • Flexible Work Hours.
  • Remote work.
  • Salary in foreign currency.
  • Certificates.
  • Workation.
#J-18808-Ljbffr

  • Senior Big Data Architect

    2 dni temu

    Warszawa, Mazovia, Polska Grid Dynamics International, Inc. Pełny etat

    We are seeking a highly skilled Senior Big Data Engineer to join our team at Grid Dynamics International, Inc. As a Senior Big Data Engineer, you will be responsible for designing and developing Big Data solutions that handle petabytes datasets.Key ResponsibilitiesDesign and develop Big Data solutions for real-time data processing and data quality...

  • Big Data Engineer

    1 tydzień temu

    Warszawa, Mazovia, Polska Grid Dynamics International, Inc. Pełny etat

    Big Data Engineer (Long term US Business Travel)Grid Dynamics, a global software services company driving enterprise-level digital transformation solutions for Fortune 1000 corporations, is looking for an experienced Big Data Engineer.Essential functionsResponsibilities:Build data pipelines to enable training of LLMs.Work with technical and non-technical...

  • Data Engineer Lead

    2 dni temu

    Warszawa, Mazovia, Polska Grid Dynamics International, Inc. Pełny etat

    As a Senior Big Data Engineer at Grid Dynamics International, Inc., you will be working on the bleeding edge of technology to develop high-performance data analytics platforms. Our customer is one of the world's largest technology companies based in Silicon Valley with operations all over the world.Job DescriptionWe are looking for an experienced Big Data...

  • Big Data Analyst Lead

    12 godzin temu

    Warszawa, Mazovia, Polska CLOUDFIDE SPÓŁKA Z OGRANICZONĄ ODPOWIEDZIALNOŚCIĄ Pełny etat

    About Us:CLOUDFIDE SPÓŁKA Z OGRANICZONĄ ODPOWIEDZIALNOŚCIĄ is a leader in cloud-based solutions.Job Summary:We are seeking an experienced Big Data Analyst Lead to join our team.Key Responsibilities:Designing and implementing modern cloud-based solutions to meet the needs of our business.Building and launching new data models and data pipelines to drive...

  • Cloud Data Engineer Lead

    3 dni temu

    Warszawa, Mazovia, Polska BitPeak Pełny etat

    BitPeak is looking for an experienced Cloud Data Engineer Lead to join our team. As a senior member of our project teams, you will lead the development of a modern data platform in the cloud for global brands.You will work in an international environment, specializing in BI & Big Data cloud architecture and the latest technologies in this area. Your role...

  • Big Data Solutions Developer

    2 dni temu

    Warszawa, Mazovia, Polska Grid Dynamics International, Inc. Pełny etat

    Join our team as a Senior Big Data Engineer and work on the bleeding edge of technology to develop high-performance data analytics platforms. As a member of our team, you will be responsible for designing and developing Big Data solutions that handle petabytes datasets.Key QualificationsMaster's degree in Computer Science or related field.Minimum 7 years of...

  • Senior Big Data Engineer

    3 dni temu

    Warszawa, Mazovia, Polska Grid Dynamics International, Inc. Pełny etat

    Our customer is one of the world's largest technology companies based in Silicon Valley with operations all over the world. In this project, we are working on the bleeding edge of Big Data technology to develop a high-performance data analytics platform, which handles petabytes datasets.We are looking for an experienced Big Data Engineer.Essential...

  • Big Data Architecture Lead

    5 dni temu

    Warszawa, Mazovia, Polska Randstad Pełny etat

    About RandstadRandstad is a global leader in the recruitment and HR services industry. We help organizations around the world find, engage, and retain the best talent for their businesses.Job TitleData Architect - Cloud SolutionsJob SummaryWe are seeking an experienced Data Architect to join our team in designing and implementing scalable and secure...

  • Data Engineer/Inżynier Big Data

    5 godzin temu

    Warszawa, Mazovia, Polska Jit Team Pełny etat

    Salary: PLN/MD on B2B Work model: elastic hybrid from Gdynia / Gdańsk / Warszawa (1 day per week in the office)Collaborate with experienced developers , expand your skills in Big Data processing , and access Jit-funded training programs Work closely with stakeholders, ensuring that data-driven decisions shape the success of the organization Join...

  • Big Data Solutions Expert

    4 dni temu

    Warszawa, Mazovia, Polska SQUARE ONE RESOURCES sp. z o.o. Pełny etat

    Job DescriptionWe are Square One Resources, a trusted partner company specializing in IT recruitment for 30 years. We understand the needs of IT experts and offer exciting opportunities for those seeking new challenges and skill enhancements.This role is designed for an experienced Data Engineer with expertise in Big Data or Machine Learning. The ideal...

  • Big Data Engineer

    3 dni temu

    Warszawa, Mazovia, Polska Team Up Pełny etat

    For our client, a leading innovator in the technology and electronics industry, we are seeking Big Data Engineers. This project aims to connect brands with TV audiences by building a cutting-edge advertising platform using machine learning algorithms to enhance targeting, personalization, and optimization. Work alongside data scientists, product managers,...

  • Big Data

    6 dni temu

    Warszawa, Mazovia, Polska ITMAGINATION Sp. z o. o. Pełny etat

    We're hiring a Big Data & Cloud Developer (Hybrid work from Warsaw, Poland) at ITMAGINATION, join our team of 400+ developers, analysts and testers ITMAGINATION helps its Clients by becoming a true extension of their software and data development capabilities. Through the readily set up, comprehensive, and self-governing teams, we let our Clients focus on...

  • Hadoop Big Data Administrator/Engineer

    3 tygodni temu

    Warszawa, Mazovia, Polska Capco Poland Pełny etat

    *We are looking for Poland based candidate.  Joining Capco means joining an organisation that is committed to an inclusive working environment where you're encouraged to #BeYourselfAtWork. We celebrate individuality and recognize that diversity and inclusion, in all forms, is critical to success. It's important to us that we recruit and develop as diverse a...

  • Data Engineer for Big Data Solutions

    5 dni temu

    Warszawa, Mazovia, Polska B2Bnetwork Pełny etat

    B2Bnetwork seeks a highly skilled ITSM Data Engineer (DevOps) to contribute to designing, building, and maintaining stable, scalable, and reliable BMC Helix data replication solutions.The project involves processing large data sets, optimizing data architecture, and implementing modern ETL solutions. We collaborate in a hybrid model, with at least one day...

  • Big Data Solutions Specialist

    2 dni temu

    Warszawa, Mazovia, Polska instinctools EE Labs Pełny etat

    As a Senior Data Engineer at instinctools, you will be part of a dynamic team that delivers cutting-edge data solutions to our clients.You will work closely with our clients to understand their specific needs and provide personalized solutions that meet their business requirements.The project involves designing and implementing scalable data processing...

  • Junior Big Data Application Engineer

    3 dni temu

    Warszawa, Mazovia, Polska DXC Technology Inc. Pełny etat

    Big Data Specialist / Specjalista ds. Big DataBig Data Specialist / Specjalista ds. Big DataApply locations POL - ANY CITY time type Full time posted on Posted 3 Days Ago job requisition id 51532389Job Description:DXC Technology (NYSE: DXC) is the world's leading independent, end-to-end IT services company, helping clients harness the power of innovation to...

  • Big Data Solutions Architect

    2 dni temu

    Warszawa, Mazovia, Polska Addepto Pełny etat

    We are looking for a highly skilled Senior Data Engineer to join our team of technology experts. As a Senior Data Engineer, you will design, implement, and optimize scalable data pipelines on GCP using BigQuery for both batch and real-time data processing.About the JobYou will develop, enhance, and maintain dashboards in Looker to provide insightful and...

  • Senior Cloud Engineer

    3 dni temu

    Warszawa, Mazovia, Polska Addepto Pełny etat

    Main Responsibilities:Design and optimize scalable data processing pipelines for both streaming and batch workloads using Big Data technologies such as Databricks, Apache Airflow, and Dagster.Architect and implement end-to-end data platforms, ensuring high availability, performance, and reliability.Lead the development of CI/CD and MLOps processes to...

  • Data Engineer Lead

    3 dni temu

    Warszawa, Mazovia, Polska Addepto Pełny etat

    At Addepto, we are seeking a highly skilled Senior Data Engineer to join our team of technology experts. As a Senior Data Engineer, you will have the exciting opportunity to work on challenging projects across various industries, leveraging cutting-edge technologies such as BigQuery and Google Cloud Platform.">Main ResponsibilitiesDesign, implement, and...

  • mid/senior big data engineer

    2 dni temu

    Warszawa, Mazovia, Polska Capco Pełny etat

    CAPCO POLAND*We are looking for Poland based candidate.Joining Capco means joining an organisation that is committed to an inclusive working environment where you're encouraged to #BeYourselfAtWork. We celebrate individuality and recognize that diversity and inclusion, in all forms, is critical to success. It's important to us that we recruit and develop as...