Highly Skilled Data Engineer for Large-Scale Data Processing

4 dni temu


Warszawa, Mazovia, Polska TN Poland Pełny etat

We are seeking an experienced Data Engineer with a strong background in Python and PySpark to work on large-scale data pipelines. Our ideal candidate will have expertise in Cloudera Hadoop ecosystem and be responsible for designing, developing, and maintaining efficient ETL pipelines.

About the Role

The successful candidate will collaborate closely with our data science team to deliver data-driven solutions, ensuring data integrity, performance, and reliability across distributed systems. Additionally, they will implement best practices for data governance, security, and performance tuning, using Git and GitHub for version control and efficient code collaboration.

Key Responsibilities
  • Design and develop scalable ETL pipelines using Python and PySpark.
  • Work with Cloudera Hadoop ecosystem to manage and process large datasets.
  • Ensure data quality, performance, and reliability across distributed systems.
  • Collaborate with data scientists and business stakeholders to deliver data-driven solutions.
  • Implement data governance, security, and performance best practices.
Requirements
  • Strong programming skills in Python.
  • Hands-on experience with PySpark for distributed data processing.
  • Expertise in Cloudera Hadoop ecosystem (HDFS, Hive, Impala).
  • Experience with SQL and working with large datasets.
  • Knowledge of Git and GitHub for source code management.
Preferred Qualifications
  • Bachelor's degree in Computer Science or related field.
  • Basic knowledge of Databricks for cloud-based big data solutions.
  • Experience with workflow orchestration tools (e.g., Airflow, Oozie).
  • Understanding of cloud platforms (AWS, Azure, or GCP).
What We Offer
  • A dynamic and collaborative work environment.
  • Ongoing training and development opportunities.
  • A competitive salary and benefits package.


  • Warszawa, Mazovia, Polska Talan Pełny etat

    We are looking for a highly skilled Data Engineer to join our team in Warsaw, Poland. As a key member of our organization, you will contribute to the development of cloud-scale data processing solutions using Python and related technologies.The ideal candidate will have strong programming skills in Python, hands-on experience with PySpark for distributed...


  • Warszawa, Mazovia, Polska TN Poland Pełny etat

    Job DescriptionCommodities Risk Technology EngineerTN Poland is looking for a skilled Commodities Risk Technology Engineer to join their team. As a member of this team, you will be responsible for designing and developing technology solutions for our clients, including commodities market modeling, risk analytics, risk/P&L calculation, VaR/Stress calculation,...


  • Warszawa, Mazovia, Polska Tietoevry Pełny etat

    Technical RequirementsWe are seeking a highly skilled Senior Software Engineer-Machine Learning Engineer/Data Engineer to join our team in Warsaw. As a senior engineer, you will play a crucial role in designing, developing, and deploying machine learning models using Python frameworks.Develop and maintain ETL/ELT pipelines using Python and SQLWork with...

  • Big Data Engineer Leader

    16 godzin temu


    Warszawa, Mazovia, Polska Tietoevry Pełny etat

    We are seeking a highly skilled Big Data Engineer Leader to join our team. In this role, you will lead the development and maintenance of ETL/ELT pipelines using Python and SQL, as well as working with Hadoop, Spark, and Big Data technologies for large-scale data processing.The ideal candidate will have 4+ years' experience with database technologies,...


  • Warszawa, Mazovia, Polska Tietoevry Pełny etat

    Job DescriptionTietoevry CreateWe are seeking a highly skilled Senior Software Engineer-Machine Learning Engineer/Data Engineer to join our team in Warsaw. As a key member of our team, you will be responsible for developing and maintaining ETL/ELT pipelines using Python and SQL.Design, train, test, and deploy ML models using Python frameworksWork with...


  • Warszawa, Mazovia, Polska NTIATIVE IT Recruitment Pełny etat

    Job OverviewWe are seeking a highly skilled Enterprise Software Developer to join our team. This role involves designing and developing high-performance, scalable software solutions for risk management in large-scale, cloud-native distributed systems.Key ResponsibilitiesDesign and develop high-performance, scalable software solutions for risk management.Work...


  • Warszawa, Mazovia, Polska SquareOne Pełny etat

    We're seeking an innovator with a strong track record as a Python or Data Engineer to drive forward the software design, implementation, and maintenance of our Remote Monitoring Patient Platform (RPMP) for processing, analysis and storage of digital biomarker data. Python – Strong experience in developing and maintaining large-scale data pipelines using...


  • Warszawa, Mazovia, Polska SquareOne Pełny etat

    We're seeking an innovator with a strong track record as a Python or Data Engineer to drive forward the software design, implementation, and maintenance of our Remote Monitoring Patient Platform (RPMP) for processing, analysis and storage of digital biomarker data.- Python – Strong experience in developing and maintaining large-scale data pipelines using...


  • Warszawa, Mazovia, Polska Samba TV, Inc. Pełny etat

    Data Engineering Role at Samba TVWe're seeking an experienced software engineer to join our Data Engineering team and contribute to the design and architecture of our data platform.This role involves developing new data pipelines, automating business processes, and improving the efficiency and scalability of data collection, storage, and retrieval...


  • Warszawa, Mazovia, Polska Google Inc. Pełny etat

    Google Inc. is at the forefront of developing next-generation technologies that change how billions of users interact with information and each other. Our software engineers play a critical role in designing and developing large-scale systems that meet the demands of our fast-paced business.We're looking for talented individuals who can bring fresh ideas...


  • Warszawa, Mazovia, Polska Axiom Software Solutions Limited Pełny etat

    Job DescriptionAxiom Software Solutions Limited is seeking a highly skilled Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing, developing, and maintaining large-scale data pipelines using Python and PySpark.Key ResponsibilitiesDesign, develop, and maintain ETL (Extract, Transform, Load)...


  • Warszawa, Mazovia, Polska Capgemini Pełny etat

    Job OverviewWe are seeking a highly skilled Senior Cloud Data Engineer to lead our data engineering team in delivering scalable and high-performance data pipelines on Google Cloud Platform (GCP).Key Responsibilities:Architect, develop, and maintain large-scale data pipelines on GCP.Design and implement GCP-native data solutions, ensuring best practices in...


  • Warszawa, Mazovia, Polska Square One Resources Sp. z o.o. Pełny etat

    We're seeking an innovator with a strong track record as a Python or Data Engineer to drive forward the software design, implementation, and maintenance of our Remote Monitoring Patient Platform (RPMP) for processing, analysis and storage of digital biomarker data.Python / Data Engineer with AzureYour responsibilitiesMaintain and develop large scale data...


  • Warszawa, Mazovia, Polska Square One Resources Sp. z o.o. Pełny etat

    At Square One Resources Sp. z o.o., we're on the lookout for an exceptional Python or Data Engineer to spearhead the development and maintenance of our Remote Monitoring Patient Platform (RPMP). This platform requires innovative data processing, analysis, and storage solutions.We're seeking a strong team player with expertise in designing, implementing, and...


  • Warszawa, Mazovia, Polska Cyclad Pełny etat

    Cyclad is seeking a seasoned Data Engineer to spearhead the development of advanced Data Lakehouse, Business Intelligence, and Advanced Analytics solutions in cloud environments.About the RoleWe are looking for an experienced professional to design and enhance data platforms, drive functional and architectural improvements, and develop and optimize ETL/ELT...


  • Warszawa, Mazovia, Polska SQUARE ONE RESOURCES sp. z o.o. Pełny etat

    technologies-expected : Python Azure DevOps MongoDB Docker Kubernetes GitLab azure data lake about-project : We're seeking an innovator with a strong track record as a Python or Data Engineer to drive forward the software design, implementation, and maintenance of our Remote Monitoring Patient Platform (RPMP) for processing, analysis and storage of digital...

  • Senior Data Expert

    4 dni temu


    Warszawa, Mazovia, Polska Cyclad Pełny etat

    Career Opportunities at CycladJob Title: Senior Data EngineerCyclad is looking for a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing and enhancing data platforms, developing and optimizing ETL/ELT processes, and collaborating with Data Engineers & Analysts to enhance technology...


  • Warszawa, Mazovia, Polska Sii Sp. z o.o. Pełny etat

    We are seeking a highly skilled Machine Learning Engineer to join our team. As a Senior ML Engineer, you will be responsible for developing robust solutions to real-world, large-scale problems that have a direct impact on the business.About the ProjectThe goal of this project is to collaborate with the TechOps department to automate and optimize their...


  • Warszawa, Mazovia, Polska Capgemini Polska Sp. z o.o. Pełny etat

    We are looking for a highly skilled Cloud and Big Data Engineer to join our team at Capgemini Polska Sp. z o.o. As a key member of our dynamic team, you will play a crucial role in delivering cutting-edge data solutions.About Us: With over 400 professionals, we specialize in Cloud & Big Data engineering, building scalable systems for complex datasets across...


  • Warszawa, Mazovia, Polska Balyasny Asset Management L.P. Pełny etat

    Our VisionBalyasny Asset Management L.P. strives to leverage technology to drive innovation in risk management and financial analysis.Job SummaryWe are seeking a talented Software Engineer to join our Risk Technology team, focusing on the development of large-scale, cloud-based systems to support risk modeling, risk calculation, and risk reporting.Main...