Senior Big Data Engineer

2 miesięcy temu

Kraków, Polska Grid Dynamics Poland Sp. z o.o. Pełny etat

We are looking for experienced Big Data Engineer specializing in Scala, Hadoop, and Spark within a large enterprise digital product company. The role involves a mix of technical proficiency, strategic thinking, collaboration, and proactive problem-solving to deliver robust and scalable data solutions.

Senior Big Data Engineer


Your responsibilities

  • Design and Develop Big Data Solutions: Architect, design, and implement scalable and reliable big data solutions using Scala, Hadoop, and Spark technologies to meet business requirements.
  • Data Ingestion and Processing: Develop robust data ingestion processes to acquire, clean, and transform large volumes of structured and unstructured data from various sources into usable formats.
  • Optimize Data Pipelines: Build and optimize data pipelines and workflows for efficient data processing, storage, and retrieval, ensuring high performance and low latency.
  • Cluster Management and Optimization: Manage and optimize Hadoop cluster resources to ensure high availability, reliability, and scalability of big data applications.
  • Real-Time Data Processing: Implement real-time data processing solutions using Spark Streaming or other similar technologies to enable real-time analytics and decision-making.
  • Data Quality and Governance: Establish and enforce data quality standards, data governance policies, and best practices to ensure data accuracy, consistency, and integrity across systems.
  • Performance Tuning and Monitoring: Monitor system performance, troubleshoot bottlenecks, and conduct performance tuning of Spark jobs and Hadoop clusters to optimize resource utilization and job execution times.

Our requirements

  • Strong Programming Skills: Proficiency in Scala and Java, including a deep understanding of functional programming concepts in Scala.
  • Expertise in Big Data Technologies: Extensive experience with Apache Hadoop ecosystem components such as HDFS, MapReduce, Hive, Pig, and HBase.
  • Advanced Spark Knowledge: In-depth knowledge of Apache Spark, including Spark Core, Spark SQL, Spark Streaming, and MLlib for large-scale data processing and analytics.
  • Data Modeling and ETL: Experience with data modeling, schema design, and implementing efficient ETL (Extract, Transform, Load) processes using big data technologies.
  • Cluster Management Tools: Hands-on experience with cluster management tools like Apache YARN, Apache Mesos, or Kubernetes for resource allocation and job scheduling.
  • SQL and NoSQL Databases: Proficiency in SQL for querying and analyzing data stored in relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).

What we offer

  • Opportunity to work on bleeding-edge projects
  • Work with a highly motivated and dedicated team
  • Competitive salary
  • Flexible schedule
  • Benefits package - medical insurance, sports
  • Corporate social events
  • Professional development opportunities
  • Well-equipped office

Aplikuj

 

  • Senior Big Data Engineer

    1 miesiąc temu

    Kraków, Polska HAYS Pełny etat

    Senior Big Data EngineerKrakówNR REF.: 1185786Your new company For our Client, international company located in Krakow, we are currently looking for Senior Big Data Engineer who will work on reporting application which provides data analytics.Your new role Holding this role you will be responsible for: Creating and maintaining data pipeline architecture...

  • Senior Big Data Engineer

    3 tygodni temu

    Kraków, Lesser Poland HAYS POLAND Sp. z o.o. Pełny etat

    Senior Big Data Engineerlokalizacja: Kraków (małopolskie)numer referencyjny:forma zatrudnienia: Pełny etatYour new company For our Client, international company located in Krakow, we are currently looking for Senior Big Data Engineer who will work on reporting application which provides data analytics.Your new role Holding this role you will be...

  • Senior Big Data Engineer

    3 tygodni temu

    Kraków, Lesser Poland HAYS POLAND Sp. z o.o. Pełny etat

    Senior Big Data Engineer Kraków, Lesser Poland Voivodeship Your new companyFor our client, a company that provides SaaS products related to sales, customer support, and other customer communications, we are looking for a Senior Big Data Engineer.Your new role Enterprise Data & Analytics (EDA) is looking for an experienced Senior Data Engineer to join...

  • Senior big data engineer

    3 tygodni temu

    Kraków, Lesser Poland Scalac Pełny etat

    We are looking for a Senior Big Data Engineer with Spark who will be working on an external project in the fintech area.But what's more important is that you will still be an integral part of the Scalac team and work with the best specialists As a Big Data Engineer, you will: Work on an external project and develop an application that is based on the Hadoop...

  • Senior big data engineer

    4 tygodni temu

    Kraków, Lesser Poland Scalac Pełny etat

    We are looking for a Senior Big Data Engineer with Spark who will be working on an external project in the fintech area.But what's more important is that you will still be an integral part of the Scalac team and work with the best specialists As a Big Data Engineer, you will: Work on an external project and develop an application that is based on the Hadoop...

  • Senior Big Data Engineer

    3 tygodni temu

    Kraków, Lesser Poland Zendesk Pełny etat

    Job DescriptionOur Enterprise Data & Analytics (EDA) team is seeking a skilled Senior Data Engineer to join our expanding analytics engineering team. We operate as a globally distributed remote-first team. You will have the opportunity to collaborate in an Agile environment, utilizing the latest engineering practices across the software development...

  • Senior Big Data Engineer

    1 miesiąc temu

    Kraków, Lesser Poland Zendesk Pełny etat

    Job DescriptionOur Enterprise Data & Analytics (EDA) team is seeking a skilled Senior Data Engineer to join our expanding analytics engineering team. We operate as a globally distributed remote-first team. You will have the opportunity to collaborate in an Agile environment, utilizing the latest engineering practices across the software development...

  • Senior Big Data Engineer

    3 tygodni temu

    Kraków, Lesser Poland Base Poland (Poland) Pełny etat

    Our Enterprise Data & Analytics (EDA) is looking for an experienced Senior Data Engineer to join our growing analytics engineering team. We are a globally distributed remote first team. You'll work in a collaborative Agile environment using the latest engineering best practices with involvement in all aspects of the software development lifecycle. You will...

  • Senior Data Engineer

    3 tygodni temu

    Kraków, Lesser Poland DCV Global Recruitment Pełny etat

    Experienced in Scala, Python or Java and Unix/Linux environment on-premisesExperience with developing RESTful APIs using Java Spring BootHands-on experience building data pipelines using Hadoop components (Apache Hadoop, Scala, Apache Spark, YARN, Hive, SQL)Experience optimizing Spark jobs Experience with time-series/analytics db's such as Elasticsearch...

  • Big Data Engineer

    3 tygodni temu

    Kraków, Lesser Poland Integer Group Services Sp. Z O.o Pełny etat

    Big Data Engineer Miejsce pracy: Kraków Your responsibilities Building a platform for processing large amounts of data Building streaming solutions Construction and maintenance of data lake solutions, No SQL databases Building CI/CD solutions, test automation CI/CD solutions, test automation Designing and maintaining ETL/ELT Applying good practices, code...

  • Senior Data Engineer z

    3 tygodni temu

    Kraków, Lesser Poland dmTECH Polska Pełny etat

    Senior Data Engineer z j.niemieckim Kraków, Województwo małopolskie Senior Data Engineer z j.niemieckim Kraków Kraków, Lesser Poland Voivodeship, Polska dmTECH to międzynarodowy zespół ponad tysiąca ekspertek i ekspertów z różnych obszarów IT i technologii wspierających e-commerce pracujących dla dm – jednej z największych sieci drogerii w...

  • Senior Big Data Engineer

    3 tygodni temu

    Kraków, Lesser Poland Zendesk, Inc. Pełny etat

    Job DescriptionOur Enterprise Data & Analytics (EDA) is looking for an experienced Senior Data Engineer to join our growing analytics engineering team. We are a globally distributed remote first team. You'll work in a collaborative Agile environment using the latest engineering best practices with involvement in all aspects of the software development...

  • Big Data Engineer IV

    3 tygodni temu

    Kraków, Lesser Poland OpenX Pełny etat

    We are operating at a scale:100% Cloud-based (GCP) platformOver 400 billion Ad requests every dayOver CPU dailyOver 140 TB RAM dailyOver 50 PB of data per weekOver 1200 production deployments a monthYou will work in all aspects of agile application development, including our enterprise platform that interfaces with a multitude of services that are dependent...

  • Senior Data Engineer

    3 tygodni temu

    Kraków, Lesser Poland Rite NRG Pełny etat

    Senior Data Engineer Kraków, Województwo małopolskie Senior Data Engineer Kraków Kraków, Małopolskie, Polska Dołącz do naszej ekipy Rite NRG jako Senior Data Engineer i stwórz z nami w Polsce centrum dostaw dla lidera branży iGaming. Będziesz miał możliwość rozwijania się w obszarze analizy danych i zarządzania danymi.Jakimi...

  • Big Data Engineer Spark

    3 tygodni temu

    Kraków, Lesser Poland Scalac Pełny etat

    We're looking for Big Data Engineer with Spark and Hadoop who will be working on an external project in the fintech area. But what's more important is that you will still be an integral part of the Scalac team and work with the best specialists As a Big Data Engineer, you will:Work on an external project and develop an application that is based on the Hadoop...

  • Big Data Engineer IV

    3 tygodni temu

    Kraków, Lesser Poland OpenX Pełny etat

    We are operating at a scale:100% Cloud-based (GCP) platformOver 400 billion Ad requests every dayOver CPU dailyOver 140 TB RAM dailyOver 50 PB of data per weekOver 1200 production deployments a monthYou will work in all aspects of agile application development, including our enterprise platform that interfaces with a multitude of services that are dependent...

  • Senior Data Engineer

    3 tygodni temu

    Kraków, Lesser Poland Rite NRG sp. z o.o. Pełny etat

    Senior Data Engineer Kraków Kraków, Lesser Poland Voivodeship, Polska Dołącz do naszej ekipy Rite NRG jako Senior Data Engineer i stwórz z nami w Polsce centrum dostaw dla lidera branży iGaming. Będziesz miał możliwość rozwijania się w obszarze analizy danych i zarządzania danymi.Projektowanie i wdrażanie infrastruktury informacyjnej oraz...

  • Senior Data Engineer z jniemieckim

    3 tygodni temu

    Kraków, Lesser Poland dmTECH Polska Pełny etat

    Senior Data Engineer z jniemieckim Kraków Kraków, małopolskie, Polska dmTECH to międzynarodowy zespół ponad tysiąca ekspertek i ekspertów z różnych obszarów IT i technologii wspierających e-commerce pracujących dla dm – jednej z największych sieci drogerii w Europie. Nasza codzienna praca na realny wpływ na biznes. Przygotowywane przez nas...

  • Mid senior data analytics engineer

    3 tygodni temu

    Kraków, Lesser Poland Transfergo Pełny etat

    Transfer Go is a growing fintech scale-up on a mission to make the world a fairer place for migrants. We strive to provide tailored, more affordable financial services that make people's hard-earned money go further. Now in our 12th year, with over 370 employees in offices across Europe, we're crafting a brilliant, relevant product that makes a difference in...

  • Mid senior data analytics engineer

    1 miesiąc temu

    Kraków, Polska Transfergo Pełny etat

    Transfer Go is a growing fintech scale-up on a mission to make the world a fairer place for migrants. We strive to provide tailored, more affordable financial services that make people's hard-earned money go further. Now in our 12th year, with over 370 employees in offices across Europe, we’re crafting a brilliant, relevant product that makes a difference...