Data Engineer with Expertise in Java and Hadoop

1 dzień temu


Warszawa, Mazovia, Polska DataArt Pełny etat
Job Title: Data Engineer with Expertise in Groovy and Spark

About the Role

We are seeking a highly skilled Data Engineer with expertise in Groovy and Spark to join our team at DataArt. As a Data Engineer, you will be responsible for designing and building efficient data pipelines, creating and maintaining algorithms and data processing code, and implementing processes for data validation, cleansing, and transformation.

Key Responsibilities

  • Collaborate with data engineers, product teams, and stakeholders to gather data requirements and design efficient data pipelines
  • Develop and maintain algorithms and data processing code in Java/Groovy
  • Implement data validation, cleansing, and transformation processes to ensure data accuracy and consistency
  • Develop Python scripts to automate data extraction from new and existing sources
  • Monitor and troubleshoot data pipeline performance in Airflow, addressing any issues or bottlenecks
  • Write SQL queries to extract data from BigQuery and develop reports using Google's Looker Studio
  • Participate in daily stand-ups, sprint planning, and retrospective meetings
  • Engage in peer code reviews, knowledge sharing, and assist other engineers with their work
  • Introduce new technologies and best practices to keep the product up to date
  • Assist in troubleshooting and resolving production escalations and issues

Requirements

  • Bachelor's degree or equivalent programming experience
  • 4-5 years of overall experience as a backend software developer, with at least 2 years as a Data Engineer using Spark with Java/Groovy and/or Python
  • Strong coding skills, knowledge of data structures, OOP principles, databases, and API design
  • Highly proficient in developing programs and data pipelines in Java/Groovy or Python
  • 2+ years of professional experience with Apache Spark/Hadoop

Nice to Have

  • Work experience with AWS (EMR, S3, lambda, EC2, glue, RDS)
  • Work experience with SQL (MYSQL is a Plus) and NoSQL Databases
  • Experience with Elasticsearch
  • Experience with Python
  • Experience with Scala (Zeppelin)
  • Experience with Airflow or other ETL
  • Certification or verified training in one or more of the following technologies/products: AWS, ElasticSearch


  • Warszawa, Mazovia, Polska emagine Consulting Pełny etat

    Job Title: Data Engineer with Java or ScalaJob Description:We are seeking a skilled data engineer with a strong software engineering background to help our clients build robust data solutions and drive innovation within our organisation.Responsibilities:Design and develop data pipelines and architectures using Java or Scala.Develop prototypes and proof of...

  • Data Engineer

    1 dzień temu


    Warszawa, Mazovia, Polska Addepto Pełny etat

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Addepto, a leading consulting and technology company specializing in AI and Big Data. As a Data Engineer, you will play a key role in designing and developing scalable data management architectures, infrastructure, and platform solutions for streaming and batch processing using...


  • Warszawa, Mazovia, Polska Link Group Pełny etat

    Key Responsibilities:We are seeking a highly skilled Backend Engineer to join our team and contribute to the development of our EV charger project.Backend Development: Design, develop, and maintain backend systems using Java, ensuring high-quality software solutions.Frontend Contribution: Occasionally contribute to frontend development using Angular,...

  • Data Engineer

    1 dzień temu


    Warszawa, Mazovia, Polska Devoteam Pełny etat

    Job Title: Data Services EngineerDevoteam is seeking a highly motivated Data Services Engineer to join our team. As a Data Services Engineer, you will be responsible for designing and implementing data pipelines and architecture using Google Cloud Platform (GCP) products.Key Responsibilities:Design and construct data pipelines and architecture using GCP...


  • Warszawa, Mazovia, Polska Infotree Global Solutions Pełny etat

    Commercial Real Estate Lending Portfolio Improvement ProjectWe are seeking a highly skilled Fullstack Java/React Developer to join our team at Infotree Global Solutions in Warsaw. The successful candidate will work on improving credit risk management capabilities for our Commercial Real Estate lending portfolio.Key Responsibilities:Design and implement ETL...

  • Data Engineer

    1 dzień temu


    Warszawa, Mazovia, Polska b2bnetwork Pełny etat

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at b2bnetwork. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data processing systems.Key Responsibilities:Design and implement distributed data processing pipelines using Spark and HadoopDevelop and maintain data storage...

  • Data Engineer

    11 godzin temu


    Warszawa, Mazovia, Polska b2bnetwork Pełny etat

    Job Title: Engineer ITWe are seeking a highly skilled Engineer IT to join our team at b2bnetwork. The successful candidate will be responsible for designing and implementing data processing pipelines using Spark and Scala.Key Responsibilities:Design and implement distributed data processing pipelines using Spark and ScalaWork closely with DevOps, QA, and...

  • Data Engineer

    11 godzin temu


    Warszawa, Mazovia, Polska Addepto Pełny etat

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Addepto, a leading consulting and technology company specializing in AI and Big Data. As a Data Engineer, you will play a key role in designing and developing scalable data management architectures, infrastructure, and platform solutions for streaming and batch processing using...


  • Warszawa, Mazovia, Polska b2bnetwork Pełny etat

    Job Title: ACC - SA:GFCP SD: IIS / Europe / FCAD Data Analyst with Data Modelling experienceWe are seeking a skilled Data Analyst with Data Modelling experience to join our Data team at Klient. As a key member of the team, you will be responsible for securing the data requirements in relation to the introduction of a new generation Financial Crime Prevention...

  • Data Engineer

    1 dzień temu


    Warszawa, Mazovia, Polska Datumo Pełny etat

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Datumo. As a Data Engineer, you will play a key role in designing and implementing scalable data systems, leveraging your expertise in Big Data and cloud technologies.Key ResponsibilitiesDesign and implement Big Data systems using cloud providers such as GCP, Azure, or AWSDevelop...

  • Data Engineer

    1 dzień temu


    Warszawa, Mazovia, Polska Datumo Pełny etat

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Datumo. As a Data Engineer, you will play a key role in designing and implementing scalable data systems, leveraging your expertise in Big Data and cloud technologies.Key ResponsibilitiesDesign and implement Big Data systems using cloud providers such as GCP, Azure, or AWSDevelop...


  • Warszawa, Mazovia, Polska AIDA projektai, MB Pełny etat

    Software Developer with Java and C ExpertiseWe are seeking a skilled software developer with expertise in both Java and C to play a key role in rewriting a software component from C to Java. The successful candidate will be involved in all phases of the development lifecycle, writing efficient, testable code, and actively supporting continuous improvement...

  • Data Engineer

    1 dzień temu


    Warszawa, Mazovia, Polska Sanoma Pełny etat

    Job DescriptionSanoma Learning is seeking a highly skilled Data Engineer to join our Technology organization. As a key member of our team, you will play a crucial role in the development and maintenance of our streaming data platform.Key Responsibilities:Design, develop, and manage Kafka-based data pipelinesWork with other big data technologies such as...


  • Warszawa, Mazovia, Polska CodiLime Pełny etat

    Your RoleAs a key member of our project team, you will be responsible for designing and implementing features in GCP's BigQuery and Cloud Functions using Python. Your expertise will be crucial in developing solutions that cater to both OLTP and OLAP approaches.• Design and implement features in GCP's BigQuery and Cloud Functions using Python• Develop...


  • Warszawa, Mazovia, Polska Nordea Bank Pełny etat

    About this OpportunityWelcome to the international team of test automation enthusiasts at Nordea Bank. We're looking for a skilled Senior Test Automation Engineer with Java expertise to join our Cards Technology team.Key ResponsibilitiesDesign and implement automated test cases using Java and Selenium frameworkEliminate manual testing and trivial activities,...

  • Senior Java Developer

    1 dzień temu


    Warszawa, Mazovia, Polska Infotree Global Solutions Pełny etat

    Senior Java EngineerWe're seeking a highly skilled Senior Java Engineer to join our dynamic team at Infotree Global Solutions. As a key member of our engineering team, you will play a crucial role in designing and developing new software for regulatory reporting, ensuring accuracy and efficiency in meeting global regulatory requirements.Key...


  • Warszawa, Mazovia, Polska Square One Resources Sp. z o.o. Pełny etat

    Unlock Strategic Decision-Making with Data Engineering ExpertiseThis innovative solution empowers organizations to make informed decisions by providing a robust data foundation. By harmonizing data from diverse sources and formats, our Data Engineer with Snowflake and DBT expertise ensures seamless data usability and value.Your ResponsibilitiesHarmonize and...


  • Warszawa, Mazovia, Polska Square One Resources Sp. z o.o. Pełny etat

    We are seeking an experienced Fullstack Developer to support the development and maintenance of key application components. The ideal candidate will have a strong background in Java and Java EE development, including frameworks, and proficiency with Angular 2+ and Typescript. Additionally, they should have hands-on experience with Node.js, npm, and...

  • Data Engineer

    1 dzień temu


    Warszawa, Mazovia, Polska Infotree Global Solutions Pełny etat

    Data Engineer - ETL Pipelines and Data ProductsWe are seeking a skilled Data Engineer with a strong background in building efficient, high-quality, and scalable ETL data pipelines and data-oriented products. This role requires deep expertise in parallel computing, SQL optimization, and a solid understanding of analytical and problem-solving techniques.Key...


  • Warszawa, Mazovia, Polska Eviden Pełny etat

    About the RoleEviden, a global leader in data-driven digital transformation, is seeking a skilled Java Developer with expertise in Apache Kafka to contribute to the development of a large-scale EU project.Key ResponsibilitiesDesign and implement software applications using Java 11Develop and integrate technological components with KafkaImplement user...