Data engineer

6 miesięcy temu

Gdańsk, Polska b2bnetwork Pełny etat

Detailed description of work task to be carried out

 

We are seeking a skilled and motivated Data Engineer with extensive experience in Hadoop and HDFS as well as with deep understanding of relational databases to our team. In this role you will be responsible for designing, building and maintaining our data infrastructure to ensure efficient, reliable and scalable data processing and storage solutions.

You will collaborate with cross-functional teams to acquire, transform and optimize data from various sources.

The ideal candidate will have a deep understanding of big data technologies, scalable data pipelines and ensuring data quality and integrity.

 

Key Responsibilities:

  • design, develop and implement data solutions that acquire, transform and optimize data from diverse sources
  • design and build a scalable and optimize data pipelines to enable for data ingestion, transformation, and storage
  • ensure data quality and integrity by implementing data validation and cleansing processes
  • contribute to the development of data models and data architecture
  • collaborate with cross-functional teams to understand business requirements and translate them into technical solutions
  • manage and maintain HDFS for distributed storage of large datasets
  • perform data transformations and aggregations to support analytical and reporting requirements
  • identify and resolve issues related to data processing and storage

 

Tasks this person could cover:

  • Development related to provisioning data from Hadoop (either for historical FACP data provisioning or any other integration we’d like to do in the future using Hadoop as a source)
  • Ownership of data structures on ODS – design, trouble-shooting, executing changes on lower-environments (currently not centralized, different teams are responsible for creating different structures, sometimes not taking data modelling principles into consideration)
  • Data requirements and oversight on downstream integration with Hadoop and communication with GDM/Hadoop Ops

 

Description of knowledge and experience

Qualifications:

  • Bachelor’s degree in Computer Science, Information Technology or a related field. Master's degree preferred
  • Proven experience as a Data Engineer with a strong focus on Hadoop and HDFS
  • Proficiency in Hadoop ecosystem tools such as Hive, Pig, Flume, Oozie, and Sqoop
  • Experience with data processing frameworks like Apache Spark
  • Strong skills in relational databases with experience in SQL development
  • Knowledge of data warehousing concepts and database technologies
  • Excellent problem-solving skills and the ability to troubleshoot complex data issues

Req. number 36124

  • Data Engineer

    7 miesięcy temu

    Gdańsk, Polska Lyreco Group Pełny etat

    Responsible for building systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. The data engineer aims to make data accessible so that the organization can use it to evaluate and optimize its performance Are you excited for your new career adventure? At Lyreco, we offer more than...

  • Big Data Engineer Specialist

    1 miesiąc temu

    Gdańsk, Pomerania, Polska B2B S.A Pełny etat

    Big Data Engineer SpecialistAt B2B S.A, we are seeking a highly skilled Big Data Engineer to join our team. This is an exciting opportunity for a talented professional to take on a challenging role and contribute to the growth of our company.The ideal candidate will have experience in handling complex data sets and a strong understanding of big data...

  • Middle Data Engineer

    7 miesięcy temu

    Gdańsk, Polska Ciklum Pełny etat

    Description Ciklum is looking for a Middle Data Engineer to join our team full-time in Poland. We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and...

  • Data Engineer

    7 miesięcy temu

    Gdańsk, Polska Groupe SII Pełny etat

    We are looking for Data Engineer for one of our projects in the automotive industry. This person will be responsible for supporting new BI & Analytics use cases and implementation and design of state-of-the-art data processing to the Scrum working methodology. Don’t miss this exciting opportunity and apply! Your role Developing features Supporting...

  • Data Engineer Specialist

    2 tygodni temu

    Gdańsk, Polska TechTorch Pełny etat

    About TechTorch">TechTorch is a pioneer in digital transformation, leveraging AI-powered platforms and global expertise to drive business benefits for private equity sector companies.Job Overview">We are seeking an experienced Data Engineer Specialist to join our growing team. The ideal candidate will have a strong background in data engineering,...

  • Data Engineer

    5 miesięcy temu

    Gdańsk, Polska Epam Pełny etat

    Description We are seeking a Data Engineer who would like to join EPAM Data Competency Center and grow professionally by participating in a project for a technological giant. Our work involves close collaboration with clients, understanding business needs, providing top-tier engineering experiences, and applying best practices to consistently create...

  • Senior Data Engineer

    7 dni temu

    Gdańsk, Polska Jit Team Pełny etat

    Opportunity OverviewWe are seeking an experienced Senior Data Engineer to join our team in Tricity or Warsaw. As a Senior Data Engineer, you will be responsible for designing and developing scalable data solutions using Snowflake and Python.About the RoleThis is a fantastic opportunity to work with a leading financial group in the Nordic countries,...

  • Data Scientist and Engineer

    1 miesiąc temu

    Gdańsk, Pomerania, Polska Square One Resources Sp. z o.o. Pełny etat

    Square One Resources Sp. z o.o. offers a competitive salary for the position of Data Scientist and Engineer, with an estimated annual income of 120 000 PLN.About Us:Square One Resources Sp. z o.o. is a dynamic company that supports data-driven initiatives by implementing new or improving existing data pipelines. Our team focuses on extracting, transforming,...

  • Cloud Data Engineer

    4 tygodni temu

    Gdańsk, Trójmiasto, Polska Antal Sp. z o.o. Pełny etat

    Nasz Klient to globalny lider w transformacji i zarządzaniu biznesem, który wykorzystuje moc technologii do budowania zrównoważonej i inkluzywnej przyszłości. Poszukujemy doświadczonego Cloud Data Engineer, który dołączy do naszego zespołu w lokalizacjach: Wrocław, Poznań, Gdańsk, Kraków lub Warszawa.Zadania:Projektowanie, rozwijanie i...

  • Data Engineer for Cloud-Based Data Warehouse

    1 miesiąc temu

    Bernarda Chrzanowskiego , Gdańsk, Polska 7N Pełny etat

    Cloud-Based Data Warehouse Engineer OpportunityWe are seeking a highly skilled and experienced DataOps Engineer to join our team working in a global container liner shipping company. Our team specializes in building a Data Lake and leveraging the Databricks platform.About the RoleThis exciting opportunity allows you to lead the development and optimization...

  • Data Quality Assurance Engineer for ETL DW Environment

    2 tygodni temu

    Gdańsk, Polska emagine Polska Pełny etat

    We are seeking a highly skilled Data Quality Assurance Engineer to join our team in Gdańsk, Poland. As a Senior Test Automation Engineer with expertise in ETL/DW environments, you will play a crucial role in ensuring the quality and reliability of our data processing systems.Industry: BankingRate: up to 165 PLN/h netto + VATThe successful candidate will...

  • Data Engineer

    1 miesiąc temu

    Gdańsk, Trójmiasto, Polska HAYS Poland Pełny etat

    Hays IT Contracting is a cooperation based on B2B principles. Our company matches IT specialists to the most interesting technological projects on the market. Join the group of 500 satisfied Contractors and work for Hays Clients!We are currently looking for candidates for the position:Data Engineer - Snowflake SpecialistWork model: Hybrid, at least 1 day...

  • Data Engineer for Travel Innovation

    1 miesiąc temu

    Gdańsk, Pomerania, Polska VIATOR Pełny etat

    Unlock the Future of Travel with ViatorWe are seeking a skilled Data Engineer to join our team at Viator, a leading marketplace for travel experiences. As a key member of our MarTech Engineering & Operations team, you will play a crucial role in developing and maintaining the tooling used to drive millions of visitors to viator.com every month.The ideal...

  • Data Scientist and Engineer

    2 tygodni temu

    Gdańsk, Pomerania, Polska Accuris Pełny etat

    Core Intelligence Platform Data Scientist and EngineerWe are seeking an experienced Data Scientist and Engineer to join our R&D team in developing the Core Intelligence platform, a critical component of Accuris' products – Engineering Workbench, Goldfire, and Parts Intelligence. Our mission is to drive digital transformation in engineering organizations by...

  • High-End Data Engineer for Cloud-Based Data Warehouse

    3 tygodni temu

    Bernarda Chrzanowskiego , Gdańsk, Polska 7N Pełny etat

    **Job Overview:**We are seeking a highly skilled and experienced DataOps Engineer to join our team working in a global container liner shipping company. Our team specializes in building a Data Lake and leveraging the Databricks platform.This role is an exciting opportunity to lead the development and optimization of data orchestration platforms, leveraging...

  • Data Engineer Alokacje

    1 miesiąc temu

    Gdańsk, Trójmiasto, Polska Silky Coders Pełny etat

    Sprawdź swoją szafę! Na pewno masz w niej choć jeden produkt z Reserved, Mohito, Cropp, House lub Sinsay. A może to nawet dzięki nam go kupiłeś/aś? Silky Coders ubiera LPP w rozwiązania IT/e-commerce/ data science i to dzięki nam rośnie zadowolenie milionów użytkowników, a wolumen transakcji zaskakuje nas pozytywnie każdego dnia! Data...

  • Senior Data Engineer with Hadoop

    5 miesięcy temu

    Gdańsk, Polska Jit Team Pełny etat

    Salary: 1000 - 1350 PLN/MD on B2BWork model: elastic hybrid from Gdańsk  Why choose this offer?You can expect a flexible work organizationThe international work environment will give you the opportunity to interact with the English language on a daily basisScandinavian organizational culture will provide you with work-life balance, you will gain time...

  • Senior Data Engineer with GCP

    3 miesięcy temu

    Gdańsk, Polska Endava Pełny etat

    Job DescriptionOur data team has expertise across engineering, analysis, architecture, modeling, machine learning, artificial intelligence, and data science. This discipline is responsible for transforming raw data into actionable insights, building robust data infrastructures, and enabling data-driven decision-making and innovation through advanced...

  • Big Data Engineer Expert in Scalable Data Solutions

    3 tygodni temu

    Gdańsk, Pomerania, Polska Grid Dynamics Poland Pełny etat

    At Grid Dynamics, we're seeking an experienced Big Data Engineer to join our team in Poland. As a key member of our data engineering team, you will be responsible for designing and developing scalable and reliable big data solutions using Scala, Hadoop, and Spark technologies.About the JobThis role offers a unique opportunity to work on bleeding-edge...

  • Senior Azure Data Engineer

    1 miesiąc temu

    Gdańsk, Trójmiasto, Polska KPMG Pełny etat

    Company descriptionZespół Data Intelligence Solutions zajmuje się dostarczaniem Klientom usług z zakresu szeroko pojętej analityki danych, modelowania platform danych i Business Intelligence. Wykorzystujemy nowoczesne technologie Microsoft i Databricks, jednocześnie stale rozwijamy się i poszukujemy nowych sposobów, by zaspokoić wymagania...