Big Data Pipeline Specialist
5 dni temu
Job Summary
Axiom Software Solutions Limited seeks a skilled Data Engineer to design, develop, and maintain big data pipelines using Python and PySpark.
Responsibilities
- Develop and maintain ETL pipelines using Python and PySpark to process large datasets efficiently.
- Work with the Cloudera Hadoop ecosystem to manage and process big data.
- Ensure data integrity, performance, and reliability across distributed systems.
- Collaborate with data scientists, analysts, and business stakeholders to deliver data-driven solutions.
- Implement best practices for data governance, security, and performance tuning.
- Use Git and GitHub for version control and efficient code collaboration.
- Track and manage tasks using Jira, and document processes in Confluence.
- (Optional) Work with Databricks for cloud-based big data processing.
Requirements
- Strong programming skills in Python.
- Hands-on experience with PySpark for distributed data processing.
- Expertise in Cloudera Hadoop ecosystem (HDFS, Hive, Impala).
- Experience with SQL and working with large datasets.
- Knowledge of Git and GitHub for source code management.
- Experience with Jira for task tracking and Confluence for documentation.
- Strong problem-solving and analytical skills.
-
Big Data Technology Expert
4 dni temu
Warszawa, Mazovia, Polska Grid Dynamics International, Inc. Pełny etatWe are looking for an experienced Big Data Engineer to join our team at Grid Dynamics International, Inc.Job DescriptionIn this project, we are working on the bleeding edge of Big Data technology to develop a high-performance data analytics platform, which handles petabytes datasets.The ideal candidate will have solid programming skills in Python and...
-
Big Data Expert
4 dni temu
Warszawa, Mazovia, Polska Infinity Quest Pełny etatWe are seeking a talented Big Data Expert to lead our data science initiatives at Infinity Quest.The successful candidate will have a deep understanding of the Hadoop ecosystem and a proven track record of delivering complex data projects.A minimum of 5 years of experience with development using Python is required.Strong knowledge of database technologies,...
-
Technical Lead for Big Data Solutions
2 dni temu
Warszawa, Mazovia, Polska Infosys Pełny etatWe are searching for an experienced Technical Lead for Big Data Solutions to join our team at Infosys.Talent Acquisition Executive (UK & Europe) at Ampstek, UKThe ideal candidate will have leadership experience in designing and implementing big data architectures.Experience with Hadoop and Cloudera distribution is essential.Key ResponsibilitiesLeading big...
-
Big Data Developer @
3 tygodni temu
Warszawa, Mazovia, Polska Sii Polska Pełny etatJoin a new team of Engineers at one of the largest financial institutions in the world as a Big Data Specialist. You will work in the area of traded risk on a strategic project to deliver a strategic aggregation platform for Risk managers and utilize a new vendor product in a cloud and on-premises hybrid environment.- At least 5 years of experience as a...
-
Cloud Native Data Pipeline Developer
18 godzin temu
Warszawa, Mazovia, Polska Cubestech Ltd Pełny etatCubestech Ltd is looking for a Big Data Engineering Expert to join our team. As a Senior Data Engineering Specialist, you will be responsible for designing and implementing scalable data pipelines using Hadoop and Spark. Your expertise in big data engineering will help us build a robust and efficient data architecture.Job RequirementsDesign and develop...
-
Warszawa, Mazovia, Polska SquareOne Pełny etatJob DescriptionWe're seeking an experienced Data Pipeline Specialist to drive forward our Remote Monitoring Patient Platform (RPMP). As a key member of our team, you'll be responsible for developing and maintaining scalable data pipelines using Python and leveraging Microsoft Azure services for efficient data storage, processing, and deployment.
-
Big Data Scientist and Software Engineer
15 godzin temu
Warszawa, Mazovia, Polska Tietoevry Pełny etatTechnical RequirementsWe are seeking a highly skilled Senior Software Engineer-Machine Learning Engineer/Data Engineer to join our team in Warsaw. As a senior engineer, you will play a crucial role in designing, developing, and deploying machine learning models using Python frameworks.Develop and maintain ETL/ELT pipelines using Python and SQLWork with...
-
Big Data Expert with Hadoop Skills
2 dni temu
Warszawa, Mazovia, Polska Infosys Pełny etatWe are seeking an experienced Big Data Expert with Hadoop Skills to join our team at Infosys.Talent Acquisition Executive (UK & Europe) at Ampstek, UKThe successful candidate will have hands-on experience with Hadoop and be able to design and implement complex big data solutions.Strong understanding of relevant tools and technology is essential.Key...
-
Big Data Analyst and Engineer
2 dni temu
Warszawa, Mazovia, Polska OPTIVEUM sp. z o.o. Pełny etatAbout OPTIVEUM sp. z o.o.We are a forward-thinking SaaS provider driving innovation in the insurance technology space, renowned for transformative software solutions enhancing critical functions like policy management, claims, and reinsurance.Our team is growing, and we're looking for a talented Big Data Analyst and Engineer to join our dynamic...
-
Data Pipeline Specialist Opportunity
4 dni temu
Warszawa, Mazovia, Polska TN Poland Pełny etatSunscrapers empowers finance and healthcare leaders to succeed through cutting-edge software, data, and AI.We specialize in software engineering, DevOps, data engineering, and data science to deliver innovative solutions. Our team designs and builds AI-powered data platforms and web applications tailored to each client's needs.About the CompanySunscrapers is...
-
Big Data Engineering Expert
18 godzin temu
Warszawa, Mazovia, Polska Cubestech Ltd Pełny etatWe are seeking a Senior Data Engineering Specialist to join our team at Cubestech Ltd. As a Senior Data Engineering Specialist, you will design and implement scalable data pipelines using AWS services such as S3, Lambda, and Glue. Your expertise in cloud-based data engineering will help us build a robust and efficient data architecture.Job DescriptionDesign...
-
Data Pipeline Expert
2 dni temu
Warszawa, Mazovia, Polska Talan Pełny etatWe are seeking an experienced Data Engineer to join our team in Warsaw, Poland. As a key member of our organization, you will contribute to the development of cloud-scale data processing solutions using Python and related technologies.The ideal candidate will have strong programming skills in Python, hands-on experience with PySpark for distributed data...
-
Big Data Processing Expert
5 dni temu
Warszawa, Mazovia, Polska brabers Pełny etatBrabers is seeking a seasoned Data Engineer to join our team and lead the development of our data systems and applications. As a Data Engineer, you will be responsible for designing and implementing data pipelines in public clouds like AWS, as well as participating in data migration from on-premise to public clouds.This position requires advanced technical...
-
Data Pipeline Architect
4 dni temu
Warszawa, Mazovia, Polska JSS ASSOCIATES Pełny etatJSS Associates seeks a highly skilled Data Engineer to lead the development of data pipelines and transformation processes.About the JobThe ideal candidate will have strong expertise in AWS Cloud services, Python, PySpark, and Databricks, with a proven track record of building and optimizing complex data processing systems.Main Responsibilities:Data Pipeline...
-
Big Data Engineering Specialist
4 dni temu
Warszawa, Mazovia, Polska JSS ASSOCIATES Pełny etatJSS Associates is hiring a Data Engineer to build and maintain complex data pipelines and transformation processes.About the JobThe ideal candidate will have strong expertise in AWS Cloud services, Python, PySpark, and Databricks, with a focus on building scalable and efficient data processing systems.Main Responsibilities:Data Pipeline Optimization: Utilize...
-
Big Data Professional
7 dni temu
Warszawa, Mazovia, Polska Tietoevry Pełny etatTietoevry is seeking a talented Big Data Professional to join our team. As a key member of our data engineering team, you'll be responsible for designing, developing, and maintaining ETL/ELT pipelines using Python and SQL. You'll also work with Hadoop, Spark, and Big Data technologies for large-scale data processing.We're looking for someone with a strong...
-
Big Data Pipeline Architect
4 dni temu
Warszawa, Mazovia, Polska TN Poland Pełny etatAbout the RoleWe are looking for an experienced Data Engineer to join our team. The successful candidate will have a strong background in data engineering and be able to manage system requirements, design, development, integration, quality assurance, implementation, and maintenance of corporate applications.Main Accountabilities:Work with product owners,...
-
Big Data Engineer
8 godzin temu
Warszawa, Mazovia, Polska Capgemini Pełny etatCapgemini is seeking a talented Big Data Engineer to join our team. As a Big Data Engineer, you will design, implement, and maintain big data platforms, data architecture, and design techniques to work with large data sets. You will be empowered to shape your career in the way you'd like and supported by a collaborative community of colleagues around the...
-
Big Data Pipeline Developer
2 dni temu
Warszawa, Mazovia, Polska Samba TV, Inc. Pełny etatAbout the JobSamba TV, Inc. is a pioneering media and advertising company that's transforming the viewing experience with its proprietary data and technology.We're seeking a talented software engineer to join our team and contribute to the design and architecture of our data platform.This role involves developing new data pipelines, automating business...
-
Big Data Engineering Manager
16 godzin temu
Warszawa, Mazovia, Polska Cyclad Pełny etatKey ResponsibilitiesThe successful candidate will have the following responsibilities:Oversee the design and development of data engineering solutions, ensuring quality and efficiency.Lead the development and implementation of data governance strategies, including data quality, master data management, data cataloging, and lineage.Represent Cyclad as a...