Data Engineer for Large-Scale Data Pipeline Development
16 godzin temu
About Us
N-iX is a US-based B2B marketplace company that relies on efficient data systems to drive business decisions.
Job Description
We are seeking an experienced Data Engineer to design, develop, and optimize large-scale data pipelines within our AWS ecosystem. The ideal candidate will possess strong expertise in PySpark and have hands-on experience with AWS services such as S3, Glue, EMR, Lambda, and Redshift.
Key Responsibilities:
- Data Pipeline Development: Design, develop, and implement scalable data pipelines using PySpark within AWS environments.
- AWS Services Utilization: Leverage AWS services such as S3, Glue, EMR, Lambda, and Redshift to build robust data solutions.
- Performance Optimization: Optimize PySpark workflows for performance, reliability, and cost-efficiency.
- Collaboration and Communication: Work closely with stakeholders to understand data requirements and translate them into technical solutions.
- Data Quality Assurance: Ensure data quality and integrity through robust testing and monitoring processes.
- Data Governance and Compliance: Implement data governance, security, and compliance best practices in all development activities.
- Documentation and Knowledge Sharing: Document technical designs, processes, and workflows to support ongoing maintenance and team knowledge sharing.
Requirements
- Education: Bachelor's degree in Computer Science, Engineering, or a related field.
- Experience: 4+ years of experience in data engineering with a focus on building and optimizing data pipelines using PySpark.
- AWS Services Expertise: Strong experience with AWS services, including S3, Glue, Lambda, EMR, and Redshift.
- Programming Skills: Proficiency in Python programming and familiarity with related frameworks and libraries.
- Distributed Computing: Solid understanding of distributed computing and experience with Apache Spark.
- Infrastructure-as-Code Tools: Hands-on experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation) is a plus.
- Analytical and Problem-Solving Skills: Strong analytical and problem-solving skills, with attention to detail and a proactive approach to troubleshooting.
- Communication and Collaboration: Excellent communication and collaboration skills, with the ability to work in a dynamic, team-oriented environment.
- Language Proficiency: Upper-Intermediate level of English.
-
Kraków, Polska N-iX Pełny etatWe are seeking a skilled Middle Power BI Engineer to join our team at N-iX in Poland. Our customer is one of the world's largest food and beverage companies, operating in 189 countries with over 2000 brands.Main Responsibilities:Design and develop optimized business intelligence solutions focused on performance and scalabilityPresent complex data insights...
-
Data Engineer for Scalable Data Solutions
3 dni temu
Kraków, Polska Capco Poland Pełny etatAt Capco Poland, we're passionate about driving digital transformation across the financial services industry.Job DescriptionWe're seeking a seasoned Senior Big Data Engineer to join our team. The ideal candidate will be responsible for designing, implementing, and maintaining scalable data pipelines and solutions on-premises, in migration, or cloud projects...
-
Data Architect for Data Pipeline Optimization
2 tygodni temu
Przy Rondzie , Kraków, Polska Tesco Technology Pełny etatData Engineer FulfillmentEstimated Salary: 120,000 - 180,000 per annum (based on Kraków location)Company OverviewTesco Technology is a forward-thinking team driving operational excellence in services and systems globally. Our agile approach enables us to create innovative solutions that improve scalability and business capabilities.Job DescriptionWe are...
-
Data Engineer Specialist for Financial Services
3 tygodni temu
Kraków, Polska Capco Poland Pełny etatAre you a skilled Data Engineer looking to drive digital transformation in the financial services industry? Capco Poland, a global technology and management consultancy, is seeking an experienced Senior Big Data Engineer to join our team.We're passionate about helping our clients succeed in an ever-changing industry. Our ideal candidate will design,...
-
Data Engineer
1 miesiąc temu
Kraków, małopolskie, Polska INEXTO POLAND Pełny etatAre you passionate about cutting-edge technology and innovative solutions? INEXTO, a global leader in Authentication, Secure Serialization, Track & Trace, and Production Volume Verification, is looking for a Data Engineer to join our team. Our technology secures over 100 billion products annually, and we're just getting started!At INEXTO, you'll be...
-
Senior Data Engineer
1 tydzień temu
Kraków, małopolskie, Polska IT-Total Development Pełny etatOverview IT-Total, is looking for a passionate and experienced Data Engineer that will play also role as Lead Data Engineer. The focus is on delivering data centric solutions with a cloud first mindset.You will be expected to be the technology wizard and deliver high quality and innovative solutions for our projects and product departments. You will also...
-
Data Solutions Engineer
3 tygodni temu
Kraków, Polska Kitopi Pełny etatUnlocking Data-Driven Growth at KitopiWe're seeking a seasoned Data Solutions Engineer to join our team in Kraków, Poland. As a key member of our data organization, you'll play a crucial role in designing and implementing scalable data solutions that drive business growth and innovation.At Kitopi, we're revolutionizing the food industry with cutting-edge...
-
Data Engineering Expert for Scalable Data Solutions
2 tygodni temu
Zabłocie A, -, Kraków, Polska N-iX Pełny etatAs a Data Engineer at N-iX, you will play a crucial role in designing and developing high-performance data pipelines within an AWS ecosystem.We are seeking an experienced Data Engineer to join our Development Team. Your expertise in PySpark will be instrumental in processing large-scale datasets, ensuring the reliability and performance of our data...
-
Data Engineer Fulfillment Transformation Specialist
2 tygodni temu
Kraków, Polska Tesco Technology Pełny etatTransforming Business Through Data ExcellenceTesco Technology is a cutting-edge team that drives operational excellence and scales services globally, creating business leading capabilities.We are an agile and industry-leading team of engineers, creating continuous integration and delivery tools for Colleague and Customer & Loyalty areas. Our solutions have a...
-
Data Engineer for Banking Projects
4 dni temu
Kraków, Lesser Poland emagine Polska Pełny etatCompany OverviewWe are Emagine Polska, a company dedicated to providing innovative solutions in the banking sector. Our team is passionate about data analysis and management.Job DescriptionWe are seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for managing and processing large datasets, building ETL...
-
Senior Power BI Data Architect
2 tygodni temu
Zabłocie A, -, Kraków, Polska N-iX Pełny etatPower BI Engineer Job OpportunityWe are looking for a skilled Power BI Engineer to join our team in Poland. Our client is one of the world's largest food and beverage companies, operating globally with over 2000 brands.Main Responsibilities:Design, develop, and maintain high-performance business intelligence solutionsCreate interactive reports and complex...
-
Data Engineer for Cloud Data Platforms
3 dni temu
Zabłocie A, -, Kraków, Polska N-iX Pełny etatExperience the thrill of working on a robust data pipeline that empowers our SaaS product to contextualize, validate, and ingest customer data seamlessly. Collaborate with product teams to unlock new user experiences by leveraging insightful data analysis.N-iX was established to fundamentally transform the execution of capital projects and operations. Our...
-
Senior Software Engineer ML Platform
1 dzień temu
Kraków, małopolskie, Polska Speechify Inc. Pełny etatMissionSpeechify is the easiest way to listen to the world’s information. Articles on the web, documents in the cloud, books on your phone. We absorb it all and let you listen to it at your desk, on the go, at your own speed, and with tools that make learning easier, deeper, and faster.What streaming services have done for audio entertainment, we’re...
-
System Architect for Large-Scale BMC Discovery Deployments
3 tygodni temu
Kraków, Lesser Poland Team Up Pełny etat**Job Summary:**We are seeking an experienced System Architect to join our team at Team Up. As a key member of our IT services division, you will play a crucial role in designing and implementing large-scale BMC Discovery systems.About the Role:The ideal candidate will have a strong background in system architecture, with a focus on BMC Discovery...
-
Data Engineering Leader for Global Projects
4 dni temu
Kraków, Polska DCG Pełny etatCompany OverviewDCG is a recruitment company that understands the importance of experienced professionals in driving business success. Our management style and partnership approach enable us to deliver personalized support to our clients.Data Engineering Leader RoleWe are currently seeking an experienced Data Engineering Leader to join our team. As a key...
-
Senior Data Engineer Python
1 miesiąc temu
Kraków, małopolskie, Polska Capco Poland Pełny etatoining Capco means joining an organisation that is committed to an inclusive working environment where you’re encouraged to #BeYourselfAtWork. We celebrate individuality and recognize that diversity and inclusion, in all forms, is critical to success. It’s important to us that we recruit and develop as diverse a range of talent as we can and we believe...
-
Senior Cloud Data Engineer
3 tygodni temu
Kraków, Lesser Poland Michael Page International (Poland) Sp. z o.o. Pełny etatAbout Michael Page International (Poland) Sp. z o.o.We are seeking an experienced Senior Cloud Data Engineer to join our team in Kraków.Salary and BenefitsWe offer a competitive salary of approximately 65,000 PLN per month, annual bonus, and benefits package.Additionally, we provide a hybrid working model (3 days per week in the office in Kraków).You will...
-
Data Scientist
1 miesiąc temu
Kraków, małopolskie, Polska IT-Total Development Pełny etatPosition Overview IT-Total, is looking for a passionate and experienced Data Scientist that will play also role as Data Engineer. The focus is on delivering data centric solutions with a cloud first mindset.You will be expected to be the technology wizard and deliver high quality and innovative solutions for our projects and product departments. You will...
-
Senior Data Engineer
16 godzin temu
Kraków, Polska emagine Polska Pełny etatAbout the RoleWe are seeking an experienced Senior Data Engineer to join our team at emagine Polska. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data processing systems on Google Cloud Platform.Key ResponsibilitiesDesign and implement scalable data pipelines using BigQuery,...
-
Data Engineer for Big Data Solutions
2 dni temu
Kraków, Polska Grape Up Pełny etatAt Grape Up, we are committed to leveraging technology to drive software innovation. Our mission is to build cutting-edge solutions that transform the way businesses operate.We are currently seeking a highly skilled Data Engineer to join our team and help us develop and maintain a robust Big Data system deployed on-premises.About the RoleData Engineering...