Senior Data Engineer
2 miesięcy temu
technologies-expected :
PySpark
Terraform
Python
SQL
technologies-optional :
AWS Glue
Redshift
Databricks
about-project :
Taking technology to the next level, facing challenges, and pioneering innovations for tomorrow, we are currently seeking an experienced Data Engineer to join our client's e-commerce company
You would be a part of a remote team focused on migrating solutions from AWS Glue and Redshift to AWS Databricks. We are seeking someone with strong technical expertise, who is willing to work with Terraform (cloud infrastructure) and is proficient in PySpark for data processing.
responsibilities :
Developing a central Lakehouse data platform based on the AWS / Databricks ecosystem
Managing data infrastructure (Terraform AWS / Databricks) and ETL code (PySpark, Delta Live Tables, SparkSQL)
Implementing monitoring, data quality tests, unit tests for code, and automated alerts into the system
Refactoring older AWS solutions (AWS Glue, Redshift) into a monitored and CI/CD deployed Lakehouse ecosystem with data expectations / data quality testing
Actively supporting the construction of a Lakehouse architecture in the medal model for ML and analytical data, and introducing data-mesh concepts
Actively supporting business analytics and ML teams working on the data platform
requirements-expected :
Experience with PySpark and Terraform is necessary, even at a basic level- very important
Commercial experience in data processing using PySpark and a good understanding of the Apache Spark data processing engine architecture
Experience with infrastructure as code (Terraform) and CI/CD deployment
Some experience in building data engineering solutions (preferably in the e-commerce industry)
Excellent programming skills in Python, including: application development, automated testing, packing and deployment
Advanced SQL processing skills
Experience with Data Lakes architecture and data modeling paradigms
Experience in building and deploying ETL applications for data engineering - CI/CD pipelines in GIT environments
Fluency in English to communicate effectively both in writing and speaking
Education in Computer Science, Mathematics/Physics, Engineering, or a any related field
offered :
Access to private healthcare
Language classes in English and German at every level
Remote work opportunities and flexible working hours tailored to your schedule
Co-finance for Multi-sport card
Office located in the center of Wrocław
Regular company social events and integrations
benefits :
sharing the costs of sports activities
private medical care
remote work opportunities
flexible working time
integration events
-
Senior Data Engineer
3 miesięcy temu
Wroclaw, Polska SwingDev Pełny etatWe are a Polish branch of a fast-growing InsurTech product company from Silicon Valley - Hippo Insurance. Our mission is to revolutionize home insurance in the US, starting from IoT monitoring devices to our industry-leading software. How do we want to get there? We need a top-notch talent, just like you! Putting a lot of effort into hiring top-tier...
-
Senior Data Engineer
4 miesięcy temu
Wroclaw, Polska SHAPED THOUGHTS sp. z o.o. Pełny etattechnologies-expected : Amazon Redshift Snowflake Google BigQuery SQL Python Java technologies-optional : AWS Glue Apache Spark Apache Flink Kinesis Kafka about-project : We are looking for a Senior Data Engineer who wants to join an outstanding engineering team that will develop a cutting-edge data platform in the insurance sector. responsibilities...
-
Senior Data Engineer
4 miesięcy temu
Wroclaw, Polska Aptitude Software Pełny etatAbout us At Aptitude we love complexity. We immerse ourselves in the intricacies of finance digitization, subscription management, compliance, and revenue management which gives us the power to make a real impact. Once we understand how an organization works, we can implement software solutions that provide the clarity, confidence,...
-
Data Engineer + .NET skills
2 miesięcy temu
Wroclaw, Polska CSHARK sp. z o.o. Pełny etattechnologies-expected : Python C# .NET PySpark Spark Azure about-project : We are currently looking for a Senior Engineer experienced in the field of data engineering as well as .NET. Your responsibilities will be divided between data analysis and processing utilizing Python and PySpark technologies, and application development in C# and .NET. The project...
-
Senior Vice President, Business Intelligence
3 miesięcy temu
Wroclaw, Polska THE BANK OF NEW YORK MELLON CORPORATION Pełny etatCorporate titile: Senior Vice President, AI / Machine Learning Data Engineer BNY Mellon offers an exciting array of future-forward careers at the intersection of business, finance, and technology. We are one of the world's top asset management and banking firms that manages trillions of dollars in assets, custody and/or administration. Known as the...
-
Data Engineer
2 miesięcy temu
Wroclaw, Polska Unit8 Pełny etattechnologies-expected : Python Java C# C++ Spark about-project : These are some examples of projects that we have been working on: Building a real-time production line monitoring system for the pharmaceutical industry in order to improve the throughput of the production processes for the Pharmaceutical Industry. Technologies: AWS, EKS, CloudFormation,...
-
Senior Software Engineer
3 tygodni temu
Wroclaw, Polska SHAPED THOUGHTS sp. z o.o. Pełny etattechnologies-expected : Java Kotlin AWS SQL technologies-optional : Amazon Redshift Snowflake Google BigQuery Kinesis Kafka Terraform about-project : We are looking for a Senior Software Engineer who is hungry to deliver quality software at pace, utilise the latest technology trends and modern Software Craftsmanship techniques, and work on developing a...
-
Cloud Data Engineer
2 tygodni temu
Wroclaw, Polska Oakfusion Sp. z o.o. Pełny etattechnologies-expected : SQL Serwer AWS Power BI SQL Python responsibilities : Realizacja strategicznych inicjatyw oraz wsparcie codziennych operacji użytkowników. Tworzenie i utrzymanie systemów raportowych, w tym systemu raportowego dla regulatorów. Migracja hurtowni danych oraz procesów ETL z SQL Server do środowiska AWS. Automatyzacja raportów,...
-
Senior Data Engineer
2 miesięcy temu
Wroclaw, Polska N-iX Pełny etatThe OVO Group aims to drive progress towards net zero carbon living. We’re a collection of companies with a single vision: to power human progress with clean affordable energy for everyone. Responsibilities: A proven track record of designing, building, monitoring and managing large-scale data products, pipelines, tooling and platforms. Work in a team for...
-
Industrial Data Engineer
1 miesiąc temu
Wroclaw, Polska Alstom Pełny etatReq ID:460658 At Alstom, we understand transport networks and what moves people. From high-speed trains, metros, monorails, and trams, to turnkey systems, services, infrastructure, signalling and digital mobility, we offer our diverse customers the broadest portfolio in the industry. Every day, more than 80 000 colleagues lead the way to...
-
Senior Network Engineer
2 miesięcy temu
Wroclaw, Polska HAYS Pełny etatSenior Network EngineerWroclawNR REF.: 1187174Hays IT Contracting to współpraca oparta na zasadach B2B. Nasza firma dopasowuje specjalistów IT do najciekawszych projektów technologicznych na rynku.Dołącz do grona 500 zadowolonych Kontraktorów i pracuj dla Klientów Hays!Obecnie dla naszego Klienta poszukujemy Kandydatów na stanowisko:Starszy...
-
Data Integration Engineer
3 tygodni temu
Wroclaw, Polska ZOOLA TECH POLAND sp. z o.o. Pełny etattechnologies-expected : SQL Kafka Python Java C# technologies-optional : AWS Microsoft Azure about-project : We are seeking a skilled Data Integration Engineer to join our team for the implementation of the OneStream Enterprise Performance Management (EPM) system. The successful candidate will be responsible for designing, developing, and maintaining...
-
C++ Senior Software Engineer
1 tydzień temu
Wroclaw, Polska Nokia Pełny etattechnologies-expected : C++ Git about-project : We are seeking a highly skilled Senior SW Engineer with expertise in C++, cmake, git, and gerrit to join our innovative team at Nokia. The ideal candidate will have a passion for software development, a strong understanding of coding best practices, and the ability to work collaboratively in a fast-paced...
-
Data and AI Platform Manager
4 miesięcy temu
Wroclaw, Polska AXA Group Pełny etatData and AI Platform Manager Poland AXA XL values data and AI assets as vital for risk management and seizing new business opportunities. They need to be high quality and delivered in a value-first rapid fashion, providing competitive edge while boosting company operational efficiency. The Innovation, Data, and Analytics (IDA) team delivers on this...
-
Senior Java Engineer
2 tygodni temu
Wroclaw, Polska ZOOLA TECH POLAND sp. z o.o. Pełny etattechnologies-expected : Java Spring Boot PostgreSQL technologies-optional : AWS Kafka Rabbit Docker Kubernetes about-project : As a Software Engineer, you'll play a pivotal role in building new generation of our client’s product - contribute to development of its core/platform services that are used by other business applications. Leveraging your 7+...
-
Senior QA Engineer
1 miesiąc temu
Wroclaw, Polska ZOOLA TECH POLAND sp. z o.o. Pełny etatabout-project : Together with the Zoolatech team, our partner, a leading construction management company, is at the forefront of developing a cloud-based software solution. This innovative platform is instrumental in the successful completion of major projects including skyscrapers, hospitals, data centers, stadiums, and schools. It streamlines the...
-
Senior Mainframe Systems Engineer
4 miesięcy temu
Wroclaw, Polska THE BANK OF NEW YORK MELLON CORPORATION Pełny etatSenior Mainframe Systems Engineer (CICS) Bring your ideas. Make history. BNY Mellon offers an exciting array of future-forward careers at the intersection of business, finance, and technology. We are one of the world's top asset management and banking firms that manages trillions of dollars in assets, custody and/or administration. Known as the...
-
Senior Security Technical Specialist – Data Security
1 dzień temu
Wroclaw, Polska HAYS Pełny etatSenior Security Technical Specialist – Data SecurityWroclawNR REF.: 1188694OFFICE LOCATION: WROCŁAWWORK MODEL: HYBRID (50% REMOTE)CONTRACT: CONTRACT OF EMPLOYMENTREPORTING TO: HEAD OF DATA SECURITY WITHIN INFORMATION SECURITY OPERATIONSOur Client has an exciting opportunity for a Senior Security Technical Specialist to join the Data Security team within...
-
C++ Senior Software Engineer
4 tygodni temu
Wroclaw, Polska Nokia Pełny etatWe are seeking a highly skilled Senior SW Engineer with expertise in C++, cmake, git, and gerrit to join our innovative team at Nokia. The ideal candidate will have a passion for software development, a strong understanding of coding best practices, and the ability to work collaboratively in a fast-paced environment. If you are a talented engineer looking to...
-
Senior Java Engineer
3 tygodni temu
Wroclaw, Polska ZOOLA TECH POLAND sp. z o.o. Pełny etattechnologies-expected : Java Spring Boot PostgreSQL Docker Kubernetes GitHub ArgoCD technologies-optional : Kafka RabbitMQ AWS about-project : At Zoolatech, we're dedicated to transforming the business landscape with our comprehensive expertise in software development. Our collaboration with our client, a top-notch supplier of quality and compliance...