Lead BigData Lead Engineer
5 dni temu
The project focuses on building a modern cloud-based data processing platform, enabling efficient management and real-time analysis of large datasets. Key aspects include implementing ETL/ELT pipelines, optimizing data processing, and ensuring compliance with industry regulations. The solution leverages Azure, Apache Spark, and Data Lakehouse architecture to support strategic business decisions.
Responsibilities:
- Engineer reliable data pipelines for sourcing, processing, distributing, and storing data in different ways, using cloud (Azure) data platform infrastructure effectively.
- Transform data into valuable insights that inform business decisions, making use of our internal data platforms and applying appropriate analytical techniques.
- Develop, train, and apply data engineering techniques to automate manual processes, and solve challenging business problems.
- Ensure the quality, security, reliability, and compliance of our solutions by applying our digital principles and implementing both functional and non-functional requirements.
- Build observability into our solutions, monitor production health, help to resolve incidents, and remediate the root cause of risks and issues.
- Understand, represent, and advocate for client needs.
- Codify best practices, methodology and share knowledge with other engineers in UBS.
- Shape the Data and Distribution architecture and technology stack within our new cloud-based datalake-house, being a hands-on contributor and senior lead in the big data and data lake space, capable to collaborate and influence architectural and design principles across batch and real-time flows.
- Have a continuous improvement mindset, always looking for ways to automate and reduce time to market for deliveries.
Mandatory Skills Description:
Your Expertise
- Experience in building Data Processing pipeline using various ETL/ELT design patterns and methodologies to Azure data solution, building solutions using ADLSv2, Azure Data factory, Databricks, Python and PySpark.
- Experience with at least one of the following technologies: Scala/Java or Python.
- Deep understanding of the software development craft, focusing on cloud-based (Azure), event-driven solutions and architectures, with key focus on Apache Spark batch and streaming, Datalakehouses using medallion architecture. Knowledge of DataMesh principles is an added plus.
- Ability to debug using tools like Ganglia UI, expertise in Optimizing Spark Jobs.
- The ability to work across structured, semi-structured, and unstructured data, extracting information and identifying linkages across disparate datasets.
- Expert in creating data structures optimized for storage and various query patterns for e.g. Parquet and Delta Lake.
- Experience in traditional data warehousing concepts (Kimball Methodology, Star Schema, SCD) / ETL tools (Azure Data factory, Informatica).
- Experience in data modelling at least one database technology such as: Traditional RDBMS (MS SQL Server, Oracle, PostgreSQL).
- Understanding of Information Security principles to ensure compliant handling and management of data.
Nice-to-Have Skills Description:
- Ability to clearly communicate complex solutions.
- Strong problem-solving and analytical skills.
- Working experience in Agile methodologies (SCRUM).
Languages:
What do we offer our employees?
- Tax-Deductible Costs on a contract of employment for all development roles.
- Stable employment based on an employment contract (or B2B).
- Special offer of banking services.
- Equipment such as a laptop and monitor.
- LuxTalent platform (webinars, training, courses with certificates).
- ...and even more
- Associate
- Full-time
- Information Technology
- Industries: IT Services and IT Consulting
-
Lead BigData Engineer
7 dni temu
Wrocław, Województwo dolnośląskie, Polska Luxoft Poland Pełny etatThe project focuses on building a modern cloud-based data processing platform, enabling efficient management and real-time analysis of large datasets. Key aspects include implementing ETL/ELT pipelines, optimizing data processing, and ensuring compliance with industry regulations. The solution leverages Azure, Apache Spark, and Data Lakehouse architecture to...
-
Data Engineer Lead
4 dni temu
Wrocław, Województwo dolnośląskie, Polska Addepto Pełny etatWe are looking for a highly skilled Data Engineer Lead to join our team at Addepto. Our company specializes in AI and Big Data, helping clients deliver innovative data projects. As a Data Engineer Lead, you will have the opportunity to work with a team of technology experts on challenging projects across various industries, leveraging cutting-edge...
-
Lead Java Software Engineer
4 dni temu
Wrocław, Województwo dolnośląskie, Polska Cyclad Pełny etatAt Cyclad, we're looking for an experienced Lead Java Software Engineer to lead our Java team. As a Lead Java Software Engineer, you will oversee the development and maintenance of our critical global component within the customer ecosystem.Key responsibilities include:Leading and mentoring junior engineers to develop high-quality Java-based solutions using...
-
Azure Data Engineer Lead
5 dni temu
Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etatWe are seeking a seasoned Azure Data Engineer Lead to lead the design and implementation of innovative data engineering solutions using Azure. The successful candidate will be responsible for developing and maintaining large-scale data processing systems, ensuring high performance, scalability, and reliability.The ideal candidate will have hands-on...
-
Lead .NET Full Stack Engineer @
3 tygodni temu
Wrocław, Województwo dolnośląskie, Polska N-iX Pełny etatWe are looking for a Lead Full Stack Engineer comfortable with C# and Angular, with a proven history of building high-performance apps. This is a great opportunity for an experienced Lead .NET Full Stack Engineer to join our super-motivated Software Development team, being part of one of our existing Scrum teams. Each Agile team includes technical...
-
Cloud-Based Data Engineer Lead
5 dni temu
Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etatAbout the RoleWe are seeking an experienced Lead Big Data Engineer to join our team. In this role, you will lead the development of cloud-based data processing platforms, enabling efficient management and real-time analysis of large datasets.Responsibilities:Data Pipeline Development: Engineer reliable data pipelines for sourcing, processing, distributing,...
-
Lead Software Engineer
2 dni temu
Wrocław, Województwo dolnośląskie, Polska JobFinder Spain Pełny etatJob SummaryAs a Lead Software Engineer (Backend), you will be responsible for leading the development of complex backend systems using Ruby on Rails and React. You will collaborate with cross-functional teams to design and implement features that meet business requirements.Key Responsibilities:Design and develop scalable backend systems using Ruby on Rails...
-
Technical Lead Data Engineer @
3 tygodni temu
Wrocław, Województwo dolnośląskie, Polska N-iX Pełny etatOur Client is a global investment advisor organization that utilizes advanced technologies and data analytics to improve the operational performance of their portfolio companies. Key objectives include delivering strong financial returns, offering diverse investment opportunities, enhancing portfolio company operations, and proactively managing risks. They...
-
Lead Cloud Data Engineer
5 dni temu
Wrocław, Województwo dolnośląskie, Polska Luxoft Poland Pełny etatJob RequirementsWe are seeking an experienced Lead Cloud Data Engineer to join our team at Luxoft Poland. In this role, you will be responsible for designing and implementing data processing pipelines using various ETL/ELT design patterns and methodologies to Azure data solution.Your expertise should include experience in building solutions using ADLSv2,...
-
AWS Cloud Engineer Lead
2 dni temu
Wrocław, Województwo dolnośląskie, Polska TN Poland Pełny etatTN Poland is a leading player in the digital transformation of the logistics industry. We are seeking an experienced AWS Cloud Engineer Lead to join our Cloud Competence Centre as a key member of our public cloud services team.ResponsibilitiesThe successful candidate will be responsible for designing, building, and managing cloud landing zone services,...
-
Software Engineering Lead
7 dni temu
Wrocław, Województwo dolnośląskie, Polska Softjourn, Inc Pełny etatAbout the Role:As a seasoned Software Engineer, you will be responsible for architecting, designing, and implementing robust full-stack applications using React on the frontend and Node.js on the backend. You will lead the technical implementation of complex features, collaborating with other team members to break down tasks into manageable pieces.About the...
-
Platform Services Technical Lead
1 dzień temu
Wrocław, Województwo dolnośląskie, Polska TN Poland Pełny etat**Job Title:** Lead Full-Stack Engineer – SmartBear Platform ServicesKey Responsibilities:Drive the future of the SmartBear Platform Services.Collaborate with cross-functional teams.Engage in dynamic technical discussions.Benefits of Working at SmartBear
-
Lead Full-Stack Engineer – Platform Services
4 dni temu
Wrocław, Województwo dolnośląskie, Polska SmartBear Software Pełny etatAt SmartBear, we deliver the complete visibility developers need to make each release better than the last. Our award-winning and industry favorite tools TestComplete, Swagger, Cucumber, ReadyAPI, and Zephyr are trusted by over 16 million developers, testers, and software engineers at 32,000+ organizations – including world-renowned innovators like Adobe,...
-
SAP LE
4 tygodni temu
Wrocław, Województwo dolnośląskie, Polska Michael Page Pełny etattechnologies-expected : SAP responsibilities : Lead the SAP LE / WM / eWM modules implementation across the organisation. Provide expert business process capability and SAP product knowledge. Collaborate with stakeholders to understand and document business requirements. Design and implement SAP solutions that meet business needs and deliver value. Conduct...
-
Lead Power BI
4 dni temu
Wrocław, Województwo dolnośląskie, Polska GlobalLogic Pełny etatLead Power BI (Business Intelligence Lead) IRC254808Job: IRC254808Location: Poland - Gdansk, Krakow, Warsaw, WroclawDesignation: Lead Software EngineerExperience: 5-10 yearsFunction: EngineeringSkills: Power BIWork Model: RemoteClient is a leading global Contract Research Organization (CRO) that provides Phase I-IV clinical trial management, market access,...
-
Data Lead Developer Snowflake
2 dni temu
Wrocław, Województwo dolnośląskie, Polska Correctcontext Pełny etatCreative Context is a global leader in media analytics, delivering accurate data to businesses across industries. As a Snowflake Lead Engineer, you'll play a key role in driving business outcomes by designing and implementing scalable data pipelines using Snowflake.We're seeking an experienced professional with expertise in Snowflake, Python, and Java to...
-
Lead Big Data Engineer
5 dni temu
Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etatLead Big Data Engineer (with Azure) - Wroclaw or KrakowLocation: Wroclaw or Krakow - Hybrid work (3 days from office)Project Description:The project focuses on building a modern cloud-based data processing platform, enabling efficient management and real-time analysis of large datasets. Key aspects include implementing ETL/ELT pipelines, optimizing data...
-
SAP LE
2 dni temu
Wrocław, Województwo dolnośląskie, Polska Michael Page Pełny etatImpactful Role in a Large OrganizationAttractive Compensation & Career GrowthAbout Our ClientOur client is a respected player in the industrial / manufacturing industry. This organisation is known for its rigorous standards, commitment to innovation, and a workforce that is dedicated to achieving the company's vision.Job DescriptionLead the SAP LE / WM / eWM...
-
Lead Big Data Engineer
1 tydzień temu
Wrocław, Województwo dolnośląskie, Polska Luxoft Pełny etatAre you ready to become one of the Luxofters?Who we are:Luxoft DXC Technology Company is an established company focusing on consulting and implementation of complex projects in the financial industry. At the interface between technology and business, we convince with our know-how, well-founded methodology and pleasure in success. As a reliable partner to our...
-
ERP Application Development Lead
19 godzin temu
Wrocław, Województwo dolnośląskie, Polska Infor Pełny etatAbout the RoleWe are seeking a skilled Software Engineer to join our team as an ERP Application Development Lead. As a key member of our team, you will be responsible for designing, developing, and documenting Infor COM application business logic and framework changes. You will work closely with Technical and Functional software engineers, Quality Assurance,...