Senior Data Engineer

Senior Data Engineer / ML Miejsce pracy: Warszawa Technologies we use Expected Python Spark S3 Glue SageMaker Lambda AWS SQL Bitbucket Git Optional TensorFlow PyTorch About the project We are looking for an MLOps/ML Engineer responsible for the end-to-end lifecycle of machine learning models - from development and deployment to automation and performance monitoring, leveraging AWS, Spark, and Python. The role includes building scalable, distributed data processing pipelines in a big data environment, collaborating with data scientists and analysts, and enhancing MLOps best practices across the organization. Experience in designing and deploying predictive and analytical models is a strong plus. Your responsibilities Managed the lifecycle of machine learning models from development to deployment and monitoring, ensuring optimal performance and reliability. Implemented MLOps principles, including continuous integration, continuous delivery, testing, and monitoring, to streamline machine learning operations. Working with Spark & Python to define and maintain data ingestions and transformation, handling both real-time and batch data processing. Building distributed and highly parallelized big data processing pipeline which process massive amount of data (both structured and unstructured data) in near real-time. Leverage Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analytics. Collaborated with cross-functional teams, including data scientists, DevOps engineers, and IT, to deliver high-quality machine learning solutions. Working closely with analysts and business stakeholders to develop analytics models. Help in optimizing our current MLOps practices and libraries, make applicable for wider range of use cases outside propensity models. Cloud solution exploration for AI/ML areas. Our requirements Proficiency in programming languages like Python & Spark - minimum 5 years. Hands-on experience with AWS services for machine learning and data processing - particulary S3, Glue, SageMaker, Lambda and StepFunctions/Airflow/MWAA - covering data ingestion, model training, batch inference and automation pipelines. Practical understanding of AWS Infrastracture setup and automation using AWS CLI, boto3 and IAM roles, ensuring reproducible and secure ML workflows. Understanding of algorithms and data structures, knowledge of statistics and linear algebra. Experience with machine learning frameworks (TensorFlow or PyTorch), including training new ML models, refining existing ones, and deploying these models into user-friendly applications. Solid understanding of distributed systems with experience in Hadoop/Hive ecosystem and/or other Big Data Technologies. Proficient in SQL (Spark/Hive SQL), previous experience in creating data flows. Experience with BitBucket and GIT, code versioning and branching strategy. Familiar with Agile/Safe framework. Optional Design and implementation ML Models (e.g. Propensity, Customer Lifetime Value, Micro-segmentation, channel recommendation models), helping use case team(s) in training and deployments. Recruitment stages Rozmowa z rekruterem Rozmowa z klientem B2B.NET S.A. B2Bnetwork jest polską firmą, specjalizującą się w obszarze Quality Assurance oraz Outsourcingu specjalistów IT. Od 2015r. zarządzamy w całości procesami testowymi u naszych klientów oraz wspieramy w znalezieniu odpowiednich kompetencji w innych obszarach IT. Aplikując, wyraża Pani/Pan zgodę na przetwarzanie przez B2BNetwork swoich danych osobowych zawartych w niniejszym kwestionariuszu aplikacyjnym i aplikacji na potrzeby rekrutacji. Jednocześnie oświadczasz, że podanie przez Panią/Pana danych wykraczających poza katalog określony w Kodeksie pracy i innych właściwych ustaw jest dobrowolne i dla Pani/Pana korzystne.