Data Engineer

Role: Data Engineer (Mid / Senior) Team: Cross Software & Data Science Reports to: Head of Software & Data Science

Location:
Kraków, or remote with at least quarterly in the office for 3 days (AHM) Contract: [UoP / B2B] About Airly Airly builds and operates air quality monitoring sensors deployed across Europe and beyond. Our devices measure particulate matter and gases, and the numbers they produce are used by cities, industrial operators and infrastructure projects to make decisions and to demonstrate regulatory compliance. That last part matters: our data is audited. We are pursuing MCERTS certification, and for a given published measurement we must be able to show which raw readings and which calibration model produced it. We run our own Data Science team doing sensor calibration, model development and data quality analysis, and our own Software team building the platform, services and APIs around it. ___ Why this role exists Airly turns raw sensor readings into air quality information that people and cities act on. Between the sensor and that information sits a lot of data, and this role is about that part. Our data platform grew quickly and organically, and it now holds us back. For the right person, that’s the appeal: you get to fix real problems that have a visible impact. Here’s what’s waiting for you:
• Give our Data Scientists their time back. Right now they spend too much of it downloading, reshaping and recomputing from base data. Every hour you save goes into calibration and analysis, the work that makes our measurements more accurate.
• Make every result reproducible. Tracing a trained model back to its inputs and deployments is our biggest piece of technical debt. Reproducing a result from months ago is manual archaeology today, and that’s a compliance risk as well as a nuisance. You’ll build the lineage that lets anyone answer “where did this number come from?” in minutes.
• Turn hand-built deliverables into products. Customer data deliverables are rebuilt by hand every period. You’ll make them parameterised, repeatable and reliable.
• Own the decisions that nobody owns yet. Everyone agrees we need a warehouse for pre-computed data, a data quality registry and proper research pipelines. None of them has an owner yet. They’re waiting for you. This is Airly’s first dedicated Data Engineer role. You won’t be maintaining someone else’s platform or working around a predecessor’s architecture. You’ll decide what our data platform becomes, and you’ll grow along with it. Where you sit You’re the bridge between Data Science and Software, and you report directly to the leader of both teams. In practice, you’ll take a need from Data Science (or a business need that comes through DS) and dig in until you fully understand it. Then you’ll work with the Software team to agree on the right technical solution: code, pipelines, database schemas, contracts and APIs. You’ll own the ingestion contracts between the two sides and decide how our datasets and metrics are defined. That makes this job as much about translation and negotiation as about engineering. Both teams have strong, sometimes opposing, opinions about tooling. We’re looking for someone who enjoys that: someone who forms their own view, backs it with evidence, and is comfortable disagreeing with either side when the evidence supports it. If you want to know how a sensor reading becomes a number people trust, and you want real ownership over how that happens, we’d like to talk. What you’ll need
• 3+ years in data engineering or a closely related role
• Strong Python for data engineering, and you write tests as a habit
• Strong SQL, including analytical queries, and a good grasp of how columnar and time-series stores differ from OLTP databases
• Production experience building and running data pipelines with Airflow or a similar orchestrator
• Data modelling

skills:
schema design, evolution and versioning, and validation at ingest
• An ownership mindset: you ask good questions about requirements, then build, test, ship and run what you deliver
• Fluent Polish (our main working language) and good written English What will make you stand out
• You can make stack decisions yourself: compare options, quantify trade-offs and defend your conclusion to both Data Science and Software
• You’ve owned a data platform, or a large part of one, and you raise the standard of data work around you
• Experience with reproducibility and lineage under audit or regulatory pressure
• ClickHouse or analytical warehouse design, Kafka, and data volumes too large to move to the analyst
• MLOps tooling such as model registries and experiment tracking (you won’t build models, but you’ll make them traceable)
• Kedro or a similar pipeline framework, since we’re evaluating it and would value your opinion
• Kubernetes, Terraform, CI/CD, Grafana and InfluxDB (but not Flux); these help, and the Software team can assist with setup
• Experience with IoT, sensor or environmental data What you will own Pipelines and data modelling (primary)
• Design and build batch and streaming pipelines with schema versioning and validation, and model databases for analytical and operational needs, such as our calibration manager, so Data Science can use our distributed production databases as one coherent whole. Pre-computed datasets (primary)
• Build the curated, pre-computed datasets that serve Data Science, replacing manual recomputation from base data. Calibration pipeline and model traceability (primary)
• Automate our sensor calibration pipeline: scheduled daily execution, metrics collected on every run, and full traceability of trained models — inputs, parameters, versions and deployments. Data quality and reliability (primary)
• Data Science defines the rules, policies and thresholds. You implement them and make sure the data quality registry actually works and stays working. Discussion and refinement between you both is expected and welcome. Research pipelining for Data Science (secondary)
• Give Data Science quality-guarded, flexible pipelining for research work, split into stages in the ETL spirit: data download, normalisation, exploratory analysis, filtering, aggregation, training against multiple candidate models, metrics, comparison, and reporting. What this role is not
• Not a web-scale role. We are at terabytes, not petabytes. We are not looking for someone to run Hadoop or Spark clusters, and a stack sized for a problem we do not have would be a step backwards.
• Not a modelling role. Calibration models and data science analysis belong to the Data Science team. You make their work reproducible, automated and traceable.
• Not a ticket-taking role. If you want a backlog handed to you, this will frustrate you. The first deliverable is your own assessment of what we should build.
• Not an on-call infrastructure role. Platform infrastructure ownership stays with the Software team. Recruitment process We keep it short:
• CV review and HR call: a quick conversation to get to know each other and check that the basics fit.
• Technical conversation with the hiring manager: we’ll talk about your experience, how you approach data problems, and the challenges described above.
• Decision: we’ll let you know our decision.

How to apply
Apply through LinkedIn or send your CV to office@airly.org