Data Engineer
Postaladdress
Charing Cross, United Kingdom
2 days ago
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
EnglishJob location
Charing Cross, United Kingdom
Tech stack
Artificial Intelligence
Data analysis
Azure
Clinical Data Repository
Data Cleansing
ETL
EHealth
Python
SQL Databases
Feature Engineering
Data Ingestion
Azure
Fast Healthcare Interoperability Resources
Spark
Data Lake
Star Schema
Health Level Seven International
Machine Learning Operations
Data Pipelines
Databricks
Job description
The RoleAs a Data Engineer, you will build, optimise, and maintain robust end-to-end data pipelines and platforms on Microsoft Foundry, Azure, and Databricks to enable analytics, AI, and digital healthcare initiatives., * Develop and maintain production-grade data pipelines using Microsoft Foundry, Azure Data Factory, ADLS, Synapse, and Databricks (Spark & Delta Lake).
- Implement ETL/ELT processes, data ingestion, transformation, and loading with Python, SQL, and Spark.
- Apply data modelling best practices (star schema and lakehouse) for reliable storage and processing.
- Handle NHS/healthcare datasets while ensuring FHIR/HL7 compliance and patient data security.
- Support ML workflows through high-quality data preparation and feature engineering.
- Monitor, troubleshoot, and continuously optimise pipeline performance, quality, and reliability.
Requirements
- Strong hands-on experience with Microsoft Foundry, Azure (Data Factory, ADLS, Synapse) and Databricks (Spark, Delta Lake).
- Proficient in Python and SQL.
- Demonstrated ability to build and deliver end-to-end data pipelines.
- Solid understanding of ETL/ELT design and data modelling (star schema / lakehouse).
- Knowledge of ML workflows (basic to intermediate).
Desirable skills/knowledge/experience:
- Experience working with NHS/healthcare datasets and FHIR/HL7 standards.
- Understanding of patient data sensitivity and compliance in regulated environments.
