Principal Data Engineer/Data Solutions Architect

We're looking for Principal Data Engineers & Data Solutions Architects to lead the design and delivery of modern data platforms across cloud environments.

This role blends hands-on engineering with architecture ownership, focused on building scalable, secure, and high-performing data solutions using Lakehouse architectures, Real Time processing, and cloud-native tooling.

You'll work closely with stakeholders across engineering, analytics, and leadership to shape data strategy and deliver production-ready platforms.

Key Responsibilities Architecture & Strategy

• Define and own end-to-end data architecture across ingestion, transformation, storage, and consumption
• Design Lakehouse architectures using platforms like Databricks/Microsoft Fabric
• Drive best practice in data modelling (medallion architecture, dimensional modelling where needed)
• Evaluate and implement modern tooling across batch, streaming, and Real Time use cases
• Align technical solutions with business outcomes and scalability requirements

Engineering & Delivery

• Build and optimise data pipelines using Python, PySpark, SQL
• Develop scalable ETL/ELT frameworks using tools like:

• Databricks
• Azure Data Factory/Fabric Data Pipelines
• Apache Spark/Structured Streaming

• Deliver Real Time and event-driven pipelines (Kafka, Event Hubs)
• Implement CI/CD and infrastructure as code (Terraform, Azure DevOps, GitHub Actions)
• Ensure performance optimisation and cost efficiency across data workloads

Leadership & Stakeholder Engagement

• Act as technical lead/SME across client or internal programmes
• Mentor engineers and support capability growth within the team
• Work closely with stakeholders to translate business needs into data solutions
• Influence architectural decisions and roadmap planning

Data Governance & Security

• Implement data governance frameworks (eg Unity Catalog, Purview)
• Define data quality, lineage, and observability standards
• Ensure compliance with security and regulatory requirements
• Manage access control and secure data environments

• Strong experience as a Senior/Principal Data Engineer or Data Architect
• Hands-on expertise with:

• Databricks (core requirement)
• Azure (preferred), AWS or GCP

• Advanced skills in Python, PySpark, SQL
• Experience designing Lakehouse architectures
• Proven delivery of production-grade data platforms
• Strong understanding of:

• Data modelling (structured + semi-structured)
• Distributed data processing
• Performance optimisation

Desirable Experience

• Exposure to MLOps/ML pipelines (MLflow, feature stores)
• Knowledge of streaming technologies (Kafka, Spark Streaming)
• Experience in consulting/client-facing environments
• Familiarity with data observability tools (eg Monte Carlo, Great Expectations)

Soft Skills

• Strong communication; able to engage both technical and non-technical stakeholders
• Commercial awareness and ability to align data solutions to business value
• Comfortable operating in fast-paced, evolving environments