Data Engineers
Role details
Job location
Tech stack
Job description
Our Data Engineering role will be responsible for building and maintaining data processing pipelines and also the transformation and optimisation of data for analytical use. As Data Engineer, you'll be part of our experienced software dev function, working in a cross-functional Agile team. We have opportunities for Data Engineers at every level within a team, so upon reviewing your application we will discuss the great opportunities for development or challenges we offer based off your professional profile. Due to the interesting work we do and the sector this team is working in, we require all candidates to hold current eDV clearance. Responsibilities Build data pipelines that clean, transform, and aggregate data from disparate sources Collaborate with stakeholders and other engineers Contribute to the completion of milestones associated with your project Contribute to continuous improvement within your team Collaborate with your peers on technical direction within your team
Requirements
Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging / Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services) SQL and NoSQL databases and storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker / Kubernetes / Openshift) Testing frameworks and best practices We appreciate you may not be an expert in every area above - we can support with training and development in some areas! Please do make an application and we will identify where we can best support your growth specific to your application. Benefits & Work Culture
Benefits & conditions
- Contributory Pension Scheme (up to 10.5% company contribution)
- 6 times salary 'Life Assurance' with pension
- 25 days holiday (increasing with service) + statutory public holidays, plus opportunity to buy and sell up to 5 days
- Company bonus scheme (discretionary)
- Flexible Benefits scheme with extensive salary sacrifice schemes, including Health Cashplan, Dental, and Cycle to Work amongst others
- Enhanced sick pay
- Enhanced family friendly policies including enhanced maternity, paternity & shared parental leave
Work Culture:
- 37hr working week
- Early 1.30pm finish Friday, start your weekend early!
- Up to 5 paid days volunteering each year.
- Flexible working culture focused on output, with more formal flexible working arrangements on request (assessed subject to role) - please highlight any requests to the Talent Acquisition team.