Tech stack
API
Agile Methodologies
Airflow
Amazon Web Services (AWS)
Data analysis
Data Transmissions
Information Engineering
Data Infrastructure
Data Visualization
Amazon DynamoDB
Python
Software Engineering
SQL Databases
Tableau
Data Ingestion
Snowflake
Spark
AWS Lambda
Cloudformation
Data Lake
Api Gateway
Terraform
Serverless Computing
Databricks
Job description
- Build, maintain and scale data ingestion and transformation processes in Databricks that deliver key data and insights to the business and our customers
- Be involved in all stages of the development of innovative data-driven products
- Advocate for best data practices throughout the organisation
- Integrate new data sources into the data platform through APIs or bulk data transfer
- Build and maintain testing and documentation frameworks for our data sources
- Work with the business to scope and deliver new data engineering projects and requirements
- Maintain and build on our existing data infrastructure and tools
- Support the internationalisation of our data infrastructure as we continue to grow globally
- Contribute to the software engineering and data engineering culture here at KrakenFlex
- Collaborate regularly with colleagues with many different professional specialities, including software engineers and data scientists, to create innovative solutions that delight our customers and colleagues
- Work as part of a team of engineers, regularly seeking feedback and growing your skills as technical professionals.
Requirements
-
In depth industry experience in data engineering, software development & design
-
Experience in Python and modern SQL analytics environments
-
Experience with modern technologies for managing data lakes at scale
-
e.g. Databricks, dbt, Spark, Snowflake, Airflow
-
A record of getting things shipped in a collaborative, agile development environment
-
A desire to mentor and pair with more junior engineers
-
Proven experience working with data, both processing and analysing
-
A proven ability to perform well in a fast-paced environment
-
Excellent analytical and multitasking skills
Nice to have :
-
Experience with BI and/or Analytics tools, e.g. Athena, Tableau, Quicksight
-
Experience with AWS or similar cloud providers, and serverless technologies e.g. AWS Lambda, Kinesis, DynamoDB, API Gateway
-
Experience developing, securing or operating cloud scale applications or infrastructure; ideally terraform or cloudformation
Kraken is a certified Great Place to Work in France, Germany, Spain, Japan and Australia. In the UK we are one of the Best Workplaces on Glassdoor with a score of 4.7. Check out our Welcome to the Jungle site (FR/EN) to learn more about our teams and culture.
About the company
Kraken powers some of the most innovative global developments in energy.
We're a technology company focused on creating a smart, sustainable energy system. From optimising renewable generation, creating a more intelligent grid and enabling utilities to provide excellent customer experiences, our operating system for energy is transforming the industry around the world in a way that benefits everyone.
It's a really exciting time in energy. Help us make a real impact on shaping a better, more sustainable future.
About Our Data Platform Team
What we do: build the most AI-driven, innovative, forward-thinking platform for energy management. From optimising resources to delivering cost-effective, exceptional customer experiences through advanced Customer Information Systems (CIS), billing, meter data management, CRM, and AI-driven communications, Kraken is powering the next wave of innovation in the energy industry.
Why we do it: future energy will not look like energy as we know it today. We need to not just think about our future, but build for it. Now.
At Kraken we've developed a data platform that is used by Octopus Energy and our other clients retail energy businesses around the world.
Also at Kraken Flex, Octopus Energy's data-driven DERMS/VPP platform. From commercial energy assets like solar farms and grid batteries to domestic EVs and heat pumps, we use real-time data and smart algorithms to keep renewable energy flowing reliably across the grid.
The platform empowers users with self-service data analytics and automates our data processing workflows, from simple ETL jobs to ML training and prediction.
The data platform team works across the whole customer domain on anything from natural language understanding of our customer communications to processing billions of smart meter readings every day to support clients in creating customised and market leading smart energy tariffs.
As the volume, scope and geographical range of our data offerings rapidly expand, we're looking for an experienced data engineer to join the team to help us build and maintain our platform, pipelines and data sources.
We're passionate about building great technology to change the way customers use, and think about, energy for the good of the planet. This is a fantastic opportunity to work with us on data problems that genuinely move us closer to Net Zero, and support the energy transition.