Senior Data Engineer - Data Pipelines

Here, the answers aren't always available. So, you'll need to bring a fearless, self-starter mindset to navigate uncharted territories. You'll harness your ceaseless energy to discover and make the necessary connections with colleagues to shape the future and achieve maximum impact., You will join a team that fuses data engineering with cutting-edge science, using HPC and AWS to deliver reproducible workflows at scale. From first ingestion to consumption by scientists and AI models, you will set the standard for reliability, speed, and governance across our data foundation.

Do you thrive where learning is continuous and bold ideas are encouraged? You will have the freedom to experiment, the support to grow, and the opportunity to see your work influence breakthroughs as they take shape.

Accountabilities:

• Pipeline Engineering: Design, implement, and operate fit-for-purpose data pipelines for bioinformatics and scientific data, from ingestion to consumption.
• Workflow Orchestration: Build reproducible pipelines using frameworks such as Nextflow (preferred) or Snakemake; integrate with schedulers and HPC/cloud resources.
• Data Platforms: Develop data models, warehousing layers, and metadata/lineage; ensure data quality, reliability, and governance.
• Scalability and Performance: Optimize pipelines for throughput and cost across Unix/Linux HPC and cloud environments (AWS preferred); implement observability and reliability practices.
• Collaboration: Translate scientific and business requirements into technical designs; partner with CPSS stakeholders, R&D IT, and DS&AI to co-create solutions.
• Engineering Excellence: Establish and maintain version control, CI/CD, automated testing, code review, and design patterns to ensure maintainability and compliance.
• Enablement: Produce documentation and reusable components; mentor peers and promote best practices in data engineering and scientific computing.

Essential Skills/Experience:

• Pipeline engineering: Design, implement, and operate fit-for-purpose data pipelines for bioinformatics and scientific data, from ingestion to consumption.
• Workflow orchestration: Build reproducible pipelines using frameworks such as Nextflow (preferred) or Snakemake; integrate with schedulers and HPC/cloud resources.
• Data platforms: Develop data models, warehousing layers, and metadata/lineage; ensure data quality, reliability, and governance.
• Scalability and performance: Optimize pipelines for throughput and cost across Unix/Linux HPC and cloud environments (AWS preferred); implement observability and reliability practices.
• Collaboration: Translate scientific and business requirements into technical designs; partner with CPSS stakeholders, R&D IT, and DS&AI to co-create solutions.
• Engineering excellence: Establish and maintain version control, CI/CD, automated testing, code review, and design patterns to ensure maintainability and compliance.
• Enablement: Produce documentation and reusable components; mentor peers and promote best practices in data engineering and scientific computing., Your wellbeing means a lot to us, and we're here to support you through all of life's ups and downs. That's why we offer an unpaid leave policy, annual leave, reduced-hours timetables and a host of benefits, including a retirement plan, long service award, and health and travel insurance.

Ready to make an impact in your career? If you're passionate, growth-orientated and a true team player, we'll help you succeed. Here are some of the skills and capabilities we look for., Seize ownership and excel with autonomy to enjoy the constant rush of ground-breaking discovery. Your ability to anticipate sudden shifts and adapt swiftly will prove critical as you make your mark in an environment that rewards initiative and resilience., * Strong programming in Python and Bash for workflow development and scientific computing.

• Experience with containerization and packaging (Docker, Singularity, Conda) for reproducible pipelines.
• Familiarity with data warehousing and analytics platforms (e.g., Redshift, Snowflake, Databricks) and data catalog/lineage tools.
• Experience with observability and reliability tooling (Prometheus/Grafana, ELK, tracing) in HPC and cloud contexts.
• Knowledge of infrastructure as code and cloud orchestration (Terraform, CloudFormation, Kubernetes).
• Understanding of FAIR data principles and domain-specific bioinformatics formats and standards.
• Track record of mentoring engineers and enabling cross-functional teams with reusable components and documentation.
• Experience optimizing performance and cost on AWS, including spot strategies, autoscaling, and storage tiers.

When we put unexpected teams in the same room, we unleash bold thinking with the power to inspire life-changing medicines. In-person working gives us the platform we need to connect, work at pace and challenge perceptions. That's why we work, on average, a minimum of three days per week from the office. But that doesn't mean we're not flexible. We balance the expectation of being in the office while respecting individual flexibility. Join us in our unique and ambitious world.

Why choose AstraZeneca Spain? AstraZeneca Spain is a rising force in our global business. With headquarters in Madrid and our global hub in Barcelona, we've become an important international centre of excellence in the fight against critical disease. Boasting vibrant universities and business schools, the Barcelona ecosystem is a place where scientists can thrive. We attract a diverse workforce from across the globe, shining a beacon for innovation in a country that's committed to clinical development. We invite you to bring your talents to Barcelona where our respiratory medicine R&D and Global Marketing centre offers opportunities in R&D, IT, Commercial and HR. Or join us in Madrid and shape our growth in our BUs (Respiratory, Oncology & CVRM ), and a range of Corporate functions. Additionally, you can find sales roles throughout the country. Together, we're contributing to a world-leading pipeline of therapeutics and delivering life-changing medicines to patients. Who do we look for? Calling all tech innovators, ownership takers, challenge seekers and proactive collaborators. At AstraZeneca Spain, breakthroughs born in the lab become transformative medicine for the world's most complex diseases. Alongside technical expertise, colleagues have the resilience, energy and collaborative mindset to change lanes, work with different teams and start projects from scratch. Here, diverse minds and bold disruptors can meaningfully impact the future of healthcare using cutting-edge technology. Whether you join us in Madrid or Barcelona, you can make a tangible impact within a global biopharmaceutical company that invests in your future. Join a talented global team that's powering AstraZeneca to better serve patients every day., There are many things I enjoy when working at AstraZeneca, mainly the Speak up culture, the great colleagues that are in my teams, the great products that AstraZeneca provides to our patients and the challenging conversations I have around our medicines. Sales Representative Oslo, Norway There are many things I enjoy when working at AstraZeneca, mainly the Speak up culture, the great colleagues that are in my teams, the great products that AstraZeneca provides to our patients and the challenging conversations I have around our medicines. See Thomas's Story Christine Recchio Working at AstraZeneca has impacted my life in such a positive way. I now have an improved work-life balance through creating my own schedule and time management, I feel a balance that I didn't have before. Sales Representative California, United States Working at AstraZeneca has impacted my life in such a positive way. I now have an improved work-life balance through creating my own schedule and time management, I feel a balance that I didn't have before. See Larry's Story Stephanie Ling There are a lot of reasons why I enjoy working in AstraZeneca, my colleagues being one of them. My team members and the managers have provided a great deal of guidance in helping me to be more confident in my daily work. Sales Representative Petaling Jaya, Malaysia There are a lot of reasons why I enjoy working in AstraZeneca, my colleagues being one of them. My team members and the managers have provided a great deal of guidance in helping me to be more confident in my daily work. See Stephanie's Story What we offer We're driven by our shared values of serving people, society and the planet. Our people make this possible, which is why we prioritise diversity, inclusivity, balance and sustainability. Discover what a career at AstraZeneca could mean for you., Diversity and inclusion are embedded in everything we do, and our different views, experiences and strengths enrich our culture. There's no salary gap at AstraZeneca, and the number of female employees has increased by four per cent over the last three years. We've also made all positions fully accessible., The success of AstraZeneca is founded on innovation, creativity and diversity. Discover what this means for you. Great culture, great work assignments, supportive management. Rotation opportunity within the company. They value our people. see our latest Glassdoor reviews here