Azure Data Engineer

We are seeking a highly skilled Data Engineer to join our dynamic team. The successful candidate will be responsible for designing, developing, and maintaining robust data pipelines and architectures to support our organisation's data-driven initiatives. This role offers an exciting opportunity to work with cutting-edge technologies and contribute to the optimisation of our data infrastructure. The ideal applicant will possess strong technical expertise in cloud platforms, big data processing, and database management, ensuring seamless integration and accessibility of data across various systems., * Develop, implement, and optimise scalable data pipelines using tools such as Apache Spark, Hadoop, and Apache Hive.

• Design and maintain efficient data warehouse solutions in platforms like Oracle, Microsoft SQL Server, and cloud-based environments such as AWS.
• Collaborate with data analysts and scientists to understand data requirements and translate them into effective database solutions.
• Utilise scripting languages including Python, VBA, Bash (Unix shell), and Shell Scripting to automate workflows and manage data processing tasks.
• Perform database design optimisation to ensure high performance, reliability, and security of stored data.
• Integrate diverse data sources through ETL tools such as Informatica to facilitate comprehensive analysis.
• Monitor system performance, troubleshoot issues promptly, and implement improvements for optimal operation.
• Maintain documentation of data architecture, processes, and standards to ensure clarity across teams.

Do you have experience in VBA?, * Proven experience as a Data Engineer or in a similar role with a strong understanding of database design principles.

• Proficiency in programming languages including Java, Python, VBA, Bash (Unix shell), and Shell Scripting.
• Extensive knowledge of big data technologies such as Hadoop, Spark, Apache Hive, and related ecosystems.
• Hands-on experience with cloud services like AWS for scalable data storage and processing solutions.
• Familiarity with relational databases including Oracle and Microsoft SQL Server.
• Strong analytical skills with the ability to interpret complex datasets effectively.
• Experience in building and managing data warehouses and understanding of ETL processes using tools like Informatica.
• Knowledge of SQL for querying and managing large datasets efficiently. This role is ideal for a motivated individual eager to leverage their technical expertise in a collaborative environment dedicated to innovation in data management.