Lead Data Engineer - Data Services and Analytics

This is an opportunity to lead a team of data engineers working on complex data pipelines, ingesting and transforming data from/to a variety of sources (email, csv, ODBC/JDBC, JSON, XML, Excel, Avro, Parquet). Data pipelines are primarily developed in AWS using a variety of tools and languages including AWS S3, AWS Athena, AWS Redshift, AWS Glue, java (Docker / Spring). 

The position requires leading the data engineering development for a team of data engineers working on a legacy Oracle data platform and delivering data integration and data retrieval services using PL-SQL. A complex transformation programme is delivering the change of people, process and technology to redesign and transform the processing of the Oracle data outputs onto the strategic data platform. This role would ideally have experience of Oracle and PL-SQL, with the aptitude to move to cloud native tooling such as AWS Athena, python, S3. Additionally, a key priority is to manage the people change to ensure the team stability and business continuity throughout the transformation. 

Main Responsibilities

As a lead Data Engineer, you will showcase a true passion for data and exhibit expertise in the following areas: 

• Manage data sources effectively to improve data quality and make them suitable for analysis. 
• Drive and develop new data models and ETL processes, working collaboratively with the analytics team and mapping data to an appropriate matching data model. 
• Utilise essential key stakeholder management skills to effectively manage expectations, actively listen and interpret business needs, communicate progress, and navigate challenging conversations, all while fostering a service orientated culture around data. 
• Managing the performance of suppliers and the data team to improve capability, as well as creating the roadmap which balances immediate user needs with long-term investment. 
• Developing processes to balance data availability, capacity and data archiving, while continually improving processes and efficiency in the acquisition and ingestion of data. 
• Identifying data issues in data pipelines and managing their resolution. 
• Proficiency in Oracle and Java for developing and maintaining data pipelines and integration solutions.
• Proven track record of leading engineering teams through transformation programs, specifically migrating from legacy platforms to modern cloud-native solutions (e.g., AWS).
• The successful candidate will be responsible for upskilling staff to transition to a new platform. They must also have leadership experience, including coaching and professional development skills. 

UK residency and security requirements       

Please note that this role requires Security Check (SC) clearance, which would normally need 5 years’ UK residency in the past 5 years. 

However, in exceptional circumstances security clearance applications for candidates who have been present in the UK for at least 3 of the last 5 years may be considered. Failure to meet this residency requirement will result in your security clearance application being rejected.