Principal Data Engineer, Networks and Infrastructure - Home Office - G6
Government Digital & Data -
Location
Croydon
About the job
Job summary
In the Central FinOps Team, you will lead delivery of data products that enable the Home Office to understand, govern and optimise public cloud spend across AWS and Azure, supporting showback/chargeback and cost optimisation outcomes. You will own the end to end approach for ingesting, normalising and enriching cloud billing and usage data, with a strong focus on CUR (including CUR2) and FOCUS datasets as foundational sources for departmental reporting.
You will ensure the organisation can confidently interpret cloud billing behaviour, including the application of commitment based discounts and commercial mechanisms, and translate this into trusted, reusable data assets. You will work closely with finance and engineering stakeholders to ensure outputs meet operational needs (including exportable, finance ready reporting) and improve the quality of the standing data that underpins allocation and recharge.
A key part of the role is enabling consistent access to insight through dashboards and datasets, including those built in AWS QuickSight using AWS and Azure billing data enriched with organisational context (e.g. cost centres and portfolios).
You will also help set minimum standards and requirements for FinOps capability across the hub and spoke model.
Where business needs allow, some roles may be suitable for a combination of office and home-based working. Where this is the case, employees will be expected to spend a minimum of 60% of their working time in the office. Applicants can raise any queries to the email address at the bottom of the advert.
Watch this short video to hear from members of Home Office Digital talking about the projects they work on and their experience of working here: Working for Home Office Digital.
Job description
Principal Data Engineers lead multi-functional delivery teams to deliver robust data services for their department, other government departments and private sector partners.
Through technical direction and leadership, you will create and nurture an environment that emphasises motivation, empowerment, teamwork, continuous improvement and a passion for providing service.
You will work with other members of the data community to set the enterprise data strategy and standards and agree implementation and governance.
Person specification
Main Responsibilities
Your main day to day responsibilities will be:
- Providing technical direction and leadership within the Data Platform team to maximise its effectiveness and inspire best practice for data products and services.
- Developing capabilities in tools and techniques to analyse large/big data-sets and from multiple/complex sources that will enable broader Data Science models and algorithms to uncover insights, trends, and patterns in the data.
- Leading in defining our data strategy and roadmap influenced by future technology changes and implemented through best practice and governance.
- Managing stakeholders at all levels, including facilitating discussions around complex topics and managing performance expectations.
- Initiating and facilitating partnership with other technical and non-technical departments and outside of government to champion data engineering and help them understand how data solutions can benefit them while ensuring easy and effective access to data resources.
- Building the data engineering capability through coaching and mentoring other Data Engineers.
Working Pattern
Due to the business requirements of this role, it is only available on a full-time basis. However, compressed hours are available.
Essential Skills
- Effectively leading and managing multiple teams, helping them deliver technical products and projects in data analysis, solution design and end user reporting.
- Promoting and monitoring continuous professional and personal development within an organisation and team.
- Interpreting requirements and presenting complex ideas and recommendations that inform wider decisions in a clear way, to technical and non technical audiences and senior customers.
- Strong expertise in data engineering and analytical tooling used to process and analyse large‑scale cloud cost and usage data, including SQL (Presto/Trino via Athena), PostgreSQL, and Python. This includes hands on experience working with AWS Cost and Usage Reports (CUR v1/v2), FOCUS files, and semi‑structured data formats such as JSON, Parquet and Iceberg, ensuring accuracy and consistency in financial reporting and analysis.
- An excellent in‑depth understanding of cloud data platforms and billing data models, particularly across AWS and Azure, including how usage, pricing, amortisation and allocation are represented. This includes deep knowledge of cloud provider billing intricacies, commitment‑based discounts and incentives (e.g. Savings Plans, Reserved Instances, Azure Reservations, enterprise agreements), how they are applied in practice, and the known quirks and edge cases that affect cost transparency and optimisation.
- Proven ability to advise on and set best practice for data engineering, automation and reliability, including version controlled development (Git/GitLab or equivalent SVE) and automated, repeatable pipelines. This includes designing and maintaining robust processes for cost ingestion, enrichment, validation and analysis, using services such as AWS Lambda, Step Functions, DynamoDB and Glue, and ensuring FinOps data products are auditable, resilient and fit for decision‑making at scale.
