Engineer (Data Test & Assurance) - Homes England

Location

Birmingham, West Midlands (England), B1 2AX : Bristol, South West England, BS1 6EH : Longstanton, East of England, CB24 3EN : Liverpool, North West England, L3 1BP : Leeds, Yorkshire and the Humber, LS1 4AP : Manchester, North West England, M1 6EU : Newcastle upon Tyne, North East England, NE4 5BZ

About the job

Job summary

Job purpose 

As a Data Test & Assurance Engineer, you will operate as a senior technical subject matter expert providing cross-platform data test assurance practice leadership across Homes England’s Azure Databricks Platform.

You will define, embed, and assure automated data and platform testing practices end-to-end across data engineering, analytics, and Data Science / AI‑enabled workloads, ensuring solutions are reliable, secure by design, ethical, and auditable.

The role carries delegated technical leadership responsibility and may require leading and directing the work of contractors and external specialists through technical authority rather than formal line management.

Engineering Community of Practice, mentoring teams and embedding modern ways of working.

This role is crucial to providing the technical leadership and governance required for the Agency to deliver consistent data services, mature its data culture, and respond effectively to strategic and operational priorities.

Job description

Key accountabilities and responsibilities

Cross-Platform Test Strategy and Standards

• Define, own, and continuously evolve the Data Platform Test Strategy, ensuring consistent assurance across infrastructure, CI/CD pipelines, data ingestion, transformation, analytics, and AI/ML workloads. Establish secure‑ by‑design testing standards and patterns adopted across internal teams and suppliers.

Leadership of Contractors and External Resources

• Lead, direct, and quality‑assure the work of contractors and external suppliers delivering data testing, platform assurance, and AI/ML validation capabilities. Set technical direction, review deliverables, and ensure outputs meet Homes England and DDaT standards.

Automated Data, Analytics, and AI Assurance

• Design and implement automated tests for ETL/ELT pipelines, data models, analytical outputs, and machine learning artefacts, including validation of feature engineering, model inputs and outputs, data drift, and reproducibility using Python and SQL‑based frameworks.

CI/CD, Secure-by-Design, and Environment Assurance

• Embed quality and security gates into Azure DevOps CI/CD pipelines. Validate environment parity across Dev, Pre‑Prod, and Prod, and apply secure‑by‑design principles including identity and access management, secrets handling, data masking, and least‑privilege access.

Quality, Observability, and Compliance

• Implement data profiling, anomaly detection, and assurance evidence to support governance, audit, GDPR, responsible AI, and regulatory compliance.

Delivery Tracking and definition

• Maintaining and processing the data test engineering and assurance backlog in collaboration with the product and delivery managers as part of the data service offered to our customers.

Data Community of Practice

• Actively participate and contribute to the Data Analytics Community of Practices from the Data Engineering perspective.

Person specification

Key skills, knowledge and experience

• Hold a relevant degree or professional qualification in computer science, data engineering, mathematics, or a related discipline, or demonstrate minimum equivalent minimum practical experience gained though foundation level Data Engineering Level 5 apprenticeship or higher.
• Strong background in data testing and assurance across cloud data platforms
• Experience with Azure Databricks, Azure Data Factory, Synapse Analytics, and Data Lake Gen2
• Proficiency in Python (PySpark, Pandas) and SQL for automated testing and profiling
• Working knowledge of data science and machine learning lifecycles, including feature engineering and model deployment
• Understanding of AI/ML assurance topics such as bias, explainability, reproducibility, and drift
• Strong understanding of secure‑by‑design principles including IAM, data protection, and least‑privilege access
• Experience embedding testing and security controls into Azure DevOps CI/CD pipelines
• Ability to communicate assurance outcomes clearly to technical and non‑technical stakeholders
• Experienced in managing large, complex structured and unstructured datasets, including collection, validation, analysis, and transformation.
• Able to define solutions for ambiguous or complex problems independently, apply innovative approaches and adopt modern technologies.
• Understanding of project management principles, Agile methodologies, and software delivery practices, capable of leading cross-functional data initiatives.
• Awareness of data governance, security, and regulatory standards, with professional competence equivalent to formal qualifications in data engineering or analytics.
• Experience working within the Homes England sector or similar public sector organisation’s, with an understanding of sector-specific data requirements, compliance, and governance frameworks.