Senior Performance Analyst (AI Chat) - Government Digital Service - G7

Location

Bristol, London, Manchester

About the job

Job summary

The Government Digital Service (GDS) is the digital centre of government — we are responsible for setting, leading and delivering the vision for a modern digital government.

Our priorities are to drive a modern digital government, by:

1. joining up public sector services
2. harnessing the power of AI for the public good
3. strengthening and extending our digital and data public infrastructure
4. elevating leadership and investing in talent
5. funding for outcomes and procuring for growth and innovation
6. committing to transparency and driving accountability

We are home to the Incubator for Artificial Intelligence (I.AI), the world-leading GOV.UK and at the forefront of coordinating the UK’s geospatial strategy and activity. We lead the Government Digital and Data function and champion the work of digital teams across government.

We’re part of the Department for Science, Innovation and Technology (DSIT) and employ more than 1,000 people all over the UK, with hubs in Manchester, London and Bristol.

The Government Digital Service is where talent translates into impact. From your first day, you’ll be working with some of the world’s most highly-skilled digital professionals, all contributing their knowledge to make change on a national scale.

Join us for rewarding work that makes a difference across the UK. You'll solve some of the nation’s highest-priority digital challenges, helping millions of people access services they need.

Job description

As a Senior Performance Analyst in GDS, you’ll be a leading voice for data and analysis within a multidisciplinary agile product team. You'll be responsible for analysing how users interact with government digital services and AI-enabled products, including LLM-powered chatbots and agentic workflows. You'll focus on digital (web and app) analytics but also work with data from other sources such as system telemetry and user feedback to understand both what users are trying to achieve and how AI systems are performing in helping them succeed. Your insights will drive prioritisation, helping product owners to focus on features that work best for users, increase task success and trust.

As a Senior Performance Analyst you’ll:

• provide leadership to product and service managers to identify what needs to be measured, develop hypotheses and test what success looks like
• own the management and development of analytical instrumentation and be responsible for accurate data collection implementation, to influence evidence-based decisions, support business cases and strategic prioritisation
• own and manage strategic work - e.g. community building and training; or analytics implementation policies for GOV.UK products; benchmarking and agreeing common performance indicators
• be proactive in working with other disciplines and datasets to enrich stories, for example
  • working closely with user researchers to combine the ‘what’ and ‘why’ of user behaviour.
  • influencing product managers, designers and developers, in the team and across departments, by developing strategies and recommendations for product improvements, based on the analysis of user data
  • work with data scientists and engineers to combine behavioural analytics, system telemetry, and qualitative feedback to understand both user behaviour (e.g. what users do and why) and system performance (e.g. hallucination risk, retrieval quality issues, incomplete task execution)
• lead the production of accurate, engaging and actionable analysis, reports and dashboards using judgement to tell engaging stories with data, relevant to the appropriate audience (from ministers to junior colleagues)

Support the Analysis Community

• help provide leadership to the analysis community of practice, both within GDS and across government. Develop best practice and knowledge transfer of digital analysis principles and techniques to wider audiences and expand your own and others’ data analysis skills. Forge links with other analysis disciplines
• line manage colleagues within a matrix structure 
• create and share examples of best practice in digital performance management across government

Person specification

We’re interested in people who:

• strong proficiency with digital analytics platforms, such as Google Analytics and Google Tag Manager working with high volumes of data and using appropriate tools such as Google BigQuery, Dataform and APIs
• experience of working in an agile, project-based environment, focused on delivering user needs
• deep understanding of business objectives with the ability to translate these into clear, actionable performance metrics that drive results
• demonstrable success in developing implementation strategy, roadmaps, new processes, measurement frameworks, instrumentation, quality assurance, administration and analysis
• detailed understanding and practical experience of using different data tools to transform and analyse different data sets. An understanding of, and practical experience with data analysis and visualisation tools, e.g. Data Studio, Power BI or Tableau or experience in either python or R
• curiosity and willingness to develop understanding of AI systems, including LLMs, retrieval-augmented generation (RAG), and agent-based workflows, and how to evaluate their performance in production environments
• experience of gathering and understanding technical requirements and working with developers to build suitable technical solutions
• strong data storytelling skills - discovering and sharing analytical insight with stakeholders with various levels of analytical expertise and at all levels of seniority
• proven capability to inspire, coach, and mentor team members, fostering a culture of insightful analysis and creative problem-solving.
• ability to upskill non-technical team members—helping them to better understand data and analytics, and ensuring they can contribute effectively to the broader goals of the team.
• strong grasp and experience of statistical methods and AB/MV testing