Formative Evaluation of NHS England and NHS Improvement’s Culture and Leadership Programme: Final Report

The Culture and Leadership Programme (CLP) is a phased organisational approach to shape leadership and culture, such that it positively effects the quality of patient care. The programme has at its core the themes of inclusion and compassion, and supports sustained focus on these for all leaders and staff. It is a national programme which has become mandated for some NHS trusts requiring fundamental performance improvement. This contributed to an increasing number of trusts engaged with the programme, hence it became increasingly important to commission an evaluation to explore, understand and affirm how the programme was working, for whom, when and how. Understanding the answers to these questions will equip NHSEI to optimise the impact of CLP.

This evaluation report distils initial scoping work through to a full formative evaluation, spanning an 18-month period. The evaluation team is formed from a collaboration between the universities of Manchester and Birmingham, and has also drawn from a wide network of academic advisors and stakeholders. As the evaluation commenced, the policy context in a post-Francis era focussed on the themes of inclusion and compassion, within NHS leadership and culture, and how this manifested in the experience of all staff in NHS organisational life.

Whilst concluding this evaluation, the Covid-19 pandemic has resulted in unprecedented, rapid and startling implications for the NHS. The evaluation findings are presented in this context, acknowledging there is an opportunity to take these evolving circumstances into account when considering the recommendations.

The principal evaluation questions were devised to understand how the CLP is being implemented, the support and resources needed to effectively implement the programme, what impact the programme is having on leadership, behaviours, and more broadly, culture itself, and the degree to which new cultural norms are reflecting compassionate and inclusive leadership. The evaluation also solicited how diverse and inclusive cultures could be made manifest in the programme’s materials, and to develop understanding about how to secure consistent support within regional structures to programme delivery, with regard to knowledge, skills and capability.

The evaluation team generated a model for understanding how the CLP is creating impact, through the interactions between sustainable implementation, engagement and inclusion approaches. The evaluation adopted a realist evaluative framework, and a more detailed account is provided in a supplementary document. Summarising the realist framework were six hypotheses, generated during the evaluation process, as follows:

Hypothesis 1: Where a programme has external status and credibility (evidential, academics, think- tank endorsements, implemented by other trusts) the CEO and board, clinicians and staff will be more persuaded that it can help them change the culture of the organisation.

Hypothesis 2: The way in which the CEO and the board/executive sponsor conceptualise, communicate and engage trust staff around the purpose and destination of the culture change will be a significant determining factor in the way staff engage with/respond to programme implementation/culture change.

Hypothesis 3: Where the approach to programme implementation is consistent with the ethos and values of the programme, aligned with other change initiatives and integrated with strategic priorities and direction, and the trust frames and communicates the task as changing the culture of the institution (rather than implementing a programme) – i.e. the programme serves to institutionalise culture change and is a means to an end not an end in itself – then the programme is more likely to become embedded rather than seen as something to be endured, with a finite end.

Hypothesis 4: The type and amount of dedicated support and resource available, influences the way the trust goes about change. Where there is sufficient and dedicated resourcing for the programme, it enables the change team to more effectively progress through the three phases, in a timely manner. The more bespoke and sophisticated this resource is - and the larger the capacity provided - the more the trust will be able to extend implementation more widely. Where there is an experience of being supported when implementing the programme, it helps maintain motivation, and increases the potential for working through the change and being innovative.

Hypothesis 5: Fidelity to the Phase 1 diagnostic model Where trusts both use the specified diagnostic tools and follow the specified structure, activities and process (fidelity to the Discover phase) it will result in - better quality data for deciding on the design and deliver phases, because it will provide a more comprehensive picture of the trust’s performance and culture. Where understanding of the programme - its ethos, values and behaviours; its processes, tools and activities; what data is needed and why; how the data can be used to craft a process of change – is not deeply understood (immersive), then the Discover phase will just be understood/approached as a data collection task to be completed (focus on content), rather than seen as a process for engaging staff and finding out what the trust’s current culture looks like, and how this needs to change. This then means that good data will not be collected or is unhelpfully narrow.

Hypothesis 6: Organisations that see the need to go beyond compliance and regulation, and work to embed the principles of equal access to opportunities, social justice, fairness and human rights into the organisation’s policies and into the ‘DNA’ of leaders’ practices, are likely to be better at developing and sustaining diverse and inclusive cultures.

A mixed methodology was devised consisting of surveys, workshops, site visits, interviews and document analysis across a sample of 20 NHS organisations engaged in different phases of the CLP programme. Data was analysed, synthesised and triangulated through a formative process of peer collaboration, involving the commissioner of the evaluation and a wider academic network of advisors.

There is a substantial amount within the current CLP programme that is positive, constructive and purposeful in supporting NHS trusts to develop a compassionate and inclusive culture. The range of findings describe the implementation of the programme, the support required for implementation, and what promoted impact and facilitated the impact to become sustained and embedded.

We identified key findings about how the programme was implemented, in that sign-up was influenced by academic credibility, energy/arrival of new leaders and/or regulatory and quality concerns. Further, the change team were seen as a critical vehicle for staff engagement and this worked well where change teams reflected staff from a wide range of front-line service, disciplines and from all hierarchical levels. A range of engagement methods were employed but the preponderance of informal encounters and interactions which facilitated credible social connection was noted. The use and efficacy of programme materials was considered in depth, and we concluded that the flexibility conferred from the overall phased programme approach was helpful. However, although some participants saw integration with wider programmes of work within individual organisations was needed, the evidence from the literature reinforces the view from other participants that such integration impacts negatively on cultural change

The need for committed and sustained infrastructure and resources was also emphasised. These findings are significant in that they provide evidence of characteristics of effective organisational development interventions in NHS trusts.

There was a strong appreciation for the range of support to implement the programme, particularly of external support from NHSEI and regional associates. However, it was apparent that support could be reconstituted more consistently in terms of amount, type and duration across the programme, to optimise the programme’s impact.

All trusts wrestled with the difficulty in identifying evidence of impact, and for some trusts, it was still too early in their implementation path. We noted a reliance on existing metrics, particularly NHS staff survey and equality, diversity and inclusion data reported on at national level. Some of the struggle to identify impact related to the degree to which the organisation’s cultural destination was articulated at the outset, and the need to track baseline evidence over time, seeing new cultural norms embedded. As part of the evaluation, a Summative Impact Evaluation Framework has been developed, which is provided as a supplementary document.

We concluded that at the programme level, the six hypotheses were upheld, however, this cannot be generalised to each individual trust, as context in each will be quite different. The hypotheses, and the related relationship between sustainable implementation, engagement and inclusion approaches can be used as both a developmental tool, to facilitate reflection on process, and a local evaluation tool, to track outcomes.

Based on our evaluative insights, we present the following recommendations:

I. The CLP phases are extended and enhanced to include additional activities with increased attention on engagement and evaluation and all phases can be viewed as one life cycle which can subsequently be repeated.
II. In the initial phase, logic models are used to facilitate and articulate the organisation’s cultural destination.
III. The connection between cultural destination and the function of teams needs to be highlighted and made much more explicit in the CLP materials.
IV. Guidance, tools and activities are developed for targeted involvement of middle-tier leaders.
V. The proposed redesign of the programme phases and activities includes feedback loops, to facilitate iteration across activity/phase and board, change team and wider organisation.
VI. Trusts need to establish dedicated infrastructure at the outset that will sustain the programme over the long term.
VII. Trusts establish and fund a dedicated infrastructure that is sustained across all CLP phases and activities with specific expertise in social media and communications, accommodating staff’s time and development needs.
VIII. CLP guidance on how change teams are established needs to be strengthened with regard to membership, recruitment and selection, and capability and skill mix.
IX. In building the dedicated infrastructure, the range of support is addressed, which is likely to involve a blend of internal and external resources.
X. The term equality, diversity and inclusion should be used consistently in oral and written communications for CLP work.
XI. WRES and WDES data is included within Discover tools.
XII. Equality, diversity and inclusion indicators are embedded in the Culture and Outcomes dashboard.
XIII. CLP tools are reviewed to address gaps in equality, diversity and inclusion.
XIV. Branding work is undertaken at the outset to capitalise upon the credibility of the CLP.
XV. A strong social marketing and communications approach is agreed and resourced at the outset.
XVI. Regular developmental and networking opportunities are facilitated to enable trusts to exchange information throughout their CLP life cycles, phases and activities.
XVII. The support from regional teams is targeted to optimise the CLP work and momentum at specific junctures: contracting, supporting work on cultural destination, impact and behaviours, change team initiation, support for synthesis with capacity to provide early support to emerging issues.
XVIII. Recruitment, selection and orientation for regional associates are standardised.
XIX. CLP guidance is developed to address how to support trusts undergoing structural change.
XX. CLP materials are developed to include guidance and case studies illustrating how new cultural norms pertaining to compassionate and inclusive leadership and culture are experienced in practice.
XXI. The Summative Impact Evaluation Framework is iterated and finalised with key stakeholders to integrate multiple perspectives.
XXII. Additional work is undertaken to improve indicators which track cultural change, to include research on evaluating behavioural change pursuant to compassionate and inclusive leadership.

In conclusion, we have appreciated the strong foundations of the CLP and looking to the future, there is rich potential to extend the programme into health and social care systems and enhance the programme by working in collaboration with patient, carer and community groups.