Accountability and improvement in the ombuds sector: the role of peer review

Accountability and improvement in the ombuds sector: the role of peer review

By Chris Gill (University of Glasgow)

On Monday 23 September 2019, the International Ombudsman Institute and the Parliamentary and Health Services Ombudsman hosted a seminar aimed at developing best practice in the use of peer review by ombuds offices. In this post, Chris Gill outlines his personal reflections on peer review and its potential contribution to enhancing accountability and learning in the ombuds sector, based on the talk he gave at the seminar.

The growth in ombuds self-regulation

Traditionally, the mainstay of ombuds accountability has been regular appearances before parliamentary committees, focused largely on year-to-year operational matters and performance. These scrutiny sessions can be seen as a form of annual performance review for ombuds. This organisationally focused scrutiny has been supplemented by ad hoc reviews, either focusing on a particular ombuds or taking a more sectoral approach. These reviews take a broader approach aimed at law and policy reform.

A notable trend in recent years, has been for the ombuds sector to develop a range of supplementary self-regulatory tools of accountability, driven by greater calls for accountability from members of the public, and more intense parliamentary and media scrutiny. More broadly, these developments can be seen as part of a trend of declining trust in state institutions and a need for ombuds to do more to demonstrate their trustworthiness. Self-regulatory approaches include:

• The development of service standards guidance by the Ombudsman Association.
• The use of Audit and Risk Committees, featuring an independent membership, and providing some independent oversight (e.g. the committee used by the Scottish Public Services Ombudsman).
• The use of independent service delivery reviewers, to handle independently individual allegations of poor service on the part of an ombuds’ staff.
• The use of stakeholder groups, such as the Local Government and Social Care Ombudsman’s Advisory Forum, involving former complainants and other stakeholders.

The developing use of peer review

In this context, we are now seeing the beginning of a trend towards the use of peer review among ombuds offices in the UK. This can be seen both as a response to more intense public scrutiny and a development of attempts to meet calls for greater accountability on a self-regulatory basis. There have been four landmarks in the use of peer review to date:

• 2009: Jerry White (former Local Government Ombudsman) was asked to conduct a review of the Scottish Public Services Ombudsman’s handling of a historic case.
• 2013: An external evaluation was commissioned of the Local Government Ombudsman, which included a panel composed of Jim Martin (the Scottish Public Services Ombudsman at the time), Richard Thomas (a former Information Commissioner) and Richard Kirkham (University of Sheffield).
• 2018: Independent peer review of the Parliamentary and Health Services Ombudsman, conducted by Peter Tyndall (Ombudsman for Ireland), Caroline Mitchell (Financial Ombudsman Service), and Chris Gill (University of Glasgow).
• 2019: Peer review conducted of the Public Services Ombudsman for Wales by the Local Government and Social Care Ombudsman.

While the use of peers either to conduct (or be part of) external evaluations and reviews of ombuds services can be seen to be at only a nascent phase, there is clearly increasing interest in this area, as shown by the fact that a workshop on peer review was held at the most recent Ombudsman Association conference and the IOI/ PHSO peer review seminar held this week. As a result, this is a good time to consider the pros and cons of peer review as an approach to accountability and improvement and to bring some analytical clarity to its future use and development.

Why use external reviews: two models of review

There are two main prompts for commissioning external reviews of ombuds: a desire to ensure and demonstrate accountability, and a desire for learning and improvement. Of course, there is overlap between these categories, since the process of ensuring accountability is likely to lead to some learning of lessons, while conducting external review with the aim of improvement demonstrates in itself a commitment to external review that contributes to accountability. The categories are also not exhaustive, and there are other potential drivers for external review, such as the use of review by the leadership of an organisation to achieve change in the face of organisational resistance.

The key features of an accountability- focussed external review are as follows:

• External impetus. The prompt for the external review is external, for example a request by a parliamentary committee or high profile media criticism.
• Public findings. Where the aim is to ensure and demonstrate accountability, public reporting of findings is key.
• Transparent and robust methodology. The methodology needs to convince stakeholders that the outcome of the review is based on a clear, transparent, and robust methodological process.
• Retrospective focus. Generally, the focus of an accountability-focused review is to establish whether current standards have been met and past practice is acceptable.
• Review will tend to be independent. Those conducting the review must be independent of the ombuds being reviewed.
• Vertical accountability. The aim of an accountability-focussed review is to demonstrate to an authoritative accountability forum that the organisation is performing well.

The key features of a learning and improvement-focused review are as follows:

• Internal impetus. The prompt for external review is a desire on the part of the ombuds organisation itself to learn, improve, and develop its practice.
• Private findings. Because the impetus of the review is internal, there is less need for public reporting, indeed, generating genuine, open, and transformative learning may be more likely where findings are not being made public.
• Flexible methodology. Methodology may be more flexible, pragmatic and outcome-focused where the review is for internal rather than external consumption.
• Prospective focus. The aim of review here is improving for the future, so review will not necessarily be restricted to past and current practice and instead can involve experimentation with new practices and a focus on what happens next rather than formal assessment against established standards.
• Review need not be independent. Independence may be valued as part of a learning and improvement-focused review, but is less essential and, in some cases, independence may be unhelpful as high levels of trust and shared understandings may be required for open and genuine learning to take place.
• Horizontal learning. Learning in this kind of review takes place through dialogue and exchange in a shared spirit of improvement, rather than a hierarchical assessment of past performance.

As this comparison of these two models of review makes clear, quite different types of external review are likely to be required depending on what the fundamental purpose of review is perceived to be.

Types of review and their relative merits

There are three broad types of review considered here: independent review; hybrid review; and peer review. Independent review involves review by an individual or panel that is entirely independent of the ombuds sector. Hybrid review involves review by a panel that is composed of a mix of independent and peer members. Peer review involves review by peers from the ombuds community.

Independent review is currently the dominant form of external review within the ombuds sector. The policy and law reform reviews referred to above are invariably carried out independently, and in the private sector where external review is more established, independent review is the norm (e.g. reviews of the Financial Ombudsman Services between 2004 and 2019). So in considering peer review as a developing practice, it is important to be clear about its potential strengths and weaknesses relative to independent review. The table below summarises those strengths and weaknesses.

In considering how to overcome the weaknesses of each model and capitalise on their strengths, one option is the use of hybrid reviews, involving panels of independent and peer reviewers. This is the approach used in the external evaluation of the LGO in 2013 and the recent review of the PHSO in 2018. Clearly, the ability of peers to ask the right questions and understand the context has significant potential to enhance the quality and outcomes of a review. There is also potential to ensure that some of the potential downsides of peer review are overcome. For example, transparency and perceived objectivity can be heightened when the review uses objective benchmarks such as the Council of Europe’s “Venice Principles” (Principles for the Protection of the Ombudsman Institution) or the Ombudsman Association’s service standards. However, in order to command public confidence in the outcome of the review, careful attention is also likely to be needed in relation to the composition of the panel. The danger of introducing peer review elements into accountability-focused reviews, is that the peer element overshadows stakeholders’ perceptions of the credibility of the review.

Conclusions: the future use of peer review by ombuds

The above analysis of the relative merits of independent and peer review suggests that:

• Independent review is likely to be of most value when the purpose of a review is the ensure and demonstrate accountability;
• Hybrid review is likely to add value in an accountability context, in helping overcome some of the weaknesses of independent review but, despite this, may struggle to command public credibility in an era of decreased public trust;
• Pure peer review is likely to be of most value when the purpose of a review is to develop, learn, and improve.

If this analysis is right, the future use of peer review by ombuds needs to focus on two areas:

• Developing a robust methodology for using peer review as part of accountability-focussed reviews (i.e. hybrid reviews) and developing guidance on the composition of review panels. Ultimately, since accountability-focused reviews need to satisfy external stakeholders and accountability fora, there needs to be consideration given to developing a convincing business case for the use of hybrid reviews as opposed to fully independent reviews in this context. A starting point might be some of the advantages of peer review identified earlier in this post.
• Developing practice in relation to the use of learning and improvement-focussed review. There appears to be considerable scope to develop peer review as a methodology for sharing learning and developing best practice within the ombuds sector, which could form part of wider moves aimed at professionalising practice within the sector (e.g. training and development initiatives). Developing peer review as a complement to reflective professional practice could lead to substantial innovation, as well as contributing to the self-regulatory efforts of the ombuds sector to demonstrate its accountability.

It is unclear at present how peer review will be used in future within the ombuds sector and current practice is at a very early stage. However, this blog has suggested that there is potential for peer review to contribute to both accountability and learning, albeit that the use of peer review is perhaps more naturally consonant with the latter objective.

Dr Chris Gill is a Lecturer in Public Law at the University of Glasgow.