Researching users’ perspectives – report from a UKAJI workshop

On 26 January 2017, UKAJI hosted an interactive workshop on researching the perspectives of users of administrative justice. Below are a summary of the event and links to papers and presentations.

Background

Understanding ‘users’ perspectives’ is one of the most sought-after aspects within administrative justice and also one of the most complex to research and therefore to understand. Some of the methodological and ethical issues that arise include confidentiality (including of personal data, of processes for challenge and redress, and of outcomes), vulnerability of many segments of the consumer-citizen population, problems with obtaining representative sampling, and access to users. There are also often difficulties arising with recording details on users and the process and gaining access to existing sources of data.

At this interactive workshop on researching users’ perspectives in administrative justice, participants  discussed what research exists, what the research gaps and needs are, and how to improve research capacity in this important area.

The workshop

The workshop was chaired by Professors Genevra Richardson (King’s College London) and Maurice Sunkin (University of Essex). It was attended by a range of researchers, representatives of advice and advocacy organisations, users, legal practitioners and government policy makers.

Presentations

The first session included presentations from four speakers on aspects of researching users’ perspectives.

Grainne McKeever (Reader, Ulster University School of Law and Director of Ulster University Law Clinic) – Finding the gaps and exploiting the opportunities, the importance of networks – identifying and accessing the users, and the impact of the users’ voice

Grainne McKeever paper

Grainne was unable to attend, but Maurice read out her paper, which explored the opportunities that administrative justice provides for empirical evaluation. Identifying gaps is key – what do we know already, and what would push our knowledge further? If as a researcher you’re involving users, it should be because their perspective is needed to fill a gap in knowledge. Gaps arise from lack of evidence but also from diversity in practice, such as that created by devolution of administrative decision-making and appeal mechanisms. Building links with partners is essential for opening doors, building trust, and raising credibility with stakeholders. Pilot, and test – and understand that research purity may have to give way to pragmatism. In the end, some compromise may be necessary.

Sarah O’Neill, independent consultant in legal and consumer policy, former member of the Scottish Tribunals and Administrative Justice Advisory Committee (STAJAC) – Why is the user’s perspective important, who are the users, and what are the possible barriers/obstacles to researching their perspective?

Sarah O’Neill slides

The administrative system wouldn’t exist if it weren’t for users, so understanding what users want is important – do they want respect, to resolve issues, to be treated fairly? Researchers need to consider not just what users want from the process, but the quality of justice delivered as well. Yet there are a number of barriers, the primary one being access to data. Sarah described when, as a member of the STAJAC, she tried to get agreement from the Scottish Courts and Tribunals Service (SCTS) to access data to look into users’ views of several devolved tribunals. The Scottish Government promoted ‘user-focused’ tribunals, but in the end SCTS declined to give permission for the research, and it was therefore not possible to get access to users.

Luc Altmann, Deputy Head of Insight, Her Majesty’s Courts and Tribunals Service (HMCTS) – What research HMCTS is currently doing in relation to tribunals and a brief summary of what evidence gaps have been identified through that work.

Luc Altmann slides

Luc gave an overview of the work his team at HMCTS (which has been in place since October 2016) is doing on users, including producing customer insight packs for internal use and commissioning primary research on users across four jurisdictions: criminal, civil, family, and administrative justice. The insight packs gather information from a range of sources and will  be used by HMCTS colleagues for designing and testing processes as part of the reform programme. Initially the packs will cover immigration, social security and employment. The department is keen to consider how policy changes (and external changes eg Brexit) have affected or will affect tribunal volumes. They are also interested to identify where users in the tribunals system require support.

Other projects being scoped include considering international data and how findings can translate to the UK context; exploring what data are available from other government  departments, eg DWP; customer demographics to inform the development of digital services; reasons for decline in volume of mental health tribunals; and drivers for choice between paper appeals versus oral hearings.

Michael Adler, Emeritus Professor of Socio-Legal Studies, and Jackie Gulland, Lecturer in Social Work, School of Social and Political Science, University of Edinburgh – The MoJ’s proposals for digital dispute resolution from the user’s perspective

Michael Adler and Jackie Gulland paper

Michael pointed out that the Ministry of Justice’s ‘Transforming our Justice System’ consultation is a slim document but anticipates massive and ambitious reform. Much of the emphasis is on digital access, which may well facilitate greater access for some users, but it is being rushed through, and there is no apparent commitment to evaluating the changes as they are rolled out. There is also no reference to users’ perspectives in the document. Although the Traffic Penalty Tribunal is often cited as an example of effective online access, social security cases are more complex and may involve users who are less computer literate.

A review of the evidence which Adler and Gulland carried out in 2003 demonstrated that appellants like the experience of coming before real people who ask them questions. They also like being able to inspect documents online, but are then less likely to feel decision-makers are impartial. The review also illustrated the difficulties people face in using tribunals. Problems and challenges faced by some disabled users might be addressed by a focus on digital processes, but ‘digital by default’ might create problems for others. The test for Online Dispute Resolution (ODR) must be: Is this aspect of the process going to produce better justice for appellants and users, rather than focus merely on justice for less cost? We must not lose the advantages of an inquisitorial approach in the transition to ODR, and changes should be introduced incrementally and properly evaluated. Simpler forms of online access and communication could be explored, including Skype.

Q&A Session

Questions following the presentations highlighted a number of challenges. For example, if researchers can make contact with users only through advice and advocacy organisations, will that skew the sample? It may be possible to access users online, via social media, for the purpose of research participation, but again this will produce a self-selected sample and perhaps reflect only negative experiences of the system. It is also important to remember that the user ‘experience’ starts before one gets into a tribunal process, and that research needs to include not just appeal and redress mechanisms but also the experience of administrative decision-making and review by public bodies themselves.

The actual nature of digitising the tribunal processes was questioned – is the aim simply to replicate current offline process in an online version, or is it about transforming the way case management is conducted? It might be useful to consider how ombud processes have developed, with greater freedom perhaps to employ hierarchies of staff and flexible procedures. The Personal Support Unit (PSU) has been working with HMCTS on piloting some of the reforms, involving users, but the extent of consultation on the reforms could be greater. Areas identified as requiring more research include:

• how to capture unmet need and reach those who fail to enter the system or fall out of the system
• the impact of rising appeal fees and changing processes
• the need for greater insight into the entire ‘customer journey’ from start to finish. This could benefit from also gaining insight from within departments if collaboration can be established between independent researchers and decision makers/public bodies.

Small group work

Five key themes were discussed in groups: Who are the users?, What is a ‘user perspective’?; What do we need to know; What data exist already?; What are the obstacles and benefits? Notes of the small group sessions are available here:

ukaji-workshop-small-groups.

Plenary presentation

Martin Partington, former Law Commissioner and member of the Council on Tribunals, presented a brief overview of how the users’ perspective has been at the heart of tribunal policy for many years, noting the historical emphasis on empathy, listening, as well as developing judges’ ‘enabling roles’. Following an overview of past research, he noted that  despite the above, there is a dearth of studies that examine whether the steps taken to facilitate the user experience have been in any sense effective.

Martin Partington paper

Next steps:

UKAJI is interested in further developing the work on researching users’ perspectives. We hope to produce a review of the existing research and establish and support a working party to consider access to users and data and appropriate methodologies. If you are interested in being part of this project, please get in touch.