Lessons Learnt – Administrative justice data scoping report

This post describes the lessons learnt during the production of a preliminary scoping report on administrative justice data on social security. The project was part of the wider scoping and capacity-building work of the UK Administrative Justice Institute (UKAJI).

The report has been made available to the community as an open-source public book via GitBook,[7] and researchers and other stakeholders can contribute to it as they work more on the data or as government departments make more data available. It is also available to download as a PDF: Data Sources for Administrative Justice Research

By Kakia Chatsiou

One of the main priorities of UKAJI’s work is to identify and develop strategies to tackle capacity constraints within administrative justice research in the UK, and in particular to improve the availability of information on administrative justice to researchers and other stakeholders. Part of that work has been to deliver a preliminary scoping study to identify relevant data across the public and private sectors and determine how these may be best collected, stored and made available, including the need for data analysis, collation and linkage.[1]

The data-scoping project’s aim was to gather information on data resources available on administrative justice related to topics such as social security/benefits, work and pensions, immigration and asylum. Another aim of the project was to equip researchers in the field of administrative justice with a toolbox that could help them identify where to look for such data and the elements they should consider when using secondary data for research purposes.

The project was originally envisaged to focus on data sources around social security, immigration, and courts and tribunals data sources. The resulting report/s would include identifying data owners and routes to access, what data are potentially easily accessible and what are not. It would also include a consolidating component which would outline the lessons learnt, next steps and ways forward to widen information gathering on data on administrative justice and outline main challenges and opportunities of using secondary data for research on administrative justice.

Project results

One finding was that the scope of the original aim was too broad and ambitious to achieve with the time and resources allocated. A decision was taken to focus on one area in the original remit: social security/benefits. This would allow us to focus on an area of immediate importance in terms of current policy debates and to develop a model for presenting the project results that could apply to the other areas to be covered in future. We have aimed to present the model for feedback from the researcher and policy-making community.

The main project output is in the form of a report freely and openly available to the research and policy-making community to consult and contribute to, in the form of a Gitbook.[2] The citation for the report is:

Chatsiou, K., Bondy, V., Doyle, M., Sena, V., & Sunkin, M. (2017). Data Sources for Administrative Justice Research: A Preliminary Scoping Report. UK Administrative Justice Institute (UKAJI), University of Essex https://doi.org/10.6084/m9.figshare.5303317.v2

The report focuses on data sources on social security and some welfare benefits and administrative decisions around them, including sanctions and appeals as well as complaints. It covers data from across the UK, mostly available from central government departments in England, Northern Ireland, Scotland and Wales.

The report helped us engage with the wider community of administrative justice both in academia and in civil service and has benefitted from comments and feedback from the following reviewers:

• Dr Libby Bishop (UK Data Archive, University of Essex)
• Varda Bondy (School of Law, University of Essex)
• Cris Coxon (Civil and Administrative Justice Research, Analytical Services, Ministry of Justice)
• Margaret Doyle (School of Law, University of Essex)
• Lisa Hayes (Head of Decision Making & Appeals, Motability and ILF Legacy, Disability Employment and Support Directorate, Department for Work and Pensions)
• Prof Gráinne McKeever (Professor of Law and Social Justice | Director of Ulster University Law Clinic, University of Ulster)
• Prof Tom Mullen (Professor of Law, School of Law, University of Glasgow)
• Dr Phil Schofield (Kings College London)
• Prof Vania Sena (Essex Business School | Director of Big and Local Government Research Centre, University of Essex)
• Prof Maurice Sunkin (School of Law, University of Essex)
• Prof. Robert Thomas (University of Manchester)
• Dr. David Webster (University of Glasgow)

The following constraints have prevented us delivering a more comprehensive report:

• The task of recording all data sources useful for administrative justice is ambitious. To make it more manageable, as explained above, we changed the scope to focus our efforts to deliver on data sources from across the UK on social security, as this area is a key research priority for the research team.
• Administrative justice data – very much like administrative data in general – is operational data and depends greatly on the administrative need they fulfil. Processes and information on how to access data sources is very much dependent on the government department or organisation’s aims and policies. The legal landscape of pathways around administrative data is a changing one, too, with the Digital Economy Act 2017 expected to change a lot of the possibilities.

Lessons learnt

This project was a pilot study of the availability of data for administrative justice research, and from our interactions with stakeholders we were able to take a more holistic look at the issue of administrative, and secondary, data for administrative justice, reflecting on areas that could guide future efforts. Through this process, we learnt a number of lessons that could help inform future similar work.

• An ever-changing landscape. The world of data and data governance is rapidly changing as policy-makers realise the importance of evidence and data in shaping the future of society. In the course of writing this report, the UK acquired for the first time a legal framework enabling government departments to share data with researchers for research purposes (Digital Economy Act, 2017); HMRC data became available for uses other than tax policy research; and the Department for Work and Pensions released data for 13 of its core benefits and decisions (up from 2 in its beta form). These are examples of the shifting landscape in administrative data sources.

In March 2016, Scotland acquired new devolved powers on some tax and welfare and employment support benefits (Scotland Act 2016). Scotland acquired legislative control over 11 benefits after June 2017 and will acquire executive responsibility of all devolved benefits by April 2020.[3] In addition to potentially introducing another variation in the types of data collected and a different set of administration and decision policies, devolution will mean that more organisations will be involved as data controllers for data generated and this is likely to break the continuum in the data collection of the devolved benefits.

This implies that sometime in the near future the way information about some of the data sources included in the report might change, as administrative responsibilities transition from central government to the Scottish government.

All of the above have been some of the reasons why any attempt to produce a comprehensive audit of the data landscape has been a challenge and the current report is bound to have missed key changes. There is also a risk that the information provided will go out of date very quickly. Hence the decision to publish the report as an editable, organic resource for researchers, owned by the community for which it was developed.

• Availability, Volume and Quality: We found the remarks of the Administrative Data Taskforce report[4] to still apply in most cases, although steps are being taken to transition to a more data-friendly and research friendly direction.[5] While in this transition phase, though, it is inevitable that there will be great inconsistency in what data, how many and of which quality, will be made available for research.

Some government departments (such as the Department for Work and Pensions) are data-minded and have made available (and are in the process of making available) increasing amounts of data in a well-documented format for researchers to access (which they can also analyse online without a need to download everything) via their online tool (StatXplore). Others (including the HMCTS in England, Wales and Scotland or NI CTS/TAS in N. Ireland) publish some data in the form of Word or pdf versions of the decisions and judgments, but not in a consistent tabular format that will facilitate statistical analysis.

The researcher is the one that most of the times will be tasked with selecting what is relevant for their research and what can be achieved within the related timescale that they have.

• Method of Access. The process of negotiating and accessing data as well as the terms and conditions have varied greatly, depending on the types of data requested, the data owners and how they have agreed with users of their service that their data will be used. We have included “How to access” sections for each of the datasets listed in this report, as a way to help researchers navigate the landscape.

• Coverage. Spatial coverage has depended on the topic and the remit of the government department or executive agency being liaised with. A good way to find this out is via their respective websites or by enquiring with their service staff directly. Getting information about temporal coverage has usually been straightforward if the administrative process has not changed much over the years and the department’s record retention policy ensures records are kept for a long time. Note that we have observed great variations in these, however: for example, HMRC has a policy of retaining all of its tax records, but some records, such as the dental health records held by GPs and the NHS Business Services Authorities, are only kept for 5 years (personal correspondence). Also, note that in the case where the administrative process has changed – e.g. if a benefit was replaced with another – data might not be cross comparable.

• Ownership. Identifying who the owner(s) of the data are (the Data Controller as per the Information Commissioner’s Office definition) has not always been straightforward, as the department that collects the information might not necessarily be the one making decisions about the data. An example of this is the JobCentre data – these are collected by staff at individual centres across the country – and are then submitted to the Department for Work and Pensions, who are the data owners (and can thus decide what can be done with the data and who can access it). Understanding how data flows (i.e. moves from organisation to organisation) and under what terms has been crucial in order for researchers to understand how they can use the data. We have also come across cases in which data are collected and managed by private companies on behalf of government departments – this is an area worth investigating further, as it could prove to be a barrier if, for example, the government department cannot make them available for research because they can no longer make decisions about the data.

• Data (Il)literacy. In the last decade or so, there have been great changes in the burden placed upon researchers when it comes to empirical research. Not only are researchers expected to have a thorough knowledge of the methodology of analysing both quantitative and qualitative data, but they are also expected to be data literate and know how to find their way around a complex system of governance (and data governance) to get that data for their research. Shifting the culture to data-driven, accountable, empirical policy-making is a long process, and change relies not just in data owners making data available for research, but also researchers being able and willing to use data for research that will impact society in a positive way and improve everyone’s lives. Researchers should be encouraged to seek every opportunity to collaborate with peers and policy-makers to advance science and ensure that policy-making is as well-informed as possible.

• The future is collaborative. Feedback we received from reviewers at the early stages of the project highlighted the need for a resource that would be easy to maintain by the UKAJI community, rather than a static publication. The current pilot study is an organic resource – as things are changing and as we get a better understanding of the field the administrative justice research landscape (see the UKAJI Research Roadmap for more information[6]), the resource relies on the community to help keep it up to date and enriched with data sources as much as possible.

Accessing and contributing to the report

The report has been made available to the community as an open-source public book via GitBook,[7] and researchers and other stakeholders can contribute to it as they work more on the data or as government departments make more data available. It is also available to download as a PDF: Data Sources for administrative justice research.

[1] http://www.nuffieldfoundation.org/sites/default/files/files/UKAJI_summary_07_08_14.pdf

[2] https://kakiac.gitbooks.io/data-sources-for-administrative-justice-research

[3] See for example this article: https://www.theguardian.com/politics/2016/nov/15/devolution-scottish-welfare-being-used-political-football

[4] http://www.esrc.ac.uk/research/our-research/administrative-data-research-network/administrative-data-taskforce-adt/

[5] Such as the creation of the Administrative Data Research Network (http://adrn.ac.uk); the Government Digital Service (https://gds.blog.gov.uk/), the Data.gov.uk data catalogue (https://data.gov.uk/) and the Digital Economy Act (2017) (http://www.legislation.gov.uk/ukpga/2017/30/contents/enacted).

[6] https://ukaji.org/2017/08/10/research-roadmap-where-have-we-been-and-where-do-we-need-to-go-with-research-on-administrative-justice/

[7] https://kakiac.gitbooks.io/data-sources-for-administrative-justice-research

About the author:

Dr Kakia Chatsiou is a postdoctoral Senior Research Officer at the Department of Government, University of Essex, as part of the Evaluation Team (Catalyst Project), where she has been working with collaborators to acquire, process and analyse administrative data to support local authorities (such as Essex and Suffolk County Council, Essex County Fire and Rescue Service) improve the impact of programmes and interventions for the communities they serve.

Icon made by Freepik from http://www.flaticon.com