By Sarah Weakley, Policy Scotland Research and Impact Officer

Policy Scotland and Scotland’s Third Sector Research Forum hosted the virtual seminar Innovating in the third sector – online research and services in 2020 with 40 people from the academic, third and public sectors.

The aim of the event was for researchers and evaluators in the academic and third sectors to share their lessons learned from adapting their methods in 2020. As with most planned research and evaluation activities, staff in third sector organisations had to make decisions quickly last spring on whether and how to carry forward their planned activities. For all presenters, their research and evaluation work will continue online or via telephone for the foreseeable future – the work continues.

The four presenters in this seminar shared how they adapted their planned research and evaluation activities to online or telephone methods, the pros and cons of online delivery for different population groups, the technicalities of ensuring digital access for populations with no devices or few skills, and the different types of data that are produced from these methods.  

• Dr Amy Calder (YouthLink Scotland) and Kelly McInnes, Northern Star, spoke about the evaluation of YouthLink’s digital youth work with LGBT young people using Transformative Evaluation.  
• Dr Jane Cullingworth shared her work in the University of Glasgow/London School of Hygiene and Tropical Medicine Project, Research with disabled people in a time of COVID, a UKRI-funded study.  
• Chris Ross (Children in Scotland) spoke about two peer-led research projects, one before lockdown (Health Inequalities: Peer Research Into the Role of Communities) and one during 2020 (Evaluation of the Home and Belonging Initiative).  
• Ruth McKenna (Waverley Care) shared two research and evaluation projects as well —  the evaluation of Waverley Care’s HIV Street Service and a study of HIV Self-Test Scotland.  

Key messages from the session 

The same principles apply, just in a different delivery context

Presenters from two of the third sector organisations (Children in Scotland and Waverley Care) talked about their projects that had a strong focus on peer-led research activities that were disrupted. These types of projects took much longer to adapt, as those in the organisations had to consider how to keep the participation of peer researchers high with new data collection methods. For example, Children in Scotland’s peer evaluators would normally go visit sites to help evaluate a project, but instead helped to analyse interview transcripts collected over Zoom/Teams. In order to make these transcripts accessible to young researchers, CiS staff had to clean these transcripts to then pass them on to researchers which took more time. For staff at Waverley Care, low income peer researchers had to be provided with mobile telephones to conduct the research and trained to do telephone research by centre staff.  

Plan in more time for data collection and research activities if delivering online

A common thread from the presenters was the recognition that doing research and running activities online need to be broken down into smaller amounts of time. Where a training session or peer research event would normally take a full day in person, presenters noted that a full day session would likely have to be spread out over multiple sessions when delivered online. Presenters found the interviews would need to be shorter (an hour or less) for participants in an online or telephone format, at it often more taxing to keep participants engaged. Researchers from YouthLink Scotland’s project on the impact of LGBT digital youth work found that the stories their youth workers collected from young people, following the Transformative Evaluation methodology¹, tended to be shorter.  

Less ‘deep’ data, but greater reach and more power to the participants?

Relatedly, presenters noted that because online data collection methods or surveys took less time, some were able to reach more interview participants than they would have in person. The University of Glasgow Research project was focussed specifically on understanding the impact of the pandemic on disabled people, so the work was never intended to be in person. Using Zoom/Teams, telephone and sometimes via email where participants were given questions to respond to, researchers will be able to reach nearly 200 participants in just 12 months. Participants were able to choose which method they preferred. This made it possible for some participants with conditions like dementia select email communication if they preferred, for participants to turn their cameras off on Zoom until they felt ready – which the presenters noted ‘centred the participant in the research more than the researcher’.  

Consider pastoral support for researchers, build in time for reflection and avoid comparison

Thinking about their work in 2021, all the presenters and those in the discussion groups noted the need for more pastoral support for the researchers and something they hope to build into their projects in 2021. The speed at which projects had to adapt and the ability for more data to be collected online meant that work has increased. Trying to stay ‘on track’ with planned 2020 projects before the crisis meant that there was real a danger of burnout. In the eye of the ‘storm’ of 2020, researchers felt like there was no time to reflect on what worked and what didn’t and how much online delivery and data collection was manageable for those doing the work. As we are likely to be in this new research and evaluation context for a while, it is useful to avoid comparison with previous ways of working, accept the adaptations and meet ourselves and our participants where they are in 2021.  

References

1. For more of YouthLink Scotland’s projects that use Transformative Evaluation, please see:

• The Impact of Community-Based Universal Youth Work in Scotland (A National Study)
• The Impact of Community-Based Universal Youth Work in Edinburgh (with the Voluntary Sector) (PDF) 

Image credit: Marco Wolff | Pixabay