Kelvin Smith PhD Project Opportunity – 2015/16

Public health – Using synthetic control methods to evaluate the impact of public service improvements

The shift towards evidence-informed policy-making, exemplified by initiatives such as What Works Scotland, requires policy-makers to demonstrate the effectiveness of interventions. Given the practical, political or ethical difficulties of applying experimental approaches to many policy interventions, evaluation methods that use observational data are now a focus of keen research and policy interest.

Synthetic control methods are a novel approach to comparative case study research using observational data. They were developed within political science but can potentially be applied to a wide range of evaluation problems in economics, public health, social policy and other disciplines. In the standard approach, an area in which a new or redesigned service is being implemented is compared with another ‘control’ area, in which there is no change, and statistical adjustment used to account for any differences between areas that might bias the comparison. In the new approach, a synthetic control is derived using data on past trends in all potentially comparable areas, thus providing a more robust basis for identifying the impact of the service change.

The PhD will identify existing applications of the method reported in the international literature, and apply it in the first case study to use synthetic controls to evaluate public sector reform in Scotland. Choice of case study will be informed by the literature review but will focus on place-based interventions that have potentially substantial impacts on population safety, health or wellbeing which can be ascertained from routinely collected administrative data.

Synthetic control methods are a potentially valuable addition to the range of techniques available for non-randomised evaluations of social, economic and public health interventions. The project will establish whether synthetic control methods can be applied to place-based interventions using routine data. If they can, we will have identified an efficient solution to a wide range of pressing evaluation problems.

The research project will involve a systematic literature review and secondary analysis of routinely collected
data. The student will be able to draw on the expertise and facilities within Urban Studies and SPHSU to
support evidence synthesis and the statistical analysis of large datasets, including a specialist Information
Scientist and facilities for the secure storage and analysis of confidential data.

The scholar will be responsible for:
(1) Conducting a systematic review of the use of synthetic control methods…
(2) Identifying one or more candidate interventions
(3) Identifying relevant sources of routinely collected data
(4) Deriving a synthetic control, and using it to estimate the impact of the candidate intervention[s]
(5) Performing graphical and statistical tests of the effect estimates
(6) Comparing the synthetic control estimates with estimates derived from standard methods such as propensity score matching
(7) Writing up the methods and findings for publication
(8) Preparing summaries for dissemination to policy/practitioner audiences

The scholarship will provide training in a range of evidence synthesis and quantitative evaluation methods. Together these will provide an excellent grounding for a research career in applied social or public health science.

We will be seeking to appoint a candidate with some training in statistics, as part of an undergraduate or masters degree in politics, sociology, economics, epidemiology or public health, but training in evidence synthesis and the specialist statistical methods required for the study will be provided.

The appointee will also benefit from working in the excellent multidisciplinary research environment provided by Urban Studies and SPHSU, and from the supervisors’ links with policy makers and other decision-makers via Policy Scotland and What Works Scotland. We envisage that the researcher will be directly interacting with policy users and other key external stakeholders and will build important ‘soft’ transferable skills relating to communication, wider perspectives and the different needs of a range of interests beyond academia.

For further guidance on the application process please click HERE

*Applications should be submitted to by 15 January 2015 as a single PDF document.*