3,135
views
0
recommends
+1 Recommend
1 collections
    1
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      The contribution of a ‘synergic theory of change’ approach to democratising evaluation

      research-article
      Research for All
      UCL Press
      theory of change, democratisation, co-production, evaluation

      Read this article at

      ScienceOpenPublisher
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          This paper focuses on an evaluation of three projects working with young people in innovative ways to tackle societal alcohol misuse. Rather than presenting the findings of the evaluation per se, the paper presents learning from using theory-based approaches in a collaborative way to evaluate these complex, multi-strand initiatives. Traditional evaluations conducted by academics without collaboration with stakeholders can fail to meet the needs of those delivering interventions. Drawing on interviews with practitioners involved in delivering the projects, the paper adds new evidence to epistemological debates by introducing the notion of a ‘synergic theory of change’, whereby academic expertise and the skills, knowledge and experiences of stakeholders are subject to dialogue, and a theory of change becomes the result of collaborative consensus building. This way of using theory of change in evaluation requires researchers to work in a spirit of co-production and dialogue, and it can move evaluation away from being an exercise that seeks to judge interventions and, by extension, practitioners, to one which prioritises a shared learning journey. Using a synergic theory of change approach has the potential to change the nature of evaluation and lead to a different kind of relationship between researchers and practitioners than traditional methods-based approaches allow.

          Most cited references43

          • Record: found
          • Abstract: not found
          • Article: not found

          Using thematic analysis in psychology

            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            Toward a Definition of Mixed Methods Research

              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found
              Is Open Access

              Understanding and misunderstanding randomized controlled trials

              Randomized Controlled Trials (RCTs) are increasingly popular in the social sciences, not only in medicine. We argue that the lay public, and sometimes researchers, put too much trust in RCTs over other methods of investigation. Contrary to frequent claims in the applied literature, randomization does not equalize everything other than the treatment in the treatment and control groups, it does not automatically deliver a precise estimate of the average treatment effect (ATE), and it does not relieve us of the need to think about (observed or unobserved) covariates. Finding out whether an estimate was generated by chance is more difficult than commonly believed. At best, an RCT yields an unbiased estimate, but this property is of limited practical value. Even then, estimates apply only to the sample selected for the trial, often no more than a convenience sample, and justification is required to extend the results to other groups, including any population to which the trial sample belongs, or to any individual, including an individual in the trial. Demanding ‘external validity’ is unhelpful because it expects too much of an RCT while undervaluing its potential contribution. RCTs do indeed require minimal assumptions and can operate with little prior knowledge. This is an advantage when persuading distrustful audiences, but it is a disadvantage for cumulative scientific progress, where prior knowledge should be built upon, not discarded. RCTs can play a role in building scientific knowledge and useful predictions but they can only do so as part of a cumulative program, combining with other methods, including conceptual and theoretical development, to discover not ‘what works’, but ‘why things work’.
                Bookmark

                Author and article information

                Contributors
                Journal
                rfa
                Research for All
                UCL Press (UK )
                2399-8121
                01 March 2022
                : 6
                : 1
                : e06108
                Affiliations
                [1]Senior Research Associate, Newcastle University, UK
                Author information
                https://orcid.org/0000-0001-6103-5213
                Article
                10.14324/RFA.06.1.08
                18bb347d-3897-4c10-9be1-ddd687a88c34
                Copyright 2022, Karen Laing

                This is an open-access article distributed under the terms of the Creative Commons Attribution Licence (CC BY) 4.0 https://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, distribution and reproduction in any medium, provided the original author and source are credited.

                History
                : 12 January 2021
                : 12 December 2021
                Page count
                Figures: 1, Tables: 2, References: 44, Pages: 17
                Categories
                Article

                Assessment, Evaluation & Research methods,Education & Public policy,Educational research & Statistics
                co-production,democratisation,theory of change,evaluation

                Comments

                Comment on this article