3,844
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      scite_
       
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Participatory evaluation for large-scale arts programmes: challenges, adaptations and unexpected shifts in culture

      research-article
        1 , , 2
      Research for All
      UCL Press
      participation, co-production, evaluation, arts, culture
      Bookmark

            Abstract

            Researchers at Staffordshire University have a long history of undertaking community-based research in and with communities in Stoke-on-Trent. Commitment to the principles and practice of participatory research by the university’s Creative Communities Unit (CCU) team led to an approach to participatory action research called Get Talking. This article highlights the use of Get Talking to evaluate Appetite, an Arts Council England Creative People and Places project in Stoke-on-Trent. Staffordshire University conducted a participatory evaluation using Get Talking for the three years of the programme’s first phase. As well as reflecting on the strengths of the approach, we also discuss the ways in which the approach was adapted in order to address some of the key challenges that were encountered, and the impact of taking a participatory approach to the evaluation on the programme’s culture in relation to community engagement and co-production.

            Main article text

            Key messages
            Creative and artistic consultation techniques can provide a powerful and highly engaging means of involving communities in conversations about art, and, in turn, of supporting arts organisations to be responsive to local needs and interests.
            There are advantages to developing a nuanced approach of using creative consultation tools underpinned by participatory principles alongside post-event surveys. This facilitates large sample sizes generating quantitative data, which supports funders to demonstrate the reach of the programme. In addition, the qualitative approaches provide a flexible and inclusive way of engaging audiences in conversations about the arts, which allows for self-expression and generates more in-depth insights. This can be valuable for audiences, participants and project leads.
            Participatory evaluation leads need to be aware of the potential negative impacts of being a community researcher, including time commitments and emotional labour. Organisations and evaluators should ensure appropriate support structures are in place for community researchers, including access to well-being services and regular opportunities to honestly articulate experiences.

            Introduction

            Get Talking is an approach to participatory action research (PAR) developed and adopted by Staffordshire University’s Creative Communities Unit (CCU) in collaboration with Kate Gant of Creative Health CIC (Emadi-Coffin, 2008; Gratton and Beddows, 2018). It is built upon principles of co-production, and it actively involves those who are affected by the research in its design and process. Get Talking was originally designed to support a Sure Start Centre in Stoke-on-Trent (UK) to take a structured approach to consultation, and it was later developed into a short course for people living and working in a specific geographical community close to the main Staffordshire University campus (Emadi-Coffin, 2008). The short course provides community researchers with concurrent support and coaching from academic members of staff to implement their course learning into live action research projects. It is accredited as a continual professional development (CPD) course, and it offers flexibility to participants, who can opt to take the course with or without the accreditation.

            Get Talking has been used in a broad range of settings, locally in Stoke-on-Trent and across the UK, to train community members and organisations in creative and participatory consultation techniques, demonstrating a long-term, sustainable impact in some areas. In one such project, a team of community researchers identified a local park as being of concern to local residents (Vincent, 2010), from which a ‘Friends of’ group has been established in response to their recommendations for action. This group was instrumental in securing substantial funding for the park five years after the initial consultation activity took place (Gratton, 2020). Another large public sector organisation used the findings from a Get Talking consultation to develop an informed and responsive Children and Young People’s Strategy (Gratton and Beddows, 2018).

            This article shares insights from applying Get Talking approaches within a £3 million Arts Council England-funded Creative People and Places programme called Appetite. A consortium of organisations based in Stoke-on-Trent and Newcastle-under-Lyme, comprised of the New Vic Theatre, Partners in Creative Learning, B Arts, Brighter Futures and Staffordshire University, launched the Appetite programme in April 2013.

            Appetite is an action learning programme that aims to develop audiences for high-quality art, and to increase the numbers of people in the city who experience and become inspired by the arts. It is concerned with the balance between offering excellence in artistic outputs and experiences and excellence in the community development processes that bring about increased arts engagement (Appetite, 2013; Gratton, 2014).

            In the first three years of the programme, Appetite supported community hubs made up of local groups and residents with little experience of commissioning art, which co-produced quality arts in localised areas. In addition, Appetite delivered a strategic programme of large-scale, high-quality outdoor arts. Project partners were also keen to explore how the evaluation could reflect the overall aim of increasing arts engagement. This led them to adopt Get Talking to evaluate the programme in its first phase, between 2013 and 2016. At the time of writing (May 2020), Appetite is in its seventh year, and it continues to provide inspiration to people less likely to engage with the arts in Stoke-on-Trent and North Staffordshire.

            In this article, we explain how Staffordshire University researchers applied the principles of PAR in developing the Get Talking approach. We then focus on the use of Get Talking in the Appetite evaluation, highlighting its strengths, limitations and overall challenges. We also highlight the impact of using Get Talking on participation and inclusion. As this is a methodological article, the research findings are not discussed, but they can be accessed through other project outputs (Gratton, 2014, 2015).

            Participatory action research and Get Talking

            Staffordshire University established the CCU in 2002, and it operated until 2018. The CCU was staffed by a multidisciplinary team of community practitioners and academics with a range of specialisms, including community development, youth and community work practice, policy development, regeneration, mentoring and coaching, and community arts. As such, staff members tended to bring with them strong connections with a wide range of communities and external organisations. These relationships and its outward-looking focus formed a strong element of CCU’s identity throughout its lifespan, and they were evident through the three areas of CCU’s practice: teaching, consultancy and research (Gratton, 2020). In this context, the values and principles of CCU were strongly aligned with those underpinning PAR.

            PAR is an approach to research that emphasises the value of relationships with local people by working with them as community researchers. Community researchers in PAR are involved in the whole research process by identifying the ‘problem’ or area of interest, analysing findings, and planning and taking action (McIntyre, 2008). Through this methodology, academic researchers adopt the role of coaches and facilitators, training and supporting a team of community researchers to take a lead in design, delivery and implementation of findings from the research project. This approach therefore aims to realign power within the research process from academic researchers to community stakeholders (Cornwall and Jewkes, 1995), although the reality of enacting such a power shift can, in practice, require skilful navigation of complex relationships, systems and procedures.

            PAR follows a continual cycle of questioning, dialogue, action and reflection (McIntyre, 2008), ensuring that an open and transparent approach to learning is integral to the research process. The use of PAR has been shown to have wide-ranging benefits for participants, including the development of transferable skills, improved communication between stakeholder groups, and services that are more responsive to community needs (Gratton and Beddows, 2018; Newell and South, 2009; Squirrell, 2012). In relation to evaluation, participation of stakeholders can lead to enhanced capacity for the evaluation team, increased commitment from stakeholders and improved likelihood that the evaluation results will be used within an organisation (Smits et al., 2009).

            Get Talking is an approach to PAR developed and adapted over time by Staffordshire University. The community practice and arts-based interests of the CCU team were influential in designing an approach to research that could help them to work with community partners in reciprocal ways, to provide opportunities for their students to conduct and understand ethical and emancipatory approaches to engagement and research, and to address power inequalities. The Get Talking model is underpinned by PAR principles and process, and it can be depicted as three concentric circles, as illustrated in Figure 1. The Get Talking model emphasises the need for each of the circles to be of equal weighting wherever possible.

            Figure 1.

            The Get Talking model (Source: authors)

            Principles

            The principles underpinning the Get Talking approach are interrelated and mutually dependent. These are participation, inclusion, honesty, flexibility and action (Gant and Rowley, n.d.; Gratton and Beddows, 2018). ‘Participation’ is an ambiguous term, although it is acknowledged that participatory practice can be transformative (Ledwith and Springett, 2010). In the context of PAR, it refers to stakeholders having an active role in the research. High value is placed upon the existing experiences and knowledge of local people as assets, and, as such, Get Talking trains and supports a team of community researchers to engage in all stages of the research process. Get Talking is proactive in reaching community researchers who are typically excluded from the research process, or those frequently identified as research subjects, by engaging people most affected by the subject in question at the centre of the research process. In a Get Talking process, community researchers should have a high level of influence over the research, and therefore flexibility is essential to afford the group the opportunity to shape the questions, investigate emerging themes and challenge assumptions. Equally, as a participatory action research approach, it is vital that Get Talking research provides community researchers with opportunities to create change and recommend actions. Therefore, commissioners of any Get Talking programmes should be conscious of the parameters within which the community researchers can affect change, and should only adopt the approach where recommended actions can be seriously considered.

            Process

            Get Talking outlines a clear, non-linear process, which comprises planning, mapping and involving stakeholders, mobilising a community research team, listening to and learning from the community of interest, cross-checking findings, and planning and taking action. This reflects the PAR process of questioning, dialogue, action and reflection (McIntyre, 2008), and it allows for learning to be gathered and acted upon at regular intervals throughout the cycle. Community researchers are engaged in, and influence, all stages of the research process, resulting in co-produced research findings, and meaningful and impactful actions based on insights from within the team. This active involvement of community researchers often brings an additional need for flexibility of timescales. While outlined here as a cyclical process, some flexibility may be required to accommodate revisiting stages as the need arises.

            Creative techniques

            A further defining feature of Get Talking is its use of creative consultation techniques. While it is not uncommon for PAR to include a broad range of inclusive and engaging methods for data collection (Buhaenko et al., 2004; Chevalier and Buckles, 2013), Get Talking places greater emphasis on the use of creativity in data collection and dissemination. The use of creative consultation techniques improves the accessibility of the research process, especially for people with lower levels of literacy or with learning disabilities, or for people whose first language is not English. It also provides an opportunity for the techniques to be aligned with the subject matter of the research. For example, in previous projects, Get Talking has used maps to discuss the needs of a geographical area, and money boxes and toy money to engage people with research about poverty. Although for Get Talking creativity has a broad definition, it provides scope within research projects to embed creative and, at times, artistic elements into the research process to increase engagement, stimulate conversation and increase the accessibility of the research process and findings.

            Get Talking and Appetite

            Creative People and Places commissioned a national programme evaluation of all funded projects which the place-based evaluations feed into. In Appetite’s first phase, Get Talking formed the core methodology for its place-based evaluation. As a result, the aims of the place-based evaluation reflected those of the national programme evaluation in addressing the following questions:

            1. Are more people from places of least engagement experiencing and inspired by the arts as a result of Creative People and Places?

            2. To what extent has the aspiration of Creative People and Places for excellence of art, and excellence of the process of engaging communities, been achieved?

            3. Which approaches to engagement, inspiration and excellence have been successful?

            To deliver the project, an evaluation team comprised of one project manager and one academic researcher was appointed within CCU in Year 1. In addition, Appetite appointed a Critical Friend, an external consultant with extensive experience of the arts and the use of Get Talking as a methodology. Get Talking’s principled approach and clear process for participation was well aligned with the Appetite programme’s philosophy of community engagement in the arts, and it was therefore welcomed by the consortium as an approach to underpin its evaluation of the programme. Given that one of Appetite’s aims was to bring art to people who do not usually engage with it, it was apparent that working with community researchers would ensure that Appetite was able to gain unexpected insights into audience attitudes to art, and uncover lay knowledge and the views of people that otherwise may not be included in the research findings. Outlined below is the implementation of Get Talking for the evaluation of Appetite, the strengths of the approach, the challenges encountered during the first year, and the resulting pragmatic adaptations that were made to the approach in the second and third years. It is written from the perspective of the evaluation team, drawing on an interview with the Creative Director for Appetite in which the impact of using Get Talking was discussed.

            Implementation of Get Talking in Year 1

            It was important to both the evaluation team and Appetite that the evaluation was embedded into the programme and produced meaningful findings that could influence the future direction of the programme. In the first year, Appetite’s programme comprised an intensive, six-week Taster Menu, a programme of high-quality arts events, an immersive circus and a selection of ‘pavement cafe’ events in the city centre each weekend. It also included a large-scale outdoor event in a city park, followed by a series of events, including a children’s theatre production and a partnership with libraries encouraging parents to engage their children in creative storytelling. The programme was designed not only to attract new audiences to the arts in the city, but also to offer new inspiration and insight into the new types of art that could be produced within the city. Alongside the programme, each consortium partner recruited a paid Appetite Builder, community engagement workers with direct responsibility for building relationships between local people and Appetite. These Appetite Builders were employed by the consortium organisations to work directly with communities, some of whom were not directly engaged in art or cultural activity, and to support them to access tickets, attend events and reflect on their experiences.

            The evaluation and project teams were keen to involve local people with the evaluation. Participation, as a core principle of Get Talking, therefore provided a useful means of involving local people, with an interest either in the arts or in regenerating Stoke-on-Trent, in the Appetite programme by involving them in the evaluation process. Twelve community researchers were recruited to influence and inform the evaluation of the programme, and to gather feedback from audiences from across the city. The team included four Appetite Builders and eight volunteers who had an interest in art, or in working with and strengthening their own communities.

            The university team gained ethical approval for the evaluation in April 2013, and they provided full training and support to the community research team. During the initial weeks of the programme, all consortium partners and Appetite staff team members were trained in Get Talking. The training covered the principles underpinning Get Talking and how these related specifically to the delivery of Appetite. It also provided the partners with an opportunity to identify key stakeholders for the project, and to embed a collective understanding both of participatory approaches and of the impact of including community partners in all stages of the process within the consortium.

            The Get Talking short course, led by the evaluation team, was delivered one evening a week over the six weeks of the Taster Menu. The sessions provided participants with an opportunity to reflect on all stages of the evaluation research process, from identifying stakeholders and interviewing skills, to analysis of findings and action planning. Delivery of the training programme in this way provided a number of benefits. First, the Get Talking approach became integrated into the Appetite programme and provided community researchers with a live project through which they could test and refine their newly developed community consultation skills. Second, members of the community research team were able to get support from the evaluation team, and from each other, during the training sessions, which was crucial to their well-being and continued engagement with the programme. This was particularly important given the intensity of the data collection period, with events being held every weekend during the six-week Taster Menu period, and the numbers of face-to-face consultations, ranging from one hundred to two hundred at each event. A third benefit of integrating the Get Talking training into the live Appetite Taster Menu activity was that it enabled ongoing review and reflection on the findings as they emerged. However, although the community research team was involved in some analysis, the sheer volume of data meant that most of the feedback collected from the events was analysed by the evaluation team. Throughout the whole training programme, the research team modelled creative methods of engagement, providing the Appetite Builders with tools for engaging community groups and the whole team with training on use of some of the more common creative consultation techniques.

            Data collection during Year 1 utilised creative consultation techniques that reflected the themes of the art available to audiences. During the Taster Menu, the evaluation team and community researchers were located in a ‘Travelling Tea Room’, a large and elaborately decorated outdoor tent, which helped to set the creative environment for audiences to engage with evaluators in conversation about the arts. The evaluation team designed bespoke creative tools fitting with the themes of the 14 performances and arts events. These tools included ‘quali-tea’ pots, where audience members were asked to vote on definitions of quality art using sugar cubes (Figure 2a), a six-foot arts wish tree to identify aspirations for art in Stoke-on-Trent (Figure 2b) and acrostic poems to gather audience feedback on a poetry performance. Each tool was designed to be creative and attract engagement. They also provided a means of collecting feedback, whether through written comments or voting. Community researchers were available to support audiences to write their feedback comments to ensure that people with lower literacy skills were not excluded from the process.

            Figure 2.

            Examples of creative consultation techniques used during Appetite’s Taster Menu (Source: intervention by Nicola Winstanley and Sarah Nadin, photographs by Jenny Harper). (a) Quali-tea pots; (b) Arts wish tree

            As well as data gathering, creative techniques were also used to illustrate stories of people’s engagement with Appetite. Evaluators worked with art students at Staffordshire University, and with audience members, community hubs and volunteers, to produce creative case studies (Figure 3). Examples of these included a textile banner, illustrations and videos. Each creative case study was accompanied by a short text to provide further detail.

            Figure 3.

            Creative case studies (Source: intervention by Nicola Winstanley and Sarah Nadin, photographs by Jenny Harper). (a) Textile banner; (b) Illustrations

            Challenges

            Get Talking provided a useful foundation for the first year of Appetite’s evaluation – one that secured the role of evaluation in the Appetite programme and created a culture with audiences of debating and discussing the arts in Stoke-on-Trent. However, several constraints experienced during Appetite’s first year resulted in a pragmatic evolution of the evaluation methodology.

            Smits et al. (2009) argue that increasing participation in evaluation can increase the team’s capacity to deliver. However, while community researchers, with support from the evaluation team at Staffordshire University, gathered feedback from relatively large audiences in Year 1, the sustainability of supporting them to engage with large audiences over a long period of time was questionable. The community researchers were a team of volunteers and, as such, management of the group was time and resources intensive. The university evaluation team reflected that during the planning stages, insufficient consideration had been given to the additional capacity and resources required to effectively support a team of community researchers to meaningfully engage in a research project of the size of the Appetite evaluation.

            Equally, participatory evaluation ‘is and should be without the controls and checks that may protect certain stakeholders’ interests over others, which may be found in some more traditional forms of commissioned research’ (Squirell, 2012: 94), and participation in PAR involves people having ownership and control of the research process (McIntyre, 2008; Buhaenko et al., 2004). The Appetite team and consortium were supportive of the Get Talking approach to evaluation, and they welcomed the creative engagement of audiences through the participatory evaluation. However, as a Creative People and Places programme, Appetite was under increasing pressure for their evaluation to address specific questions, and to include larger sample sizes and extensive demographic data. Ledwith and Springett (2010) point to conditions that create barriers for authentic participation in organisational development. These include short-term funding; the introduction of top-down targets and indicators; some forms of knowledge, usually ‘scientific’, being revered over others; and the continued drive towards evidence-based practice, resulting in linear processes that limit opportunities to learn from dialogue and reflection. Equally, MacDowall (2015: 1) points to the increased dependence of cultural policy in recent years on the production of ‘accurate and relevant data’. It cannot be denied that some of the conditions described by Ledwith and Springett (2010) were present during the latter stages of Year 1, in particular the increased importance placed on participants’ demographic data and the specific needs of funders. This presented an ethical dilemma for the evaluation team. By labelling our approach to evaluation ‘Get Talking’, we were implying that local people would have an element of control over the whole process, and the team understood that participation can, if mismanaged, cause damage to communities and the reputation of organisations.

            Perhaps the main challenge for Get Talking as a participatory approach was the scale of the evaluation programme. In comparison, the community research team was relatively small, with many having other family or work commitments. They were therefore unable to engage in all stages of the evaluation, and their activity became restricted mainly to data collection, the listening stage of the Get Talking process. For well-funded projects such as this, there are ethical questions of how much we are willing to ask of a small team of committed, but unpaid, volunteers. While PAR seeks to address ethical concerns of using outside researchers, Minkler (2004) points out that it is often the ‘professionals’ working on a project that are perceived by participants as having the most to gain from collaborative approaches to research, retaining paid posts and producing research outputs. As an evaluation team, we reflected on the ethics of requesting so much voluntary engagement of community researchers. It was clear that a more pragmatic approach was required through which we could offer a more realistic menu of opportunities for participation in the evaluation process (Gratton and Beddows, 2018).

            A further challenge for the evaluation was the funder’s requirement for larger sample sizes than the conversational Get Talking approach could provide. As such, the evaluation team adapted the methodology to incorporate larger sample sizes and to supplement the largely qualitative data collection from Get Talking with quantitative and demographic data, reducing the time committed to collecting qualitative data, and increasing capacity for design and analysis of survey data.

            Finally, the creative techniques used to evaluate Appetite events were intended as a way of starting a conversation that would be recorded afterwards by the researcher in written format. This approach ensured that details of conversations were captured without being intrusive for the research participant. Semi-structured interviews, however, take significant time with large audiences and this, combined with the need to gather feedback from larger samples, meant that we were relying increasingly on the creative consultation tools as a way of recording findings, reducing the emphasis on conversation as a means of learning. Creative consultation techniques therefore became a recognisable and attractive feature of the Taster Menu evaluation, with audience members seeking out the wish tree and other tools to offer their feedback.

            These challenges (and opportunities) led to discussion and critical reflection among the evaluation team at the end of Year 1 of the Appetite evaluation. The university team felt under pressure to continually design creative tools, to attend events to support community researchers and to analyse findings. In addition, feedback from community researchers collected as part of the Get Talking short course indicated that although they enjoyed the engagement with Appetite, the schedule was intensive and exhausting. Appetite’s Critical Friend also played a significant role at this stage in supporting the evaluation team to reflect on the Get Talking process and implementation, how well it addressed the needs of the national evaluation, and the adaptations required to reach the sample sizes required and to maintain the well-being of the evaluation and community research teams. Overall, it was clear that, despite the evaluation team’s commitment to the principles of Get Talking, some pragmatic adaptations were needed for Year 2.

            Pragmatic adaptations

            During Year 2, the evaluation team introduced pragmatic adaptations to all three of the Get Talking elements – principles, process and creative techniques – driven by the need to deliver an evaluation that meets the needs of the funders, the programme staff and the communities engaged with Appetite.

            Get Talking’s use of creative techniques, both to engage people with the evaluation and to tell the story of Appetite through case studies, was one of the most successful and recognised elements of the Year 1 evaluation. However, in Year 1, most of the creative tools and techniques had been created by the evaluation team or students for the case studies. Reduced staffing capacity at Staffordshire University in Year 2 meant that this arrangement was unsustainable. Equally, the artistic quality of the creative tools was inconsistent. Therefore, two socially engaged artists were commissioned to develop the creative techniques and, in many cases, to conduct the consultations, allowing the evaluation to mirror the artistic quality of the programme. The tools created by the artists were of high quality, and they attracted audiences to engage in discussions about the art on offer and, through support from the trained community research team, added capacity to the research team for the collection and analysis of data. Refocusing the evaluation methodology on the creative engagement techniques, and bringing in artists to deliver that element of the work, provided the research team with additional capacity to introduce more traditional research methods, including online surveys and focus groups, increasing the sample size as required by the programme and funders (Figure 4).

            Figure 4.

            Artists’ creative consultation tools (Source: intervention by Nicola Winstanley and Sarah Nadin; photographs by Jenny Harper, 2015). (a) Ribbon wheel; (b) Firegarden light umbrella

            Reflecting on the Get Talking model outlined in Figure 1, creative techniques therefore became a significant element of the Appetite evaluation. However, the core principles underpinning PAR took differing levels of priority in light of the challenges, resulting in an adapted version of Get Talking better suited to the needs of a large-scale arts evaluation.

            The principles of participation and inclusion remained an important element of the evaluation, and community researchers continued to play a role in the collection of data by supporting delivery of the artist-led creative consultations. However, the evaluation of Years 2 and 3 became far more driven by a sole lead researcher who was more directive in delivery of the evaluations, in line with the Creative People and Places questions, and who ensured a balance of artistic creative consultations and online, post-event surveys to maximise sample sizes and demographic analysis. A crucial change was that the programme was no longer promoted as a community researcher-led project, therefore bringing expectations in line with the realities of working within a large-scale programme with clearly defined evaluation parameters.

            The addition of online, post-event surveys was contrary to Get Talking’s inclusive principles, which traditionally avoid the use of techniques that rely on levels of literacy to participate. However, given the continued use of creative consultation techniques at Appetite’s events and performances, people were still offered an opportunity to provide feedback in a less traditional, more conversational way. The use of surveys provided the evaluation team with larger sample sizes and data through which qualitative data could be cross-checked.

            However, the research team were acutely aware that, despite the broader reach and improved artistic quality of the creative consultation techniques, the reduced involvement of community researchers raised questions about how participatory the evaluation had become. Cornwall and Jewkes (1995) make the distinction between collaborative and collegiate participation. Our aspiration for the Appetite evaluation was that it would adopt a collegiate approach to research, in which the community research team were equal partners in the research process. However, in Year 2, our methodology evolved to accommodate the needs of the Appetite team and funders to maximise learning within the context of a Creative People and Places programme. Our amended model was a collaborative approach in which ‘researchers and local people work together on projects designed, initiated and managed by researchers’ (Cornwall and Jewkes, 1995: 1669). We were able to use this collaborative approach to value the contribution of local people in the process, within the wider constraints of the broader evaluation framework.

            The wider impact of Get Talking on the Appetite programme

            The wider impact of Get Talking and the participatory principles that underpinned the evaluation of Appetite was felt beyond the confines of the programme’s evaluation. During an interview in May 2020, Gemma Thomas, Creative Director for Appetite, outlined the ways in which Get Talking impacted on Appetite, including building a culture of learning, increasing audience engagement, building an inclusive framework for critical reflection and extending community involvement in decision-making. This section outlines each of these impacts, drawing upon quotations and findings from the interview.

            Building a culture of learning through an inclusive framework for critical reflection

            Get Talking is underpinned by a cycle of questioning, dialogue, action and reflection. For three years, this provided Appetite with a valuable structure for action learning. With the active involvement of audiences and an ‘army of community researchers’, evaluation became recognised as a team activity, and seen as ‘a form of learning, rather than an opportunity to point blame at [what] hasn’t worked so well’. This built strong relationships and trust, and it enabled Appetite to be more responsive to community needs. In addition, the training offered to all consortium partners at the start of the Appetite programme embedded an understanding of Get Talking at both strategic and operational levels with Appetite. This, in turn, led to a shared commitment from partners to contribute to the reflective cycle, further enabling Appetite to be responsive to emerging findings.

            Increasing audience engagement with Appetite

            The use of Get Talking for the evaluation had a positive impact on audiences’ engagement with the programme. ‘Get Talking helped facilitate unbiased conversations with local people about what [art] they might like to see’, and the creative engagement techniques adopted ‘helped to fire everyone’s imagination’. For audiences, the evaluation of Appetite remained distinctive in its use of creative and artistic techniques for data gathering and, over the first three years of the programme, it helped to initiate and sustain a conversation about arts and culture among people who had previously had little experience or interest in the arts.

            Extending the reach of Get Talking and Appetite

            As a result of Get Talking training, the Appetite team, Appetite Builders and consortium partners gained new skills in participatory research and creative consultation techniques. The Appetite Builders used these skills and techniques to reach communities who initially had little experience of engaging with arts and culture, and to enhance their own practice: ‘this was something [Appetite Builders] could take away from the work they did as well’. This enabled them to build and sustain relationships with Appetite’s ‘community hubs’, generating ideas and consulting on options for art in local communities. Consortium members also applied their Get Talking training to other aspects of their work, ensuring that ‘the learning did not just stay with Appetite, but spread further into the city’.

            Extending community involvement in Appetite

            Community researchers became the ‘foundations on which the [Appetite] programme was built’. Working with community researchers enabled Appetite, and Staffordshire University as evaluators, to understand Appetite in the ‘context of the community’. For Appetite, ‘working with community members as decision-makers was a new thing’, and the use of Get Talking in the first year of the programme provided a framework that Appetite could use to co-produce more of their programme with community and audience members as they moved into Year 2. Appetite delivered an annual strategic programme of large-scale, high-quality, outdoor arts. At the end of the first year, the Appetite team used the principles underpinning Get Talking to inform a framework for community engagement with the strategic programme. They initially recruited two representatives from each community hub to join a group to discuss ideas for the next strategic programme, which became the Supper Club. Founded on participatory principles of co-production and inclusion, this group met monthly, over supper, to discuss new ideas, experience ‘tasters’ of art or visits to other areas of the country to experience new art forms, and to co-produce and evaluate large-scale, Appetite arts events. Now in its seventh year, strategic decisions about the programme are still made through consultation with the Supper Club. This model of co-production is firmly embedded into Appetite, and it has been used to attract funding for expansion of the programme into an adjacent area of low arts engagement.

            Reflection and conclusion

            In this article, we have highlighted the challenges of participatory research as part of a large-scale arts programme. To build on our experiential learning, we acknowledge both the limitations and successful aspects of the work, and we have used this learning to shape further research projects since 2013. We recognise that full participation in research which sets out to be PAR is not always realistic. As Chevalier and Buckles (2013: 177) state, participation in research is more accurately summed up as ‘doing the right thing at the right time with the right people, with success in mind’, and is dependent on a number of conditions, including the workload for participants, resource availability and the complexity of the issue being researched. In the Appetite case study above, the limited ability for community researchers to take control of the research, and the need for larger sample sizes, resulted in pragmatic adaptations to the original conception.

            On reflection, we would have been clearer with community researchers at the start of the evaluation process about the aims and objectives of the evaluation, and the collaborative parameters within which community researchers operated. The university research team had been optimistic about their ability to adopt a more collegiate approach in Year 1. However, in reality, the constraints of aligning local activity with the national programme evaluation, along with the impact of an intensive activity schedule on everyone involved with the evaluation, led to review and change.

            While the views of community researchers contributed towards the engagement of professional artists in Year 2, we did not evaluate the impact that this shift had on the community research team, or gain feedback on whether this move supported their well-being or felt disempowering as the control over the evaluation tools and process moved away from them and to the commissioned artists. In hindsight, community researchers could also have been involved in the process of artist recruitment, which would have helped to retain a sense of audience and community ownership over the evaluation and programme.

            Using our learning from Appetite, we implemented several changes to how we deliver Get Talking with communities and partners. For example, Get Talking is now primarily used with smaller independent projects, through which community researchers can take a more active role in the entire research process. Equally, Staffordshire University has recognised the potential negative impacts of being a community researcher, in relation to well-being, capacity or emotional impacts. As a result, we now offer all community researchers access to our staff well-being services, including online support pages and a telephone counselling service. The university research team have also implemented a system of informally interviewing community researchers following Get Talking projects to evaluate the project’s impact on them.

            Despite the challenges and adaptations, the principles of the approach used in Appetite have shaped a culture of co-production and community decision-making that reaches far beyond the original conceptions of Get Talking as a way of involving local people in a participatory evaluation. The participation at the heart of the PAR process, and therefore Get Talking, continues to be used by Appetite to give local people significant influence over the programming of art in the area. Many of the existing community researchers continue to work with Appetite as volunteers or as Supper Club members. Therefore, although Get Talking’s role in the evaluation of Appetite has come to an end, participatory principles and approaches at the heart of the methodology continue to have a positive impact on Appetite and how they work with communities to shape art in the city.

            Acknowledgements

            The authors would like to thank the Appetite team, in particular Gemma Thomas, Appetite Director, and the consortium members. Special thanks go to Kate Gant, Critical Friend to Appetite, and to the community researchers who committed their time and energies to the evaluation of Appetite.

            Declarations and conflicts of interest

            Research ethics statement

            The authors declare that research ethics approval for this article was provided by Staffordshire University ethics board.

            Consent for publication statement

            The authors declare that research participants’ informed consent to publication of findings – including photos, videos and any personal or identifiable information – was secured prior to publication.

            Conflicts of interest statement

            The authors declare no conflicts of interest with this work. All efforts to sufficiently anonymise the authors during peer review of this article have been made. The authors declare no further conflicts with this article.

            References

            1. Appetite. 2013. Business Plan for 2013–2016. Stoke-on-Trent: Appetite.

            2. Buhaenko H, Butler V, Flower C, Smith S. 2004. What Men and Women Want: A practice guide to gender and participation. Oxford: Oxfam GB.

            3. Chevalier JM, Buckles DJ. 2013. Participatory Action Research: Theory and methods for inquiry. London: Routledge.

            4. Cornwall A, Jewkes R. 1995. What is participatory research? Social Science and Medicine. Vol. 41(12):1667–76. [Cross Ref]

            5. Emadi-Coffin B. 2008. Get Talking: Community participation and neighbourhood learning. Widening Participation and Lifelong Learning. Vol. 10(3):30–4. Accessed 29 November 2021 http://eprints.staffs.ac.uk/id/eprint/5141

            6. Gant K, Rowley J. n.d.. Participatory Appraisal: Practitioner pack. Accessed 25 November 2021 http://blogs.staffs.ac.uk/ccu/files/2012/10/participatory-research-resource-pack.pdf

            7. Gratton N. 2014. Get Talking with Appetite: An evaluation of Year 1. Accessed 25 November 2021 http://blogs.staffs.ac.uk/ccu/files/2014/08/Appetite-evaluation-2013-2014-report.pdf

            8. Gratton N. 2015. Get Talking with Appetite: Year 2. Accessed 25 November 2021 https://blogs.staffs.ac.uk/connections/files/2021/06/End-of-year-two-report-final.pdf

            9. Gratton N. 2020. From engagement to strategy: The journey towards a civic universitySengupta E, Blessinger P, Mahony C. University–Community Partnerships for Promoting Social Responsibility in Higher Education. Bingley: Emerald Publishing. p. 105–20

            10. Gratton N, Beddows R. 2018. Get Talking: Managing to achieve more through creative consultationStout M. From Austerity to Abundance? Bingley: Emerald Publishing. p. 141–60

            11. Ledwith M, Springett J. 2010. Participatory Practice: Community-based action for transformative change. Bristol: Policy Press.

            12. MacDowall L. 2015. Introduction: Making culture countMacDowall L, Badham M, Blomkamp E, Dunphy K. Making Culture Count: The politics of cultural measurement. London: Palgrave Macmillan. p. 1–8

            13. McIntyre A. 2008. Participatory Action Research. London: Sage.

            14. Minkler M. 2004. Ethical challenges for the “outside” researcher in community-based participatory research. Health Education & Behavior. Vol. 31(6):684–97. [Cross Ref]

            15. Newell CJ, South J. 2009. Participating in community research: Exploring the experiences of lay researchers in Bradford. Community, Work & Family. Vol. 12(1):75–89. [Cross Ref]

            16. Smits P, Champagne F, Blais R. 2009. Propensity for participatory evaluation. The International Journal of Learning. Vol. 16(6):611–32. [Cross Ref]

            17. Squirrell G. 2012. Evaluation in Action: Theory and practice for effective evaluation. Lyme Regis: Russell House.

            18. Vincent P. 2010. Quality Streets Feedback. Accessed 25 November 2021 http://blogs.staffs.ac.uk/ccu/files/2014/01/Quality-Streets-Short-Report.pdf

            Author and article information

            Journal
            rfa
            Research for All
            UCL Press (UK )
            2399-8121
            20 January 2022
            : 6
            : 1
            : e06101
            Affiliations
            [1 ]Associate Professor of Community and Civic Engagement, Research Innovation and Impact Services, Staffordshire University, UK
            [2 ]Research Impact Manager, Research Innovation and Impact Services, Staffordshire University, UK
            Author notes
            Author information
            https://orcid.org/0000-0001-7456-793X
            https://orcid.org/0000-0002-9700-9161
            Article
            10.14324/RFA.06.1.01
            6ce21327-f704-4d6e-a518-4dc7708224ef
            Copyright 2022, Nicola Gratton and Jackie Reynolds

            This is an open-access article distributed under the terms of the Creative Commons Attribution Licence (CC BY) 4.0 https://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, distribution and reproduction in any medium, provided the original author and source are credited.

            History
            : 07 July 2020
            : 17 August 2021
            Page count
            Figures: 4, References: 18, Pages: 14
            Categories
            Article

            Assessment, Evaluation & Research methods,Education & Public policy,Educational research & Statistics
            culture,co-production,evaluation,arts,participation

            Comments

            Comment on this article