Introduction
Public engagement, as an integral part of the research process, is increasingly being emphasised by policy makers, research institutes and funders alike as a form of research ‘impact’ (Langdridge et al., 2019), in attempts to accelerate public understanding of research, expand the impact of research on societies and enhance the accountability of research practice (Stern, 2016). It has been defined as a ‘two-way process’ through which the ‘activity and benefits’ of research are shared with the public (NCCPE, 2022). In addition, the public’s contributions can in turn be fed back into the research to generate new knowledge and future research agendas (Percy-Smith and Carney, 2011). Public engagement, therefore, is capable of stimulating dialogue and critical thinking, as well as offering a platform for the exchange of expertise between researchers and the public, by allowing diverse groups to contribute to research findings and their impact (Ball et al., 2021).
The arts are an important vehicle through which public engagement can occur. They have the capacity to make complex research findings more accessible, broach sensitive subjects (for example, through metaphor), create spaces where assumptions can be challenged, communicate nuance, ambiguity and emotion, as well as stimulate reflection on topics that audiences may otherwise not have encountered (Ball et al., 2021; Born and Barry, 2010; Boydell and Gladstone, 2012; Fraaije et al., 2022). While critiqued as a form of disciplinary ‘objectification’, whereby one discipline is harnessed to meet the agendas of another (Reinsborough, 2020), arts-based approaches to public engagement have continued to proliferate, and there is evidence of a wide range of mediums being employed to communicate research findings through the arts, including: film and animation (Langdridge et al., 2019; Toye et al., 2020; Vaughn et al., 2013), photography (Perez et al., 2016), theatre (Feldman et al., 2013; Lewando Hundt et al., 2011; Reinsborough, 2020), games (Wendler and Shuttleworth, 2019), audio drama (Weston, 2019), dance (Austin, 2016; Woodgate, 2018), music (Byrne et al., 2018), visual artworks (Bevan-Jones et al., 2017; Cook et al., 2017) and creative writing (Byrne et al., 2018; Miller and Brockie, 2015; Roeser et al., 2020).
Immersive art installations, which have been defined as a ‘synthesis of art event and art work’ (Nollert, 2003: 4) refer to physical or virtual interactive spaces in which ‘multiple art techniques are employed’ (Ball et al., 2021: viii). They have been used as a means through which to communicate research findings to the public on diverse topics. Tischler et al. (2020), for example, toured UK venues to disseminate their research findings on dementia using a ‘pop-up’ cartoon-style living room space (complete with artwork by research participants) and interactive games. Drumm et al. (2015) toured a large ‘singing sculpture’ (a 9.5-tonne giant aeolian harp), with associated talks, exhibitions and workshops with a range of audiences to raise awareness of acoustics science. Through touring cultural sites, scientific venues, festivals and cafes, both immersive art installations were able to engage large and diverse audiences who would not have otherwise encountered the work being presented. While facing both logistical and conceptual challenges in the creation of the installations (Drumm et al., 2015), both projects were ultimately positively evaluated by audiences, with demonstrable impact on public attitudes and views.
Inspired by the nascent literature on arts-based public engagement, and on immersive art installations in particular, this article presents evaluation data from a four-year public engagement project, I:DNA (2018–22). I:DNA was a UK-based touring multimedia art installation with associated public engagement events (including theatre, poetry, crafts and film). It was created in collaboration with STAMP Theatre and Media Productions CIC (www.stamproductions.co.uk/) as a means through which to engage the public with the findings of the Imagining Futures research programme (www.warwick.ac.uk/imagining_futures), funded by the Wellcome Trust. This research explored the lived experiences of people with inherited conditions, and their views about genetic screening. Through documenting the range and reach of public engagement with I:DNA, the challenges and opportunities of generating research impact through public engagement will be explored, as well as the implications for future public engagement and public engagement evaluation.
Imagining Futures research programme
Imagining Futures was a mixed methods research programme (2017–21) that explored the lived experiences of people with a wide range of genetic conditions, and their families, illuminating their reflections on expansive genomic medicine. Conditions were selected for the study based on their prevalence and impact, and, as such, represent a broad spectrum of presentations, ranging from intellectual impairment (fragile X syndrome) to physical disability (spinal muscular atrophy), and included both life-limiting (cystic fibrosis) and treatable conditions (haemophilia, thalassaemia). Using 177 in-depth interviews and national surveys of 1,723 respondents across these five conditions, the study charted the way that participants’ lived experiences informed their attitudes towards genetic screening, and their views on the potential prevention or amelioration of their condition in future generations.
The research revealed considerable ambivalence among participants towards future genetic screening programmes for their condition (Boardman and Clark, 2022; Boardman and Hale, 2018; Boardman et al., 2019). While many supported the idea of expanded reproductive autonomy, they simultaneously raised concerns about the capacity of the general public to make informed reproductive decisions regarding conditions that they may never have encountered (Boardman, 2021; Boardman et al., 2019), and the inherent difficulties in determining which conditions are severe enough to warrant their prevention through genetic technologies (Boardman and Clark, 2022).
The data that emerged from the Imagining Futures research programme, while rich and varied, were also deeply emotive, touching on highly sensitive topics, including the lived realities of disability and the social and ethical implications of disability prevention, as well as complex concepts around genetic inheritance, screening and use of reprogenetic technologies.
I:DNA
I:DNA (2018–22) was a project funded by the Wellcome Trust (Research Enrichment – Public Engagement award) to increase public engagement with the Imagining Futures research programme. By providing a public platform for the voices of people living with inherited conditions, I:DNA was designed to act as a catalyst for critical reflection on the broader social and ethical implications of expanding genomic medicine, which is of relevance to the whole of society.
I:DNA comprised a multimedia immersive art installation, developed collaboratively between Felicity Boardman’s research team, STAMP Theatre and Media Productions CIC, visual artist Esther Appleyard-Fox (https://appleyardfoxart.com/), and Entify Theatrical Design and Creation. Conceptual and physical production began in August 2018, and took place over 14 months, with the launch of the installation taking place in September 2019.
I:DNA was designed to be an immersive experience, based around the themes of journey, identity and filtering as metaphor for some of the debates that currently surround genomic medicine. The installation took visitors on a ‘journey’ through a space physically reminiscent of an airport check-in, with an entry arch resembling a body scanner, labelled ‘DNA check-in’ (see Figure 1). At the heart of the installation was a 3m x 4m x 6m aluminium de-naturing double helix sculpture, with luggage of different shapes and colours hanging from its spokes. The luggage represented both the DNA nucleotides and the ‘baggage’ that all individuals carry with them as part of their DNA (Figure 1). A short film documenting the making of I:DNA is available at www.warwick.ac.uk/idna/creation.
Within the installation, a soundscape was projected (www.warwick.ac.uk/idna/sculpture), alongside filmed portraits of diverse faces. The soundscape directly communicated the words of the research participants, using verbatim text from the research interviews spoken and sung in contrasting musical styles.
Following a two-day preview on the University of Warwick campus, the installation began its tour in September 2019, visiting FarGo Village (Coventry) as part of the British Science Festival. This was followed by Oxford Science and Ideas Festival (October 2019), Coventry Cathedral (as part of ESRC Festival of Social Science, November 2019) and Millennium Point in Birmingham (February–March 2020). The tour concluded with a nine-month residency at Leamington Spa Art Gallery & Museum (May 2021–January 2022) as part of their 20-year anniversary ‘Picture of Health: Art, Medicine & the Body’ exhibition.
In addition to launch events at each location (where the creative and research teams gave talks and answered questions from the public) several parallel public engagement activities took place around I:DNA. These included:
A filmed talk about the research, and an interactive live-streamed discussion. Due to the Covid-19 pandemic, I:DNA’s physical tour was suspended from March 2020 to May 2021. During this time, the research and creative team together produced several online resources. These included a filmed talk by Boardman, followed by a live-streamed discussion panel around the talk, involving three of the original research participants (who have different genetic conditions), and with public interaction. These can be viewed at www.warwick.ac.uk/idna/events/online.
Two craft workshops for children (November 2021), one held in a primary school and one in Leamington Spa Art Gallery & Museum. These events, entitled ‘I:DNA: Who Am I?’ provided children with an age-appropriate talk by a geneticist (in-person and available online). The children were then invited to create a piece of artwork reflecting their identities. The children’s artworks were collated by artist Tammy Woodrow, and installed in the art gallery as part of the I:DNA exhibition (see Figure 2). Details and pictures from the event can be found at www.warwick.ac.uk/idna/events/whoami.
A poetry workshop, spoken word performance and anthology, as part of the Resonate Festival, which celebrated Coventry’s year as UK City of Culture. Entitled ‘DNA: Our Stories’, and in conjunction with poet Nigel Hutchinson, a poetry workshop was held at Leamington Spa Art Gallery & Museum, where aspiring poets and creative writers were supported to write poetry inspired by the installation (which was in the gallery at the time). The poetry was performed at Warwick Arts Centre in January 2022. Films of both the workshop and the performance, as well as a link to the published anthology of poetry can be found at www.warwick.ac.uk/idna/events/ourstories.
An ‘invisible theatre’ performance (Millennium Point, March 2020). In conjunction with Birmingham City University, seven drama students familiarised themselves with interview transcripts from the research. They mingled with audiences at I:DNA’s launch event, each adopting the character of one of the research participants, and telling their story, without revealing their ‘true identities’ as actors. You can read one drama student’s account of conducting this invisible theatre at https://www.bcu.ac.uk/conservatoire/about-us/news/blog/applied-theatre-genetic-disease-project.
An online game, iDNAKnowing, created by visual artist Esther Appleyard-Fox in collaboration with the research team and BRiGHTBLaCK productions. It is set in a futuristic airport setting, mirroring the experience of the physical art installation, and increasing its accessibility to those who could not visit I:DNA in person. Using excerpts from the research interviews, the game presents different hypothetical screening scenarios, and players can choose their route through the game by making decisions about what information they would want from their genome. At the end of each decision pathway, the player hears the voices of actors performing verbatim text from the interview transcripts relevant to the decision that the player made during the game. The game is no longer available, but images and a description of it are available at www.warwick.ac.uk/idna/idnaknowing.
Children’s artwork in situ, I:DNA exhibition, Leamington Spa Art Gallery & Museum (Source: Ben Robinson)
Where possible, evaluation data were gathered from all of these events, including attendance numbers and evaluation forms.
Evaluating the impact of arts-based public engagement activities
Evaluating the impact of public engagement activities is a complex task, with little available guidance and a dearth of robust evidence supporting its conduct (Ball et al., 2021; Langdridge et al., 2019; STFC, 2017). According to the Research Excellence Framework (REF) – the system used to assess research quality in higher education institutions – impact can be understood as ‘an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia’ (UKRI, 2022: n.p.). However, in relation to public engagement activities, impact can be challenging to assess, given that methods of public engagement evaluation are often emergent and developed in a context-sensitive way during the course of the activity, rather than designed a priori. In addition, impacts can be ‘subtle’ (making a ‘contribution’ to change, rather than instigating change alone), hard to isolate from other influences, subject to change and revision, and also often occur a considerable time after the public engagement activity has been completed (Reed et al., 2018). For these reasons, capturing public engagement impact may require the collection of detailed and longitudinal evaluative data, which can be both challenging and costly to obtain. Indeed, lack of available funding to conduct rigorous evaluation has significantly hampered the development of an evidence base for the evaluation of public engagement impact, particularly in relation to arts-based practice. Boydell and Gladstone (2012), in their review of arts-based public engagement evaluation, have further argued that positivist methods of measuring impact, for example, by quantifying attendance rates, ‘dwell time’ – the length of time an audience engages for (STFC, 2017) – and evaluation scores may be at odds with the methods of engagement used within arts-based practice. Indeed, public engagement can produce tangible impacts that are not captured by standard quantitative indicators, suggesting that commonly used evaluative tools may be unsuitable in arts-based public engagement.
Despite the challenges associated with impact evaluation, Ball et al. (2021), through their review of the literature, and building on the framework of arts-based knowledge translation developed by Kukkonen and Cooper (2017), outline three core impact areas that have been described or evaluated within the arts-based public engagement literature.
Public engagement as a goal in itself, incorporating changes to audience awareness, the stimulation of dialogue and debate, increased accessibility of, and engagement with, the research, and increased profile of research study and institution.
The improvement of quality or effectiveness of both current/future research and public engagement activities through capacity building, the creation of new knowledge and the production of high-quality artistic outputs.
Wider impacts (including behaviour changes within communities, and impacts on local culture, as well as practice/service changes).
Using Ball et al.’s (2021) framework to categorise our public engagement impacts, this article will present evaluation data and outputs gathered from the I:DNA project. The data were collected in a variety of ways, representing a range of formats, including: attendance numbers, website/video viewing figures, qualitative evaluation (through online forms, postcard comment boxes and emails sent to the project email address), and interview data, as well as artistic creations (for example, poetry and artwork). The challenges and opportunities associated with evaluating a complex and multidimensional project such as I:DNA will be discussed. Recommendations and reflections on future arts-based public engagement will be outlined, including the use of the arts as both a public engagement activity output, and as a source of evaluation.
Methods
Different methods were used to both obtain and analyse evaluation data across the various iterations of the I:DNA project.
Data collection
I:DNA immersive art installation
I:DNA was exhibited at six venues across four locations in the Midlands and South East of England between 2019 and 2022: Oxford, Coventry (three separate venues), Royal Leamington Spa and Birmingham. The majority were associated with science in some way (British Science Festival, Oxford Science and Ideas Festival, Millennium Point), or were social science festivals (ESRC Festival of Social Science). Through its exhibition at Leamington Spa Art Gallery & Museum, however, I:DNA also appeared for nine months within an artistic context as part of an exhibition.
To capture numbers of visitors, as well as their reactions and responses, visitors at all locations (except Leamington Spa Art Gallery & Museum) were counted on entry by the exhibition curator (research assistant in public engagement) using a footfall traffic counter, and were invited to provide an evaluation using a postcard. The postcard had images of the installation on the front and an open space to write their reflections on the back. The staff at Leamington Spa Art Gallery & Museum curated the installation as part of the wider exhibit in the gallery, and provided approximate visitor numbers for this period. Due to Covid-19 health and safety concerns associated with visitors sharing pens to complete evaluation cards, we displayed a QR code at this venue, which linked to an online version of the evaluation form. The gallery also provided us with the feedback they received for the whole ‘Picture of Health: Art, Medicine & the Body’ exhibition, which included some specifically related to I:DNA.
The decision was made early in the evaluative planning not to collect demographic data in our evaluation forms, so as to facilitate capture of immediate reactions, focused on a single question, without privacy concerns associated with the collection of personal data (Burchell, 2015). While this strategy proved successful initially, the approach was later revised so that data relating to age and gender could be collected to identify under-represented audiences, and to inform the design of subsequent public engagement activities. In total, the installation received 663 in-person visits across the Coventry, Birmingham and Oxford sites between 2019 and 2020, who collectively left 193 evaluation cards (see Table 1). These evaluation cards typically contained one to three sentences on visitors’ reflections, both on the subject matter of I:DNA and on their experience of the installation. While considerably shorter than is typical for qualitative data, they were nevertheless rich and varied in their content. In addition, six visitors – selected at random – participated in short ‘vox pop’-style filmed interviews after leaving the exhibition. While everyone approached agreed to an interview, resource constraints precluded the collection of further filmed interviews. Examples of the vox pop interviews can be viewed at www.warwick.ac.uk/idna/visitorviews. Eight visitors left contact details to participate in a follow-up interview about their experience; however, attempts to reach them were ultimately unsuccessful.
Engagement levels with I:DNA relative to evaluations received
| Dates and duration | Visitors | Visitor evaluation* | |
|---|---|---|---|
| I:DNA installation tour dates | |||
| British Science Festival 2019 (Coventry) | 11 to 12 September (2 days) | 326 in-person visits | 93 (53.1%) postcards and interviews |
| Preview, University of Warwick campus | 13 September 2019 (5 hours) | ||
| Evening showcase event, BSF 2019 FarGo Village Takeover | |||
| Imagining Futures Oxford (Oxford Science and Ideas festival) | 26 to 28 October (open approx. 5 hrs per day) | 75 in-person visits | 32 (42.6%) postcards and interviews |
| Coventry Cathedral, part of ESRC Festival of Social Science 2019 | 1 November 2019 (evening launch event) | 185 in-person visits | 29 (14%) postcards and interviews |
| Open to public 2 to 7 November (2 to 5 hours per day, excluding Sunday 3 November) | |||
| Millennium Point, Birmingham | 18 February to 3 March 2020 (open approx. 6 hours per day) | 77 in-person visits | 39 (50.6%) postcards and interviews |
| Leamington Spa Art Gallery & Museum | 20 May 2021 to 31 January 2022 (184 days, initially 5 hrs/day 3 days/week, increased to 6 days/week autumn 2021) | 18,400 † | 11 online evaluation forms; estimated 128 visitors provided an evaluation to curators in relation to Picture of Health exhibit, 32 specifically related to I:DNA (<1%) |
| I:DNA associated events | |||
| I:DNA Online, part of ESRC Festival of Social Science 2020 | |||
| Pre-recorded film (23 min.) about the Imagining Futures research | Film online from 29 October 2020 | 97 (online) visits on 11 November, 1,897 in total | 17 (2%) online evaluation form |
| Live presentation and panel discussion | 11 November 2020 (1 hour) | ||
| Recording of live panel discussion available online | 12 November 2021 | ||
| Children’s Craft Activity, I:DNA Who Am I?, ESRC Festival of Social Science 2021 | 85 children participated across both locations | 0 (0%) online evaluation form | |
| Local primary school | 12 November (2 hours) | 526 visits to event web page | |
| Drop-in workshop at Leamington Spa Art Gallery & Museum | 20 November (5 hours) | ||
| Web page with film of geneticist talk and gallery of artwork | November 2021 | ||
| Poetry Workshop, Event & Anthology, DNA Our Stories | Workshop – 7 in-person | 8 (14%) email after the event | |
| Workshop at Leamington Spa Art Gallery & Museum | 11 December 2021 (3 hours) | Event – 15 performers (3 online) and 34 audience members | |
| Live (and live-streamed) spoken word event (Resonate Festival) | 22 January 2022 (90 minutes) | 67 web page views | |
| Web page with films of workshop and event, link to anthology available online | February 2022 | ||
| I:DNA online resources | |||
| I:DNA webpage ‡ www.warwick.ac.uk/idna | Active from September 2019 | 3,432 visits | |
| iDNAKnowing online game | Active from November 2019 | 2,047 online visits | 7 (3%) online evaluation forms 2019–2020** |
| Total | 19,197 in person | 268 | |
| 7,376 online |
*Evaluation in multiple formats (postcards, emails, online evaluation forms, in-person interviews or comments). †Visitor and evaluation numbers are based on estimates provided by Leamington Spa Art Gallery & Museum. ‡Visits to main I:DNA web page, including visits to subpages, from September 2019 to 24 May 2022. Figures do not include visits to YouTube that came via other sources (for example, YouTube searches or via STAMP CIC webpages). **Evaluation forms only gathered for first year game active.
Associated activities
For the supplementary I:DNA activities, the following evaluation data were collected:
viewing figures for the Imagining Futures Talk (www.warwick.ac.uk/idna/events/online)
viewing figures and questions/comments received during the subsequent live-streamed panel discussion
website visits to the iDNAKnowing game
attendance figures at the children’s craft activity event
attendance figures for the poetry workshop
audience figures and evaluation data from audience members at the live poetry event.
Alongside these qualitative and quantitative evaluation data, the artistic outputs of the events (the children’s artwork, the poetry, and the films of all the events), were also treated as contextual data for evaluation of impact.
Data analysis
Given the wide range of data types collected for the evaluation, quantitative (descriptive statistics) and qualitative (thematic analysis) approaches to analysis were employed. Summary figures for the quantitative attendance/viewing figures, with the proportion of participants leaving evaluative data across the various iterations of I:DNA are detailed in Table 1. For the qualitative data, key elements of Braun and Clark’s (2006) thematic analysis approach were conducted. Initially, this involved familiarisation and categorisation of the data to identify the features of the data (Braun and Clarke, 2006). The resulting broad categories included: ‘delivery of the installation/activity’, ‘visitor satisfaction’, ‘responses and reactions to the content of I:DNA’, ‘changes in knowledge/awareness’ and ‘changes to attitudes and/or behaviour’.
After this initial grouping of the data, a thematic analysis was undertaken using coding, to identify patterns of meaning within the text, including both semantic (what is said) and latent (what broader conceptualisations underpin what is said) themes (Braun and Clark, 2006). This same approach of familiarisation, categorisation and thematic analysis was also applied to analysis of the artwork, poetry and films produced by the project. While methods for the qualitative analysis of artworks are still emerging, thematic analysis has been demonstrated as a means through which the meaning making of social actors can be accessed (Lakh et al., 2021). In order to conduct the thematic analysis, the same coding framework that was used for the evaluation forms was applied. Artistic outputs were first categorised by type and topic, before broad semantic themes were identified that cut across the artworks – for example, ‘identity’ – before undertaking finer coding that revealed latent codes, such as ‘embodied self’ and ‘genetics and belonging’. Through a process of reviewing and comparison of the themes across the data sources and contexts, the themes were further refined and developed until all of the collected data could be accounted for. Finally, the coding framework containing the themes and subthemes was mapped on to the three key domains of public engagement impact identified by Ball et al. (2021) to explore their range. Responses to the installation content and its impacts on awareness were mapped to the domain ‘public engagement as a goal in itself’, whereas data and subthemes linked to the broad theme ‘delivery of the installation/activity’ were mapped to Ball et al.’s (2021) second domain, ‘improving the quality/effectiveness of research and public engagement’. Finally, data and subthemes relating to knowledge and behaviour change were mapped to the domain ‘wider impacts’. The distribution of the analysed data across these three core areas of public engagement impact was then explored through the creation of a meta-matrix. The results of this deductive analysis are presented according to Ball et al.’s (2021) three domains, with exploration of the themes and subthemes contained within them.
Results
Attendance and evaluation figures
In total, 663 people visited the I:DNA installation in person at the science and social science festival locations prior to the Covid-19 pandemic (September 2019 to March 2020), and we received 193 comment cards, 29.1 per cent of the total visitors. The gallery estimates suggest that at least 18,400 people visited the ‘Picture of Health: Art, Medicine & the Body’ exhibit at Leamington Spa Art Gallery & Museum, in which I:DNA was displayed (May 2021 to January 2022). We received strikingly low levels of audience evaluation (n = 11 online evaluation forms, 32 spoken evaluations recorded by curators) at this location, particularly considering the duration (nine months) of the I:DNA exhibition there. This may have been due to the necessary shift from physical (pen and card) to virtual evaluation forms (with the use of a QR code, which was not highly visible) necessitated by the Covid-19 pandemic, restrictions on the numbers and timing of visitors inside the space at any one time, and an overall reduction in gallery attendance by members of the public.
For the online event (ESRC Festival of Social Science), there were 45 viewers of the live discussion and 819 views of the filmed talk during the course of the festival (November 2020); 4 people asked questions live, and 3 in advance of the event, and evaluation forms were received from 10 people (4 per cent of those who viewed the panel discussion). In total, the web page containing both the talk and panel discussion has been viewed 1,897 times.
The online game was visited 267 times between November 2019 and November 2020, with 7 evaluation forms received in that time. Over 2,000 people have interacted with the game in total.
The arts and craft activity was attended by 55 children, and a further 30 participated in a school setting. The poetry workshop had 7 participants, and 15 people performed at the poetry event (3 online), with 34 audience members. Evaluations were received from 4 audience members, and from 4 of the performers.
Domain 1: Public engagement as a goal in itself
Ball et al. (2021) highlight that there are a range of impacts that emerge from public engagement when public engagement is treated as a goal ‘in itself’. These impacts include increases in audience awareness about an issue, the creation of debate and dialogue, changes in the profile of the research, and increases in research engagement and accessibility (Ball et al., 2021: 56).
Changes to knowledge, attitudes and perspectives
There was evidence within our evaluation data that through an engagement with I:DNA, our audiences gained new knowledge and insights into perspectives that they had not before considered. For some, this change in knowledge and perspective was explicitly stated in their evaluation:
It expands understanding and appreciation of the complexity of genetic conditions to link the scientific research with artwork, and it asks important questions. (Visitor I:DNA, Coventry Cathedral)
I’ve never really thought about the ethics of screening before, it makes me wonder what will come next … what will we start screening for? (Visitor I:DNA, Oxford Science and Ideas Festival)
It has allowed me to look from a new perspective about something I have not fully thought about. The effect of genetic disorders in families – it’s a difficult one and it is useful to be able to have this new perspective. (Visitor I:DNA, British Science Festival)
I didn’t realise I share 60% of my DNA with a banana! Or that a test could reveal so much about me. Made me look at it all and what could be done with it very differently. (Poetry event audience member)
Accessibility and transferability of key concepts
As well as new knowledge and perspectives, there was also evidence that audiences were able to relate the content of I:DNA to broader questions about the future of genomics (for example, the value of the ‘natural’ versus the engineered, and the harms and benefits of increasing knowledge), demonstrating that they were able to link the content explicitly presented to them to complex debates:
Genetics shape humans and nature. I can’t imagine what the future looks like if people have power to control that. (I:DNA online panel discussion viewer)
Genetic testing can be like opening Pandora’s box, but can also be an enormous relief and allow you to plan for the future. (Visitor I:DNA, British Science Festival)
It’s not always best to know about any conditions you may have. Sometimes it’s best not to know, especially if you can’t do anything about it. (Player of iDNAKnowing online game)
There was also evidence that the format of the I:DNA installation rendered a complex topic more accessible, facilitating engagement with the content:
It portrays lives that are both ordinary and extraordinary which makes the production very relatable. (Visitor I:DNA, Coventry Cathedral)
It was mind boggling to see something so complex broken down so simply. It expands understanding and appreciation of genetic conditions. Makes it understandable for everyone. (Visitor I:DNA, Coventry Cathedral)
Loved it! Important advances communicated in a different way making ideas accessible to a wider audience. Really made me think! (Visitor I:DNA, Oxford Science and Ideas Festival)
Affective impact
There was also evidence of ‘affective impact’ (Langdridge et al., 2019) from visitors, suggesting that I:DNA, particularly the soundscape, had a deep emotional impact on visitors:
The stories were both heart wrenching & heart-warming. (Visitor I:DNA, Coventry Cathedral)
I never thought I would be so touched and impressed with an exhibition quite like this one. (Visitor I:DNA, British Science Festival)
I didn’t expect it to be as emotional as it was. The music and words were both poignant and hopeful, you can’t help but be moved by them. (Visitor I:DNA, Oxford Science and Ideas Festival)
While the emotional impact of public engagement activities can be an important component of memory, facilitating retention of the event’s content (Langdridge et al., 2019), Gardner et al. (2021) have highlighted that arts-based approaches may be especially vulnerable to producing unintended emotional consequences for audiences. This potential for emotional impact becomes particularly ethically complex when public engagement activities are installed in well-used, and perhaps unexpected, public spaces. While this use of public space generally increases the diversity of audiences, unintended distress, such as that reported by Scott-Dearing and Pegram (2019) in relation to the ‘Departure Lounge’ installation (exploring bereavement, and exhibited in a shopping centre) can be heightened when contact with the installation was unanticipated or unavoidable. Like ‘Departure Lounge’, I:DNA also dealt with sensitive topics, including experiences with genetic conditions, reproductive decision making and genetic testing. While no explicit evidence of emotional harms appeared within our evaluation data, nothing can be known about the impacts on those who did not contribute data.
Domain 2: Improving the quality and effectiveness of research and public engagement
Ball et al.’s (2021) second area of public engagement impact relates to the possibility of improved research and public engagement activities in the future through capacity building and implementation support. In relation to I:DNA, the wide range of artistic outputs used (theatre, song, poetry, artwork, film, game, sculpture) enabled a comparison across different types of arts-based public engagement delivered across different venues and mediums, to contribute to the emerging public engagement evidence base.
Complexity of media and message
Overall, visitors to the I:DNA installation were enthusiastic about the installation, with its use of sculpture, film and song: ‘fantastic’, ‘amazing’, ‘beautiful’ and ‘inspiring’ were all common adjectives that were used to describe it. Many viewers expressed appreciation for the use of mixed media. For instance, there were many positive comments about the singing:
Loved the singing! The song is very interesting. (Visitor I:DNA, British Science Festival)
An amazing project. Turning the spoken words into song was very nicely done and really set it apart from other installations I have seen. (Visitor I:DNA, British Science Festival)
Lovely music – great way to convey the words & stories. (Visitor I:DNA, Oxford Science and Ideas Festival)
However, alongside the positive reactions, some found the use of airport imagery as a metaphor for genetics confusing, and had sought explanation from the curator:
Beautiful to look at. Airport analogy a bit baffling, but was explained! Thank you. (Visitor I:DNA, Coventry Cathedral)
Once I had had the baggage explained, I could really appreciate the purpose of the exhibition. (Visitor I:DNA, Coventry Cathedral)
The decision was made to include data highlighting an extensive range of ideas and experiences, reflective of the entire Imagining Futures data set, and while some visitors appreciated the myriad of perspectives presented by the installation, some found this volume of perspectives confronting and overwhelming. For these visitors, the meaning was diluted as they struggled to identify the ‘main message’, and they instead wanted greater clarity regarding a simpler ‘take home’ message:
I wasn’t sure what the message was. At places, it looks like a positive one, but the music and tones and at times the speakers felt negative. (Visitor I:DNA, Millennium Point)
It has acknowledged the reality of genetic screening without being biased in any way, which is ironic as art is meant to motivate people to think about the artist’s perspective. (Visitor I:DNA, Oxford Science and Ideas Festival)
I felt confused about the film’s message and uncertain about how this would make people feel about those who are different because of their genetics. (Visitor I:DNA, British Science Festival)
Powerful in parts but a little confused in message in places. (Visitor I:DNA, Leamington Spa Art Gallery & Museum)
While the arts have the capacity and freedom to provoke reactions in audiences, research findings are typically presented in a way that is faithful to the range and complexity of data collected. This can produce conflicting aims between the collaborating artists and researchers (Watermeyer and Chubb, 2018); however, it can also be considered part of the dynamics of the creative process (Bartlett, 2015).
Artistic freedom, experience and engagement
Thematic content analysis of the poetry and artwork created in the workshops brought to light the challenges associated with engaging members of the public in the creation of artwork on a topic with which they did not have prior experience. Indeed, while Gameiro et al. (2018) successfully co-opted their stakeholders as de facto artists through the creation of drawings on the topic of infertility, the topic was one already of great significance to the women participating. While the children’s art workshops (November 2021) focused on age-appropriate content relating to genetics – such as identity and relatedness – a broad range of artwork emerged (see www.warwick.ac.uk/idna/events/whoami). Some of this artwork replicated genetic imagery (such as the DNA helix and DNA base pairs), while other children focused on representing their name or physical characteristics (for example, a blue eye, the letter ‘A’). However, the vast majority opted to create artwork relating to topics that interested them, which included astronomy, marine biology, sport and video game/television/film characters. Their focus on these topics as the basis for their art often meant that it was hard to assess their engagement with the research topic areas.
Similarly, the broad range of topics – and interpretations of genetics – covered by members of the public and poets who participated in the poetry workshop and performance event (December 2021–January 2022, collated and published as an anthology: www.warwick.ac.uk/idna/events/ourstories/anthology_idna_our_stories.pdf) meant that it was often challenging to trace engagement with the research themes for the purposes of evaluation of impact. However, evaluations received after the event (from both the performers and the audience members) suggest that the experience was nevertheless personally impactful around the theme of genetics:
I have been so inspired by I:DNA that I have continued to develop the themes of genetics and identity from the project in my work. This has led me to receiving a recent commission from [charity] to develop some community based art … Thank you so much for what has been a life-changing opportunity. (Poetry workshop participant and performer)
It’s not something I’ve given much thought to before. This event has prompted me to think about genetics, my own and in general. (Poetry event audience member).
Locating the impact: process and outputs
This feedback highlights some of the challenges of treating the artworks produced through arts-based public engagement both as an output of the project and as a source of data for impact evaluation. While the research themes were challenging to identify once they had been transformed through the medium of poetry and artworks, evaluations received suggest high levels of engagement with the content of I:DNA. As such, the artistic process of transformation, through which the outputs were created, may provide more accurate insights into levels of engagement with the research than an analysis of artistic outputs themselves. As one poetry workshop participant commented:
I went on a journey with this. Even though I’ve never had a genetic condition, through the stories of their lives, I found my own. (Poetry workshop participant)
As noted by Gameiro et al. (2018), post-artworks discussion may be a useful forum for the collation of engagement and impact data; however, this was not possible following the poetry workshop and event, due to time constraints and Covid-19 restrictions.
Finally, an analysis of our evaluation data across the various venues and iterations of I:DNA reveals patterns relevant to the development of future arts-based public engagement, and evaluation planning. While short, postcard-style evaluation forms (or an electronic version of these) were our most commonly used method for the collection of evaluation data, completion rates varied dramatically. While our online event (I:DNA online, November 2020) generated the highest level of viewing (relative to the length of the event), the rate of evaluation received was considerably lower (7 per cent) than for our in-person exhibitions, particularly those with a dedicated curator on hand – British Science Festival (53.1 per cent completion), Imagining Futures Oxford Science (42.6 per cent completion) and Millennium Point (50.6 per cent completion). The only exception to this was Coventry Cathedral, where only 14 per cent of visitors provided an evaluation, despite curation.
The context of engagement
The different types of audiences attracted by each venue are likely to have influenced this range of evaluation rates. Whereas those attending the festivals or science-oriented venues were expecting to see installations based around science and academic research, I:DNA’s placement in a cathedral (Figure 3) meant that most visitors found the installation unexpectedly.
Although I:DNA did receive some local media coverage while in situ in the cathedral (a local radio interview with our exhibition curator), our evaluations indicate that most visitors encountered the installation by chance. These different ways of approaching the installation impacted on the evaluation data in various ways: first, it meant that I:DNA reached new audiences (other than festival goers/science enthusiasts), and second, it meant that the installation was interpreted differently than it was in other contexts. For example, when in a cathedral setting, the singing associated with the installation was described as ‘spiritual’, ‘haunting’ and ‘ethereal’ – words not used elsewhere in the evaluation data. However, there were also practical implications. The noise and footfall within the cathedral impacted the degree to which some visitors could engage with the piece:
I’m afraid I struggled with the choral and the voiceover and the ambient noise. But I enjoyed the idea. Thank you. (Visitor I:DNA, Coventry Cathedral)
Whereas others found the context and the installation content hard to reconcile:
I’m not sure why this is here. DNA baggage from God?? I think we should treasure what we have been given. (Visitor I:DNA, Coventry Cathedral)
Similarly, while some visitors to the ‘Picture of Health’ exhibit (Leamington Spa Art Gallery & Museum, May 2021–January 2022) – which also included works from artists such as Damien Hirst – were pleased to see contemporary artworks on display, others expressed disappointment at the lack of fine art, specifically paintings. These data highlight the significance of venue and context to the interpretation of touring immersive art installations, and the subsequent evaluation data. Environmental features of the venues (lighting, noise, physical space, decor, atmosphere), the availability of a curator to answer questions, and the expectations of the visitors, all played a role in the way that I:DNA was experienced, and the degree, and nature, of the visitors’ engagement.
Domain 3: Achieving wider impact
The final area of impact identified through arts-based public engagement by Ball et al. (2021: 63) relates to impact at the level of ‘individuals, communities and the practice/policy landscape’.
The key way in which this was realised through the I:DNA project was through behaviour change (or intended behaviour change) among audiences, stakeholders and participants.
Changes in attitudes, knowledge and behaviours
Lack of previous knowledge about genetic conditions, and of advances in genomic medicine, prompted intended behaviour change among several participants. While most of these related to plans to find out more information and become ‘educated’, others relayed a change in attitude and perspective that would affect their behaviour towards disabled people:
I’m now going to learn more about genetics and biology. (Visitor I:DNA, Oxford Science and Ideas Festival)
It made me realise how much I didn’t know about the basics of our existence, our genes. I will be making sure I educate myself now! (Visitor I:DNA, British Science Festival)
I’d never considered that living with a genetic condition could be a positive experience. This has changed my attitude a lot and the way I relate to disabled people. (I:DNA visitor, Millennium Point)
This type of impact was not only observed among audiences visiting I:DNA, but also among stakeholders – people with genetic conditions themselves – who took part in the research interviews and panel discussion in November 2020:
It [I:DNA] has re-affirmed to me that people with [condition] have important stories to tell, and … relevant stories to tell. I enjoyed it more than I thought I would. I’ve shied away from stuff like this before. But I would definitely feel more confident talking about it now, you know through [condition charity] for those newly diagnosed. (Email evaluation from stakeholder with genetic condition)
While our data indicate intentions to change behaviour across audiences and stakeholders, the lack of long-/medium-term follow-up data limits how far actual changes in behaviours can be evidenced. Follow-up interviews were attempted with eight visitors who left their contact details; however, efforts to contact them were ultimately unsuccessful. As much of the impact of immersive installations such as I:DNA appears to occur some time after the event (as spontaneous contact by two audience members some months after the installation attests), evidence of this important impact is lost without long-term data. This highlights the need for the development of methodologies that can overcome the logistical, financial and ethical barriers to long-term follow-up.
Discussion
This article has presented evaluation data (attendance/participation data and a qualitative analysis of 263 evaluation forms and postcards) collected from a multimedia, four-year public engagement project, I:DNA. The analysis of these data has been mapped to the three core areas of public engagement impact identified by Ball et al. (2021), and recommended for evaluation by their guidance on the conduct of arts-based public engagement (Gardner et al., 2021). While other frameworks and guidance for the evaluation of public engagement activities exist (for example, NCCPE, 2017; Reed et al., 2018; STFC, 2017), Ball et al.’s (2021) framework focuses on the operationalisation of impact from public engagement, a concept that currently carries significant currency among researchers, research organisations and institutions, and is high on the agenda of research funding bodies (UKRI, 2022). The findings demonstrate that I:DNA achieved demonstrable impact in all three core areas identified by Ball et al. (2021); however, the process of conducting the evaluation raised several important observations and findings regarding public engagement impact evaluation that may be of relevance to future researchers and artists wishing to undertake arts-based public engagement.
In the first instance, responses to our requests for evaluative data were highly variable across the different components and iterations of I:DNA. Response rates were highest among those visiting the physical installation in person (where a dedicated I:DNA curator was on hand to answer questions about the installation, and to hand out evaluation postcards), and lowest among those attending online events (see Table 1). Covid-19 and the switch to online evaluation forms also negatively impacted response rates. Moreover, while our evaluative data were primarily qualitative, the quantity, length (often two or three sentences) and depth of responses received were atypical for qualitative research, where profundity and meaning are prioritised over breadth and representation. Despite this potential limitation, however, the data provided were nevertheless rich and nuanced, demonstrating a high degree of insight and engagement with the research topic. Thus, while data saturation was not achieved (or necessarily aimed for, given that this evaluation did not constitute primary research), the diversity in interpretations and responses to the piece were striking. Attempts were made to increase the representativeness of those engaging with I:DNA, and, consequently, the representativeness of evaluation data. For example, basic demographic information (age, gender, ethnicity) was requested from those completing evaluations, and subsequent I:DNA events were targeted at under-represented groups – for example, children – through the art workshops. Despite these efforts, however, the sample was invariably limited to those who chose to engage with an artistic installation (in its various guises), and it cannot be described as representative of the UK population. Future work is needed to identify and reduce social barriers to engagement with arts-based events, as well as to better understand how the various harms and benefits of non/engagement are borne across social groups (Bone et al., 2021).
A further area that limited the validity of our impact evaluation was the failure of our methods to gather long-term follow-up data. While eight I:DNA visitors left contact details, attempts to arrange follow-up interviews (two to four weeks after their visit) were unsuccessful. While nothing can be known about the reasons for this non-response, it is possible that visitors’ reflections on, interest in, and memories of their visit to I:DNA diminished over time, making them harder to access without the stimulus of the physical installation. This may be particularly true for an installation exploring a topic not of immediate relevance to the majority of visitors’ lives. Indeed, while there was evidence of different types of impact from our participants (in terms of increased awareness, knowledge and understanding, and intended behaviour change), we were not able to explore how far awareness and knowledge were retained in the long term, nor whether intended behaviour changes occurred in practice. We became aware that some behaviour change had occurred, but evidence of this emerged sporadically when the participant/audience member initiated contact with the team (for example, a poetry workshop participant who went on to explore the themes from I:DNA within their next piece of work). Future research may usefully engage in methodological development for medium-/long-term follow-up for arts-based public engagement, particularly those conducted with public audiences lacking a pre-existing connection to the research topic. These audiences are likely to be among those most difficult to retain in long-term evaluative research.
Finally, analysis of the I:DNA evaluative data has demonstrated the challenges and opportunities associated with collecting, and integrating, diverse data types. The artworks created by public audiences, for example, were treated both as a project output – capable of generating further public engagement (such as the inclusion of the children’s artworks within the I:DNA installation and the poetry spoken word event) – and as evaluation data in and of themselves. The thematic analysis of these artworks, however, was challenging, given their diversity (both in form and content), and the varying levels of abstraction used by the artists. Indeed, as has been observed within other analyses of artworks (for example, Hall, 2015; Kisovar-Ivanda, 2014; Pascuet et al., 2010), we relied on a significant degree of researcher interpretation as we attempted to ‘unravel’ the complex processes of translation and transformation (from research topic into artwork) in order to locate and trace the engagement and impact. The use of focus groups and/or qualitative interviews alongside this analysis would likely have been illuminating, and would have supplemented our interpretations; however, the resource implications of this additional data collection and analysis impeded its adoption.
Conclusions
Despite the methodological limitations described above, I:DNA produced diverse and novel evaluative data evidencing high degrees of both public engagement and impact, including changed perspectives, emotional resonance and a desire to learn more. It attracted significant numbers of attendees (both online and in person), and it stimulated insightful and sophisticated dialogue and debate with audiences previously unfamiliar with genomics and genetic conditions. By using the arts, I:DNA was able to communicate ambiguity and nuance using a multiplicity of perspectives, and through a range of mediums and contexts. While the focus on public engagement impact among research funders and policy makers may require researchers to pay greater analytic attention to these consequences and outcomes of arts-based public engagement, the need to also critically evaluate the processes by which the public engagement occurs, and the influence of these processes on the elements of public engagement that get translated into impact (for example, behaviour change), has been demonstrated through this evaluation of I:DNA. The need for methodological development to capture and evaluate these processes, and how they shift and reformulate over time (short-, medium- and long-term), is now of critical importance.