Standards Consultation Summary Report (November 2017)

1. What are the Public Involvement (PI) Standards and Indicators? Good public involvement helps ensure that research is for patient and carer benefit and ultimately leads to improvements in treatments and services. The Standards are a set of statements that describe what good public involvement in research looks like. Together they provide a framework for improving the quality and consistency of public involvement in research. The Indicators are statements of good practice that you can demonstrate you are working towards, meeting or exceeding the standard.

2. Why and how was the consultation carried out? An online consultation was conducted to allow review of, and feedback on, a set of draft standards and associated indicators by a wide range of stakeholders. The consultation responses will provide the basis for redrafting the standards and indicators.

3. Who responded? 677 online responses were submitted along with 3 responses to the easy read version. The responses came from a cross section of stakeholders responding as individuals and 347 on behalf of groups. This included 15 responses from international organisations. The proportions of these responses are illustrated below in Table 1 and Figure 1. TABLE 1: TABLE SHOWING THE ONLINE SURVEY PARTICIPANTS CLASSIFICATION FOR INDIVIDUAL RESPONSES. Answer Choices (Individuals could select

as many options as they liked) Public, e.g. service user, patient, carer, consumer User researcher (including survivor researcher) Person from a voluntary sector/charity Researcher / academic Other research role (including research manager, commissioner, funder, advisor) Clinician/practitioner/service provider or manager (including social worker, health care worker) Public involvement lead/specialist

21/11/17

1

Percentage of individual responders (593) 57.34%

Number of responses

6.24%

37

17.88% 22.26% 10.29%

106 132 61

8.09%

48

18.89%

112

340

Source of Group Responses Research Organisation

16%

Patient/carer group, service user group or organisation Charity (research or other)

23%

15% 10%

Healthcare organisation including residential care home University

23% 13%

Other

FIGURE 1: CHART REPRESENTING THE SOURCE OF GROUP RESPONSES TO THE ONLINE SURVEY.

4. Summary of the survey results Respondents completed sliding scales (from 1-10) within the survey to indicate either their support for or against the standards and the indicators associated with each standard. Respondents could also add free text responses to give context and add comments to their scores. A research team analysed these free text responses to identify key issues and concerns across the responses. The researchers also met with a specially convened sub-group of the NIHR Research Design Service North East (RDS NE) public involvement consumer panel to get their input to the initial analysis. The summary findings were presented back to the subgroup of the NIHR RDS NE consumer panel as a way of finalising themes that were generated from the data. Finally, the sub-group of the consumer panel identified a set of recommendations and assisted in the production of this report, deciding what to include and helping to make sure the information was clear and avoided jargon. Before scoring and commenting on the indicators for each standard, respondents were asked for their views on some broader issues. These questions looked at the set of draft standards, to make sure nothing was missing but also to understand the potential difference the standards would make to individuals and organisations. Question 1: The Standards describe what good public involvement in research looks like.

Average score given

10.00 8.00

Responses for the draft standards 7.95 7.80 7.69 7.44 7.53 7.34 7.61 7.22 7.20

6.00 4.00 2.00 0.00

Level of Agreement with Standards

Academic/researcher Clinician Charity and Voluntary Sector Did not identify Other research infrastructure Public Involvement Lead Public User researcher Overall Average

FIGURE 2: BAR GRAPH REPRESENTING THE AVERAGE SCORES FROM EACH STAKEHOLDER GROUP ILLUSTRATING SUPPORT FOR THE DRAFT SET OF STANDARDS. 21/11/17

2

Overall there was support for the set of standards from across all stakeholder groups, as shown in Figure 2 above. There were no further suggestions for additional standards. The overall average score out of 10 was 7.54 from 411 responses. Question 2: The Standards provide a framework for improving the quality and consistency of public involvement in research.

Average score given

Responses for standards providing a framework 10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00

7.85 7.23 7.52

8.00 6.62

7.08

7.75

Academic/researcher 7.28 7.42

Clinician Charity and Voluntary Sector Did not identify Other research infrastructure Public Involvement Lead Public User researcher

Provide a Framework

FIGURE 3: BAR GRAPH SHOWING SUPPORT ACROSS STAKEHOLDER GROUPS THAT THE STANDARDS WOULD PROVIDE A FRAMEWORK TO HELP IMPROVE QUALITY AND CONSISTENCY OF PUBLIC INVOLVEMENT IN RESEARCH.

There was a range of responses from disagreement (0) through to strongly agree (10) but the majority of responses were in agreement that the draft standards would provide a useful framework to help improve the consistency and quality with an overall average score of 7.42 out of 10 from 408 responses. Question 3: What difference, if any, will it make to you to have a set of National Standards for public involvement in research? Responses to this question ranged from positive to negative. Positive views included: • The standards will operate as a useful reference and as guidance for best practice, bringing clarity and standardisation. • They will help establish and/or legitimise public involvement as a required component of research, providing support for justifying public involvement, and the related funding. • Patients and the public will benefit from standardisation of remuneration, knowing what to expect, acknowledgement of their ‘rights’ and support for when things go wrong. • Standards may promote a more comprehensive inclusion of public involvement throughout the research process. There was some scepticism about the difference the standards would make in practice. This was mainly because of uncertainty around who would implement them and how they would be monitored. Negative views included: • A few respondents saw the standards as making no difference to them at all. • More directed at organisations than individuals. • There was overlap with existing guidance. • Similar practices were already being followed in their work. 21/11/17

3

Question 4: Whose responsibility is it to implement the Standards? Respondents saw a wide range of organisations and/or professionals, as being responsible for implementing the public involvement standards. These included: government and research institutions, research organisations, funders, universities, researchers, public involvement leads/specialists, charity/volunteer organisations and/or patient/carer groups. Several people commented that ‘everyone’ or ‘everyone involved in all stages of the research process’ should be responsible for implementing the standards. Some respondents added further detail on the different roles they felt the respective parties should play in implementation. Different levels of responsibility were identified that groups could and/or should have e.g. responsibility for monitoring adherence to the standards. Responses for the remainder of the survey questions are summarised on the following pages under each of the standards.

5. Recommendations Overall recommendations 1. Standards and their indicators need to use plain English and avoid jargon to prevent ambiguity. 2. Clarity over meaning of words such as meaningful, representative, diverse, competence needs to be addressed and consistency applied in the use of any terminology. 3. More detail is needed on how these standards will be implemented, resourced, monitored and evaluated. 4. Supporting information is needed on whose responsibility it will be to implement and monitor the standards. 5. Clarity on how the standards will connect with existing public involvement guidance is needed. For further updates please sign up to the standards network https://sites.google.com/nihr.ac.uk/pistandards/standards-network

The full report will be made publically available early 2018.

21/11/17

4

Standard 1 - Inclusive Opportunities Standard 1 pr and accessible for range of people ch.

r

r, for

Overall the comments and scores for this standard were positive. The mean score for this standard was 7.63 and average scores for the individual indicators ranged from 7.37 to 8.10 out of 10. Respondents felt that this standard in particular offered a systematic way to collect evidence of persistent and common barriers “It would enable organisations to have a clear framework for public involvement and for those involved provide guidance for challenging or holding accountable those who do not proactively involve/ listen to / engage with the public”. (Patient/Carer Group)

Comments were raised regarding the need for signposting to make people aware of what opportunities were available, to improve inclusivity. “Signposting offering opportunities has not been evident to the general public, who may have a specific interest in a condition or subject”. (Public)

Respondents found it difficult to see how to improve diversity, for example how to reach people who aren’t on the internet or people who are working. Case studies or examples would help. “It is crucial to embrace diversity and for research to be relevant and inclusive to all those involved, especially those who would have conventionally been the "subject”. (User researcher)

Many comments highlighted that, if public involvement is to offer inclusive opportunities, people may need support (such as financial, IT support, mentoring and support for mental health problems) to participate fully.

21/11/17

5

Standard 2 - Working together

Standard 2 and respectful r and ch. working r

for

fective

The mean score for this standard was 7.79 with the scores for the indicators ranged from 7.47 to 8.09 out of 10, with comments focused on the need for a respectful, co-created, defined relationship which would in turn promote respectful involvement. “The concept of working together is important – vital that involvement is viewed as a partnership, and that there’s shared understanding between organisations/individuals and involved public” (Charity) All groups of respondents highlighted that experience in public involvement ought to be valued. It should not be necessary to have new public involvement representatives for all work, the role of an experienced public involvement member could change but should not be disregarded. “Experienced PPI are an asset and can work with research organisations in several other ways, not least in capacity building”. (University) Some respondents went further to request that different roles for public involvement need to be defined. A concern raised by several respondents was that a public involvement member may not have made an identifiable contribution but still played a valuable role; they may contribute to an idea but not necessarily be taken up. Respondents found it unclear how such a contribution would be measured.

21/11/17

6

Standard 3 - Support & learning Standard 3 & undertaken with confidence and

Scores for the indicators for standard 3, ranged from 7.77 to 8.65 with an average of 8.07. This included the highest scoring indicator which was “We have a clearly identifiable point of contact for information and support” Respondents requested more information on how this standard fits with existing INVOLVE guidance and other existing standards. “A cross reference to other work currently underway in this area may be useful e.g. Going the extra mile”. (Research Design Service) More clarity was requested regarding remuneration and the potential tax and benefit implications this may have to participants. Additional comments made by the respondents highlighted the need to emphasise training and the funding required to support this. “Resources should be made available for researchers to involve people in the development of research proposals, this would include training and implementation of PPI”. (Research Organisation) The issue of recruitment also required some more detail, including the consideration for support and budgetary requirements and a specific indicator for ‘recruiting the right people’. “Recruitment with proper 'job' descriptions is a necessary precursor to this standard”. (Public) Finally, having specific details on what ‘emotional support’ entailed was an issue raised by several respondents.

21/11/17

7

Standard 4 - Communications

Standard 4 pr and r of

and

Standard 4 again had a predominantly positive response with scores ranging from 7.70 to 8.31 the mean score 7.97 out of 10. Specific feedback included adding ‘active listening’ to the definition of communication and a request for more detail about the benefits of feedback. The respondents felt there was some overlap around the topic of ‘diversity’ across standards 1 and 4 which requires clarification. Further clarification was also needed on the frequency, timeliness and content of updates. Additional comments highlighted the need for consideration of inclusion and diversity in relation to communication methods. More support was needed with writing in plain English. “As the public representatives should be drawn from a wide range of people some of whom may not have English as a first language, communication has to be clear and relevant and understood by all parties".(Public) “The language used sometimes in research can be off-putting and so it is essential that due attention is paid to this area”. (Charity)

Respondents also requested examples of what a communication plan, referred to in this standard, would involve in practice and case study examples illustrating how the different components of this standard can be achieved. We would like to see clear information on what the communications with people with learning disabilities & non-verbal communications looks like”.(Patient/Carer group) “How will you measure this? Who decides what is clear?”(Public Involvement Lead)

Finally, several respondents commented on the importance of recognising the time, work and resources required to enable effective communication.

21/11/17

8

Standard 5 - Impact

Standard 5 and r r

of ch.

Indicators on Standard 5 had consistently positive scores, the mean score was 8.04 with a range of responses from 7.97 to 8.11. Overall, respondents were unclear what was meant by ‘impact’. It was thought that a definition and examples of what impact should look like would be an essential part of this standard. “Impact means different things to different stakeholders”. (PPI Lead) Furthermore, there was some concern regarding measurement and whether this would mean subjecting the public contributors to more scrutiny and evaluation. “Impact is so important and therefore so is the need to measure it. However, it can be very difficult to assess impact and I think there will be barriers”. (Public) Additional comments from respondents focused called for details on how successful or unsuccessful impact is defined. “The public involvement process may have a positive or negative impact on the individuals involved in that process and surely this should be considered”. (University) When respondents were asked to comment on what was missing several key points were raised, mainly focused around the need to include outcomes, assessment of outputs and monitoring. Emphasis on openness, transparency, clarity of roles and responsibilities in relation to impact were also identified by the respondents.

21/11/17

9

Standard 6 - Governance Standard 6 of and implement, r for

heard, making. W and accountable

Standard 6 - Overall the comments and scores received about this standard were positive the mean score was 8.17 with the responses ranging from 7.76-8.48 out of 10. Many thought it was critically important to emphasise that adequate and appropriate governance is essential to public involvement in research. “Governance should ensure management structure is in place”. (Healthcare organisation) Questions were raised by many on what would be the implications of achieving or not achieving the standard; as well as how and by whom would they be monitored “If members of the public can use the standards to hold organisations/individuals accountable, there should be guidance on how they can go about this”. (Charity)

21/11/17

10