We had a great time with a group of Citizen Space users in London yesterday, kindly hosted by one of our Westminster customers, the Department of Health (DH). These user groups are a regular opportunity to catch up, see how others in similar roles are using Citizen Space to manage their consultation activity, and share ideas and tips.
We always come away from user groups with a host of new information and insight. Hearing people’s experiences ‘at the coalface’ is invaluable. And even if you weren’t there in person, you needn’t miss out on that wisdom – here’s 10 of our take-away points from the day:
1. Words matter. Write simply.
Several people stressed the importance of content design in successful consultation.
We had a fantastic presentation from DH’s Content Designer and Lead Editor, Juliet Brooker, about how their content team works with policy teams to make consultations as accessible and effective as possible.
And it was great to hear others endorsing the same message: when it comes to words, keep it simple! For instance, Marilyn Booth and Gwenny Whelan from the Department for Business, Energy and Industrial Strategy (BEIS) ran through lots of things they’d learnt about boosting participation – one of which was that ‘just being forced to write in a simple way does half the work for you.’
2. Improving consultation is not a one-team job – internal collaboration is a huge deal.
This was a theme that came up time and again over the course of the day: so much of what determines the success of a consultation is about internal collaboration, cooperation and buy-in.
Is your policy team bought in to the process? (There’s limited value to getting a whole pile of feedback if it will only be listened to with half an ear). Has the whole organisation got into the habit of telling you about upcoming consultations months in advance, allowing you time to plan well (rather than just pinging over a cursory email the day before one is due to launch)? Are you able to liaise successfully with your marketing or social media teams to promote consultations, or is it something that never really gets much of their attention? What do you do if you’re dependent on your IT team for anything ‘website-related’ but they’re already massively overloaded with other work?
So many of the blockers we heard weren’t technical or skills issues – instead, it’s about getting organisations as a whole more geared up for effective consultation in their internal culture and processes.
3. There’s no one-size-fits-all solution. Appropriateness is key.
Another repeating refrain on the day: no two consultations are the same, and it’s important to determine what will work best in each case, rather than being inflexibly wedded to a single, fixed approach.
Sometimes, content should be complex and detailed. Sometimes, it will help to use an open-ended response mechanism like Dialogue alongside a more traditional survey. Sometimes, a short, snappy survey is best; sometimes, you want a very involved, chaptered or routed questionnaire.
The key word there being ‘sometimes’. Understanding and adapting to the context and audience of each individual consultation is a crucial skill in designing appropriate, and therefore effective, involvement activity.
(An interesting detail from BEIS on this: they know that some consultations will generate many more responses than others [a large scale public consultation vs one targeted at a small number of expert stakeholders, for instance]. They use a ‘Gold’ assignation to identify cases where they expect a high volume of responses. This helps them manage timings, expectations and resources appropriately: if there are multiple ‘Gold consultations’ coming up, they can anticipate needing to bring in additional analysis capacity.)
4. Citizen Space helps boost participation rate – statistically
BEIS have been Citizen Space customers for a while. They know they like using it but they wanted to (in their words) ‘find out if it really helped’. As in, does Citizen Space actually improve their consultation activity and/or success in a tangible way? They did some thorough investigation of their statistics, traffic patterns etc and we were treated to a wealth of interesting findings.
Just to quickly highlight a couple: when they first started using Citizen Space, they were getting a completion rate of around 7%. They were happy with this at the time – but they now average a completion rate of 25%, and a high point of 52%: a vast improvement! And evidence that creating engaging consultations is an art and a science that can be learned. They also told us this:
Interesting stat about @beisgovuk consultation conversion rates – Citizen Space: 21% vs gov.uk: 3%. That's a huge difference.
— Ben Fowkes (@ben_fowkes) October 4, 2016
In summary: (their words again) ‘so, does Citizen Space help? Yes’. (Phew!)
5. Email is a precarious response mechanism!
We heard of one case where the primary means of responding to a consultation was simply sending an email. Unfortunately, the inbox in question had a pretty meagre message size limit (as is often the case with government organisations). And it really wasn’t geared up for the huge scale of public reaction that the topic prompted.
Tens of thousands of responses arrived in short order and quickly, predictably, crashed the email server. This led to thousands of people per hour getting bounce-backs saying that their message could not be delivered – obviously not a situation the organisation in question would have wanted. Further proof, if it were still needed, that email isn’t really a sufficient standalone online response mechanism.
6. There’s a big push to keep improving on reporting back
Rowena, one of our account managers, gave a presentation on the importance and value of reporting back to participants. It was encouraging to then hear from several attendees about how there’s already a growing desire in their organisations to improve on this front.
TfL, for example, have recently instituted a target of reporting on 90% of consultations within 90 days (that’s not to say they’ll always have reached a definitive decision by that point, but they will update participants on findings and progress to date). This is tracked and monitored by the consultation team and, in the past quarter, they actually saw 100% of consultations reported on within 90 days. That’s a great precedent and hopefully will increasingly become the norm.
7. Only asking open-ended questions is really problematic
Juliet from DH told us about how one of their annual consultations had previously been ‘just a PDF form with 5 free text questions’. Perhaps unsurprisingly, this didn’t render the most useful results. Participation was low and the responses they did receive were difficult and time-consuming to analyse. She pointed out that open-ended questions can often be simplified to, or at least supported by, quantitative question types instead. Done well, this makes both participation and analysis easier.
8. But so is asking only closed questions!
It can then be tempting to produce surveys that only use quantitative questions. But that comes with its own risks. TfL explained that this could leave people feeling like they weren’t really being given the chance to express themselves, and consequently feeling un-listened-to.
So it’s important to have a good mix of question types – not just determined by what’s easy to analyse, or what’s easy to write, but ultimately by ‘what is the most appropriate way to discover the information we need?’
One interesting specific detail on this: TfL asked their participants for feedback on the consultation process itself and found that people ‘really liked ranking-type questions’. Now, that won’t necessarily be the case in every instance (see point 3) but an intriguing little insight nonetheless.
9. Interim reporting can really improve results
Juliet from DH extolled the virtues of interim reporting (rather than waiting until a consultation is completed before running any numbers/analysis). The example she gave was from their young carers consultation. Obviously, this was a hugely important topic and the Department was keen to hear from the many diverse perspectives of people who would be affected.
They ran an interim report, while the consultation was still live, to see how it was going. One of the findings was that, whilst stakeholder and campaign groups had been quick to take part, there hadn’t been many responses directly from young carers themselves.
Because they discovered this before the consultation close date, DH were able to pivot their plans to address the issue. For instance, they went to dedicated young carer events and launched a new social media campaign designed to boost their engagement from that audience in particular.
10. Don’t just ‘replicate documents and hope for the best’
We heard of a few cases where a consultation would be written in one part of an organisation and simply ‘delivered’ to the consultation team or Citizen Space admin user in paper form, with little thought given to its digital implementation.
Or where a survey would be produced as a written PDF document with instructions like ‘if you answered ‘yes’ to question 4, please turn to question 9′ – and then simply copy and pasted into the survey tool, without realising that there would be better ways to present that information online.
Rather than just replicating documents on Citizen Space, it’s important to take advantage of the design tools at your disposal to present content as effectively as possible. That includes using things like expandable Fact Banks for detailed information (to prevent pages getting overwhelmed with huge walls of text), or implementing skip logic for routing questions (to minimise friction for participants without confusing them or making them feel like they’ve missed out on something).
11. (Bonus one!) BEIS is pronounced ‘baize’
Not ‘bees’, as we initially thought. We still think it should be bees, really. Bees.