Hello, we’re looking for an experienced web developer to join us in Bristol, UK.
Author: Jess Henderson (Page 1 of 2)
Over the past couple of months we’ve been focusing our development efforts on improving our hosting and associated product environment via an appropriately titled ‘production infrastructure sprint’.
Although this doesn’t sound as exciting as adding features to our products, it’s a vital part of Delib’s service to our customers, as it helps to ensure that we continue to meet our uptime and performance commitments. Here’s a little overview of what we’ve been up to.
What we’ve been doing
Up until recently we hosted all our customer instances on large multi-tenancy servers. ‘Multi-tenancy’ means that several Delib customer sites run side-by-side on the same machines, although all their data is stored in separate databases. These servers live in secure data centres, physically located in the same territory as the customers they serve. The data centres are responsible for providing Internet connectivity for the production servers.
Over the past few weeks, we’ve been moving customers slowly and carefully in batches from our current hosting providers to new providers who can better meet our service and uptime requirements.
Why we’re changing our hosting infrastructure
The reasons for migrating to new hosting providers are threefold:
1. Improvements in availability
In the UK, we are moving all our hosting to Rackspace, the market leader in cloud hosting, which offers a 100% uptime guarantee. Since our uptime is necessarily bounded by that of our upstream providers, it’s important to use the best that we can get. We are researching the best providers in other territories, to ensure that we continue to meet and exceed our commitments for all our customers.
We use a server monitoring service that notifies our account managers and developers by text message whenever a customer’s instance is unavailable for any reason (even if it’s in the middle of the night) so we’re all keen to ensure that these improvements pay off as soon as possible!
2. More hosting options for customers
After migration, every Citizen Space and Budget Simulator instance will live on its own virtual machine. This allows us to offer different hosting packages for different usage patterns: we can now tailor the system specification (RAM, disk space, number of processors) to the requirements of the customer. Furthermore, large spikes in one customer’s traffic can no longer adversely affect the response times of other customers’ sites.
Dialogue App instances will continue to run on a multi-tenancy setup by default. However, customers with heavy usage requirements (eg large, heavily-publicised national dialogues), will have the option to host their Dialogue App instance on its own machine.
3. Consistent configurations and automation
As our number of customers grows, our developers have been spending more and more time engaged in administrative tasks such as rolling out new instances and upgrading existing customers. While this is vital to the business and to our customers, developers would much prefer to spend their time developing new features and fixing bugs in the products.
At the same time as moving customers to the new hosting infrastructure, we’ve been improving our suite of developer tools so that more of the day-to-day tasks can be done without developer intervention.
For our customers, this means that planned maintenance should soon be able to take place, as far as possible, outside working hours. It also means that developers will have more time to spend on improving our products, resulting in a better user experience for our customers and end users.
Find out more
If you are interested in finding out more about the improvements we are making please feel free to get in touch with either Louise or Rowena.
As in previous years, I took the last month’s logs from our web servers, and ran them through an open source analysis package called Visitors. In contrast to previous years, these stats now cover all our servers worldwide – not just in the UK.
Since visits to our Australian servers now account for a notable proportion of our apps’ traffic, this year I have also included a separate breakdown just for our Australian stats.
Worldwide browser usage
I’ve generated two different reports: one for our apps’ management pages (i.e. pages that can only be accessed by logged-in admin users), and one that includes public-facing pages as well. Here are the figures for our admin users:
It’s a relief to see that IE6 has not made a reappearance in the past year, and also that IE7 usage has dropped from 15.3% in April 2013 to less than 10% a year later.
We currently provide Level 2 support for IE7, which means that all functionality and navigation must work, and all content must be readable in IE7. However, we’d much prefer to spend our time developing new features that benefit everyone, rather than fixing bugs that only appear in this eight-year-old browser. Over the next few months we hope to encourage our customers to move to more modern browsers so that we can drop admin support for IE7.
The second chart shows visits to all Citizen Space and Dialogue App pages, including visits from members of the public:
As you would expect, this shows a much wider range of web browsers, including a few visits from IE6. By the way, I’ve excluded visits from crawlers, bots, RSS feed readers and other things that aren’t conventional human-controlled web browsers.
It’s interesting to compare these stats with last year’s numbers. In particular, the most popular browser is now Google Chrome, which has overtaken IE8 – last year’s frontrunner. However, usage of IE8 still remains far higher amongst our users than you’d predict based on global figures from StatCounter.
Browser usage in Australia
Although the majority of traffic to Delib’s apps is served from our UK servers, traffic to our Australian servers now constitutes a notable proportion of visitors (8.5%, based on last month’s stats).
IE8 is still the most popular browser amongst our Australian admin users, but its lead is far less marked than in the UK.
When we consider all pages, not just admin pages, it’s interesting to see that the most popular browser used to visit our Australian sites is Firefox. This is in contrast to StatCounter’s figures for Australian Browser usage, which pegs Chrome usage at more than double that of Firefox. The numbers here are fairly small (16K visits a month) so this could have been skewed by one particularly Firefox-heavy demographic of survey respondents, for example.
* Visits to our newest app, Budget Simulator, are not included here, as a lot of the administration is done by Delib account managers, and our choice of browser would skew the statistics quite a lot. Also, the visitor profile of Budget Simulator leans quite heavily towards mobile users. I feel another blog post coming on!
** For the purposes of these statistics, a ‘visit’ comprises all the requests from a given IP address and useragent on a given day.
Back in the Summer, Karl wrote an article about the various channels of support available to Delib’s customers. Maybe I’m just naturally nosey, but I’m fascinated by other companies’ support staff. When I’m talking to my bank, my phone company or whatever, I always wonder what it’s like for the person on the other end. What is their desk like? Do they like their job? How many other people do they work with? So I thought some of you might like to hear five secrets from support@delib.
1. We’re not a vast, outsourced call centre
If you are an existing Delib customer, you will know that you have your own account manager. You can phone up and ask for them by name, and they will know who you are, which organisation you work for, and that you’ve just got back from holiday in France.
But did you know that the personal service continues when you email email@example.com with your query? The first person who sees your query is likely to be your account manager. Often, they will be able to answer your question very quickly. If not, they’ll usually drop you a line to say that they’ve passed the question on to a developer who can look into it in more detail. The useful thing about emailing the support address is that it pops up in our support ticket system and I (the support developer) can see all the info you’ve provided, and the account manager’s response, straight away.
I might never speak to you on the phone, but I almost certainly recognise your name too, and know which organisation you work for. I might even know you went to France last week, if I overheard your account manager chatting to you about it. You see, in the UK office we all work within earshot (and potentially nerf-shot) of each other. When we answer your question, we sign our names, so hopefully you’ll get to know us too.
At any time, there will be one or maybe two developers answering support queries, along with one or two testers to check our work. That means it’s very likely that your entire query will be answered by the same developer. If you ask a couple of questions in the same week, you will probably get the same developer each time.
2. All Delib developers take turns on support duty
We think it’s vital that the people who develop Delib’s apps are the same people who support them. This insight works two ways: As one of the people who designed, coded and deployed the feature you’re having trouble with, I am the best placed to help you use it, or to track down the bug (sorry!) and work out how to fix it.
Conversely, having talked to you – a real user of the system – I have a much better insight into how you use it every day in your job. This really helps us when prioritising new features and understanding how we can improve the usability of our apps. In some companies, developers see their users as an anonymous “them”. When we plan the work for each release, we always like to use real names. “Bob from Westonshire Council had trouble using feature X last week. Is there anything we can do to make it easier to use?”
At the moment, we have a fortnightly rotation and hand over the support baton on a Thursday morning, so if you ask a question on Wednesday you may get answers from two different developers. Don’t worry though, we always have a handover chat with the person taking over, so they know which issues are open and how far we’ve got with resolving them.
3. We love our Knowledge Base
If you’re not sure how to use one of the features of our apps, the first place to look is in our Knowledge Base. If you don’t already know about it, I recommend you bookmark it. It’s a huge, searchable repository of user guides, FAQs, tips and tricks. If you’re using Citizen Space and are logged into your site as an admin, you can also get to the Knowledge Base via the ‘support’ link at the top of every page.
However, we don’t only recommend the Knowledge Base to our customers – we use it for all our internal documentation as well (obviously you have to be a Delib employee to see that stuff). One of the things that support developers do when we’re not answering customer queries is to keep the documentation up to date, and write handy how-to guides for our future selves, so that the next time a particular question comes up, we can answer it even more quickly.
4. We like it when people say ‘thank you’
Sometimes when I contact other companies’ customer support and they solve my problem, I wonder, “Should I drop them a line to say ‘thanks’, or is that just using even more of the support person’s valuable time?” Well I still don’t know about other companies, but here at Delib, we always appreciate a quick “Thanks, that did the trick.”
Of course it makes us smile to know you’re happy, but also it means we know we can close the support ticket without offending you, or worrying that we’ve left any of your questions unanswered.
5. This is what my desk looks like
- This flag means I am on support duty (we cheesily refer to the role as ‘Support Superstar’). The flag is attached to the side of my monitor with velcro. This morning I will give the flag to Alan, who will take over from me. Support Superstar doesn’t only do customer support; they’re also the go-to developer for any sales or operational questions that need a technical answer, so this flag makes it obvious who to ask.
- Kanban board. This is one of the ways we visualise the tasks we have in progress. It helps keep our turnaround times low on things like new customer rollouts. Jo wrote about this in her post on how we support our government clients.
- Consultants sit here.
- Account managers sit here (see, I told you they were nearby).
- Samuel the spider plant. He is a bit under the weather at the moment but I am expecting a full recovery.
- This blog post (paradoxically)
- Other developers, testers and sysadmin sit just there.
So, I’m signing off from support duty for now, but if you write to firstname.lastname@example.org in a few weeks’ time, there’s every chance that I might be in touch.
About a year ago, I wrote a post introducing Citizen Space’s API. APIs (or Application Programming Interfaces), make software developers happy because they provide stable, documented ways for different pieces of software to communicate with each other.
In the case of Citizen Space’s API, we’ve focussed on providing access to the public-facing consultation listings. With some programming knowledge, the API allows you to embed a consultation search form on your own website, send a query to Citizen Space, display the results of the search, and drill down to display individual consultation records.
Version 1 of the API did this in a very straightforward way: requests to the API returned simple, un-styled snippets of HTML, which could then be included in the page markup and styled as required.
However, this simple approach doesn’t provide a huge amount of flexibility in what is displayed. For example, if you call the search_results API method and give it a set of search criteria, it returns an HTML list of consultations that matched the query, including a short snippet of the overview text and the consultation’s start or end date. That’s great if that’s what you want to display on your website, but if you only wanted to display the consultations’ titles, you would have to write code to do a lot of fiddling with the output from the API method, to remove or hide the details you didn’t want. And if you wanted to display more data than was included in the HTML snippet, you’d be in trouble!
We also provide a method called json_consultation_details. This returns a JSON structure containing all the info you’d need to reconstruct an individual consultation’s overview page on your own website, with markup, layout and styling to fit in with the rest of your site.
For all the gory details, you can read the API documentation.
Integration with external sites is something we really want to prioritise in Citizen Space development. If you are looking to integrate more closely with Citizen Space and would like to see a specific piece of functionality added to our API, please drop us a line at email@example.com. We try to ‘jump the queue’ with API feature requests and if they’re simple, we can often squeeze them into our next monthly release. Right now we’re focussing on access to the public-facing consultation data, but in future we want to introduce an authenticated API to allow, for example, the export of responses, or the programmatic creation of new consultation records.
In what seems to have become a roughly annual tradition, I’ve just done a survey of the browser usage amongst our UK Citizen Space users. As in previous years, I took the last month’s logs from our UK-based web servers, and ran them through an open source analysis package called Visitors.
I’ve generated two different reports: one for Citizen Space’s management pages (i.e. pages that can only be accessed by logged-in admin users), and one that includes public-facing pages as well. Here are the figures for our admin users:
This first chart shows something wonderful: last month, nobody used Internet Explorer 6 to administer their UK Citizen Space site.
For those who don’t know, Microsoft’s venerable Internet Explorer version 6 (AKA IE6) was released back in 2001, and is notorious for rendering web pages very differently from modern, standards-compliant browsers. To support IE6, web developers have to spend a lot of time writing workarounds to make web pages display correctly in all browsers. Of course, this increases the cost of product development without benefitting those users who don’t use IE6. Last year Delib decided to stop actively supporting IE6 in our apps, and these latest figures clearly vindicate that decision.
The second chart shows visits to all Citizen Space pages, including visits from members of the public:
As you would expect, this shows a much wider range of web browsers, including a few visits from IE6. The ‘Unknown’ useragents are mostly made up of crawlers, bots, RSS feed readers and other things that aren’t conventional human-controlled web browsers.
It is interesting to compare our statistics with current worldwide browser usage. At the time of writing, Google Chrome and Firefox are the most widely used browsers, followed by IE9 and then IE8. In contrast, our stats show that IE8 is by far the most common browser used to access Citizen Space.
My theory is that most of the activity (both administration and participation) on our Citizen Space sites takes place during the working day, where people are less likely to have a choice about the software they use, and are more likely to be stuck with a slightly out-of-date standard-issue browser. Incidentally, Edd at GDS made some interesting observations about this same phenomenon last month.
* For the purposes of these statistics, a ‘visit’ comprises all the requests from a given IP address and useragent on a given day.
We’ve always been keen to encourage Citizen Space to play nicely with other friendly applications on the web. For example, you can:
- Link your consultation records to awesome sites such as EventBrite, so that consultations you’re running elsewhere on the web (or even offline) are still listed and searchable through Citizen Space.
- Embed rich media from sites such as Google Maps, YouTube and Scribd into your consultation records.
- Couple Citizen Space with third-party mailing tools, like MailChimp, to stay in touch with people who want to be kept updated.
We love these features and so do our clients, and they’re definitely not going away. However, we really want to provide more flexible ways to open up the data held within Citizen Space, and that’s why we’re so excited about our new API. “API” is short for “Application Programming Interface”. This is a set of documented procedures that third-party systems can use to communicate with an application.
API integration with Citizen Space requires more technical know-how than the features described above, so it’s something for your web or IT team to get their teeth into. The important thing is that the API doesn’t care what platform or language you use for your existing infrastructure. Citizen Space is written in Python and uses the Zope application framework, but other organisations may use WordPress, .net or Drupal for their websites, which are written in other languages such as PHP. Using the API, your tech team can use the languages and platforms that are familiar to them, but still access and manipulate the data held in Citizen Space.
Version 1 of Citizen Space’s API (included with April’s release of Citizen Space) is all about getting public consultation details out of the system so that you can include them in your own site. We’re providing ways to embed search forms, search results, consultation listings and consultation overview pages into your site.
But that’s only the first step. Version 1 of the API provides all of its output in simple, cleanly formatted HTML. This is quick and easy to work with, but we’d like to build on it by providing other output formats such as JSON and XML. We also want to publish more plugins for other systems, code snippets and example projects, because even we haven’t investigated all the exciting opportunities that this API provides. We’ve set ourselves up on Github, so all our Citizen Space integration projects will live there.
Looking further ahead, we’re going to think about how the API could be extended to include data from Citizen Space’s admin areas (with suitable authentication of course). For example, would you like to check your organisation’s intranet dashboard and see how many responses your QuickConsult consultations have had? We also want to consider how the API could be used to get information into Citizen Space.
Is there information in Citizen Space that you’d particularly like to access from elsewhere? Are there applications you’d like Citizen Space to play more nicely with? Are you using our API already? We’d love to hear from you.
Update – March 2015: The Citizen Space WordPress plugin is now deprecated. We no longer recommend it as a way to integrate Citizen Space with WordPress. If you are using this plugin in your WordPress site, please see the article How do I integrate Citizen Space into my existing website? for alternatives, or get in touch with us at firstname.lastname@example.org.
We are very excited about the new API (or Application Programming Interface) that’s included with version 1.6.5 of Citizen Space, because it provides a standardised, documented way for third-party software to communicate with Citizen Space. However, the second word of that acronym, ‘programming’, is a bit off-putting for some people; writing software is not everyone’s cup of tea!
We know that lots of our clients currently use WordPress for parts of their online presence, so the first thing I wanted to build using the API was a plugin for WordPress. I wanted people to be able to embed consultation listings, search facilities and entire consultation records into their WordPress sites, without having to know PHP or any other programming language.
WordPress provides a handy facility called shortcodes. These are keywords written in square brackets [like this]. If you type one of these shortcodes into a blog post or page, WordPress interprets it in a special way and replaces it with dynamically generated content. If you download and install the Citizen Space plugin in your WordPress site, you will have four new shortcodes available to you:
If you want to embed a fully functional consultation finder in your WordPress site, all you need to do (after installing the plugin) is paste the 3 lines below into your page or post:
If you are confident with writing PHP code, please feel free to fork the plugin code on Github and submit your own contribution, or use it as the basis for your own Citizen Space integration project. If you find this code useful, we’d love to hear from you.
Almost a year ago, we published some interesting browser statistics based on the logs from one of our Central Government Citizen Space servers. We ran the logs through a piece of open source analysis software called Visitors, and this gave us an anonymous breakdown of all visits to Citizen Space, showing the browsers and versions that were used. We looked at the statistics for all pages, and compared them with the stats for pages only accessible to admin users. The results were insightful but rather scary: Internet Explorer 6 accounted for more than 1 in 3 visits by our Central Government admin users.
For those who don’t know, Internet Explorer version 6 (lovingly known as IE6) is a web browser that Microsoft released over a decade ago. Because it renders web pages differently (in some cases dramatically differently) from more modern browsers, web developers spend a great deal of time creating workarounds so that IE6 users can still access our websites. Of course, this increases the cost of product development without necessarily offering any benefit to the majority of web users who don’t use IE6. Coupled with the fact that IE6 now only receives limited support from Microsoft, almost everyone is in agreement that this ancient and decrepit browser must be phased out.
This morning, we had a comment on last year’s post from a reader who was interested in how Citizen Space’s browser stats had changed. Thank you Perry – you reminded me that I’d been meaning to re-do this analysis soon. So here are the graphs comparing the numbers 11 months ago with where we are today:
Central Government Citizen Space – all users (admin and public)
These statistics roughly follow the browser trends of the general internet population*, with IE8 and 9 increasing in popularity while the older IE versions decrease as expected. Pleasingly, IE6 usage has roughly halved since last May.
Firefox, Chrome and Safari have gained more of a stronghold in the past year, although interestingly, Internet Explorer as a whole has retained a far larger share of Citizen Space users than worldwide browser usage statistics* would predict.
Central Government Citizen Space instances – admin pages only
When looking at the statistics for our admin users, the most exciting thing is that usage of IE6 has crashed by 90% – from 35% down to 2.4% of visits. This is a great relief to us, and shows the huge effort that has taken place in government IT departments to upgrade users away from this insecure, ill-supported browser.
It’s worth noting that overall, usage of Internet Explorer among our Central Government users is more than 90%, compared to 34% worldwide*.
The interesting question is what levels of support to provide for different browser capabilities. We currently provide Level 2 support for IE6, which means that all content must be readable and navigable, but differences in styling and layout may exist. This works OK for our products at the moment, but as web users come to expect a richer and more fluid experience, the likes of IE6 are going to lag further and further behind. How small does the percentage of IE6 users need to be before we can stop supporting it at all?
As always, I’d love to hear your views.
*Worldwide browser statistics from statcounter.com.
Earlier this month, Chris blogged about adding a Stakeholder Database to your consultation for $5, by making use of Campaign Monitor‘s opt-in email service.
I have to confess I was dubious. As a developer, my initial reaction to this kind of proposition is usually “it can’t possibly be as simple as you think it is”. So I tried it with one of our Citizen Space consultations, and it worked! Sorry for doubting you, Chris.
The whole thing took me about 45 minutes to set up and test, but that was only because I was taking screenshots at every step and making notes so I could document the process. You can probably do it in 15. So anyway, here’s the step-by-step guide:
I should note that the whole process of setting up the database and collecting stakeholder information is actually free. You only incur the $5 cost when you use Campaign Monitor to send an email to one of your mailing lists (totally worth it for the stats and reporting you get at the end). By the way, we’re not affiliated with Campaign Monitor, but we like them and use them to send our newsletters.