API Thoughts

Our medium to long term plans for Public Whip are to introduce an API (Application Programming Interface: if you don’t recognise the term, you can just gloss over this whole post) to help third parties integrate with Public Whip – without having to “screen scrap” the content. An example of a service currently utilising the content is “Keep track of how your MP votes” Boxee application.

Things we’d like to know:

  • What format of data would you prefer the data: CSV, TSV, XML, JSON, SQL, other?
  • How would you prefer to access it: SOAP, REST, HTTP GET, HTTP POST, data download (we do currently offer this!), other?
  • How often would you like to refresh the data: weekly, daily, hourly, “as live as possible”?
  • What sort of information would you like to be able to access?
  • Would you prefer, in the first instance, “documents” or “code” integration guides?
  • Would this be for a commercial or “open source/free access” project
  • What sort of volume of requests would you expect to push to us? (And would it be possible to introduce caching/proxying your end)
  • Would you like to “push” data to us? If so, what sort (comments, policy division revisions, etc etc)
  • Anything else you think relevant 🙂

Basically, we’d just like to hear from you how you would like to use the data. At the moment, we’re not going to make any guarantees about anything, but if we can get ideas from YOU then we may be able to start making plans…

Weekend work: Security patches and forum access

Due to numerous inherited flaws in the codebase we’re seeing rather a lot of spam, both in the forums and in the email of registered users. As such this weekend we’ll be applying a number of patches, and stop-gap measures to tighten things up.

Our primary concern is the reports we’ve received from people who have begun to receive spam email to addresses that are only utilised on PublicWhip. We despise spammers and can categorically state that we have never, and will never, share or sell the personal data that our users entrust with us. If you have received spam email to an address that has only been used on PublicWhip, would you please let us know the address details (email team@publicwhip.org.uk) to help us identify where the data leak is coming from and to plug it.

One of the main culprits is a very out of date installation of PHPBB and whilst we look at alternatives, and gather input as to the usefulness of the forum (beyond acting as a changelog for divisions and policies), we’ll be taking the forum offline.

We’re still getting to grips with a system and a codebase that is relatively new to us, but we’re determined to make the site the best that it can be, and data security and data integrity will always be at the heart of what we do.

Okay, so what are you going to do with it?

On the first of August 2011, after 8 years of hard graft and dedication, Francis Irving and Julian Todd handed us the reins of The Public Whip.

So, what are we going to do with it?

For a small question, it’s got quite a lengthy answer, so we’ll try to give you a summary:

  • PublicWhip.org.uk will remain a free to use, not-for-profit and open source website
  • After eight years, some of the code is getting a bit janky, so we’ll be stripping down and re-writing it
  • We want to engage the community in the site and the wider issues it raises, and will always be grateful for volunteers of all skill sets
  • We’ll be looking to give the front-end a bit of design love, making the site easier to use and to navigate (and a bit easier on the eye too)
  • We’ll be looking into sponsorship and/or donations to help fund development of new functionality
  • We’re looking at building a seperate API for NGOs, newspapers, lobby groups and campaigning organisations – so that they can make the most of the juicy raw data (if you’re any of these and would like to chat about how we can make the project more useful for you – get in touch!)
  • We’ll be looking to globalise the project, getting regionalised versions of PW into as many countries as possible (if you’re interested in being involved in this, give us a shout)

Along the way, we’re going to try and document as much as we can, keeping you up to date with what’s happening on the site and the wider project. We’ll also be talking a lot about the technologies we’re using to rebuild the site, why we’ve made the choices we have etc, with the hope that it’ll be helpful to other open data projects of all sizes.

If you want to contact us about the site, or the wider project, there’s loads of ways you can do that. The Public Whip is on Facebook and Twitter, you can email team@publicwhip.org.uk, or even leave a comment here on the blog.

If you’ve got a press enquiry, want to ask us about Raraunga, sponsoring The Public Whip, or are interested in the API service, you can also find us on Facebook and Twitter, or email hello@raraunga.com.

That’s all for now, we’d best go get some work done! Thanks for supporting The Public Whip thus far, there’s a very long road ahead, but we’re looking forward to the challenge and hope you’ll come along for the ride.