ADD378 Hosting and development of London DataStore

1.1. When it was launched in 2009, the original London DataStore (LDS) was a highly innovative first step into open data for London and the UK. In the intervening period, a number of other cities around the world have copied the GLA. Digital technology and the open data agenda have also advanced significantly in this time.

1.2. The development of the enhanced LDS was procured using a competitive procurement process in April/May 2014. The contract for £60,000 was won by DataPress whose team had carried out much of the work on the national data.gov.uk.

1.3. A full working product was delivered and launched on 23rd October 2014 and included the initial 12 months of development and support. Provision under this arrangement is to continue until March 2016 with additional costs of £10,000 incurred. The DataStore is built on open source products CKAN and WordPress and uses Amazon Web Services and as such is not restricted to one service provider and as such a mini-competition will be held to identify further provision.

1.4. The provision of support to date covered, and future provision would seek:
• Development of the platform
• Web hosting
• Security
• Backups
• updates to CKAN

1.5. The DataStore consists of:

- A searchable repository of approximately 1,600 tables of key data about London
- supporting information about each dataset (including licensing and descriptions)
- A system of user accounts, which allows people to register for notifications when datasets that they are interested in are updated
- A publishing area which allows other trusted organisations (for instance LFB) to publish and update their own datasets
- Management functions to help ensure that datasets are kept up to date
- A website system for publishing pages of further information and tools to help people make use of the data

The following objectives and outcomes are expected through continued support to the DataStore:

- 24 months cloud hosting;
- Deployment, testing and updating of security to prevent unauthorised access to user information or the back end systems;
- Daily backups;
- Implementing updates to CKAN; and
- Development of additional functions to the platform, including:
- Improvements to the data preview functions;
- Reporting on dataset usage; and
- Applying design guidelines from new GLA website.

The London DataStore is delivered in way that meets the GLA’s duty not to discriminate against people with protected characteristics. In particular, it has been designed to be at least as accessible to people with visual impairment as per the main GLA website.

4.1 Key risks and issues

Compatibility with other data stores and catalogues

The DataStore uses internet standards and open data standards to ensure that it is able to harvest data from other data stores and catalogues regardless of their software platform. It is harvested itself by the national data.gov.uk and the ODI certification process.

Capacity and resilience of infrastructure

The DataStore is hosted in an Amazon Cloud (where multiple instances are kept in different physical data centres) leading to over 99.97% up time in 2015.

Problems caused by either not applying updates to the underlying software, or through the update process

This takes up a significant amount of staff time for the data.gov.uk team, but is handled by DataPress for the London DataStore.

4.2 links to Mayoral strategies and priorities

Smart London Plan

Delivery of an enhanced London DataStore directly fulfils the following ‘Measures of Success’:

- Evolve the London Datastore into a global exemplar platform by 2016
- Double the number of users on the Datastore and Dashboard by 2018

It also supports the Smart London Borough Partnership in its work to encourage boroughs to free up London’s local level data – either by harvesting data from borough data stores or by providing a ready-made platform for boroughs to use if they wish.

Supporting Open Source

The GLA has a strong track record in using Open Source software (for instance Drupal for the main GLA website) and we are continuing to move away from proprietary software to open source (e.g. moving from Oracle to Postgres). This approach is economically advantageous to the GLA, avoids ‘vendor lock-in’ and supports the SME / high-tech community.

4.3 Impact assessments and consultations

The DataStore receives nearly 50,000 visits per month and is a flagship project for the GLA, receiving an international award for Open Data Publishing from the Open Data Institute in June 2015. The impact to the GLA of not continuing to operate it and improve it would be damaging to the organisation’s reputation.

Consultation on the functional requirements took place across the Intelligence Unit (who use and maintain the current DataStore on a daily basis) and more widely across the GLA. Through feedback forms and the Borough Data Partnership would also have feedback from users on the next set of developments.

Consultation on the interaction with the main GLA website and the GLA’s infrastructure has taken place with the Technology Group.

5.1. The total estimated cost of the £40,000 expenditure will be funded from the Ordnance Survey Budget code (GG.0220.003.003). The financial implications per financial year will be as follows:

2016-17 £20,000
2017-18 £20,000

5.2. Any changes to this proposal, including budgetary implications will be subject to further approval via the Authority’s decision-making process. All appropriate budget adjustments will be made.

5.3. The Intelligence Unit within the Communities & Intelligence Directorate will be responsible for managing this contract and ensuring that all expenditure and associated activities comply with the Authority’s Financial Regulations, Contracts & funding Code and Expenses & Benefits Framework.