Big data is an important feature of our contemporary lives. Everywhere we go, from our homes and workplaces to holiday destinations and shopping trips, we generate huge amounts of data which are stored, analysed and used by companies, authorities and organisations.

Big data enables major insights, for example, into improving transport flows across a city or gaining a better understanding of diseases by analysing medical data. However, big data also raises concerns around privacy, security and the ethics of gathering and using personal data.

The EU-funded e-SIDES project delved into the ethical implications of using big data. ‘We are aiming to improve people’s confidence in big data by exploring what can be done with personal data without straying into potentially non-ethical practices,’ says Richard Stevens, Director of IDC - European Government Consulting and e-SIDES project coordinator.

Filling data lakes

Big data means a large amount of data being generated continuously in a wide variety of different formats. For example, city authorities may be monitoring sensors placed at several places across the city. These sensors could be gathering data on air quality, traffic density, outside temperatures, noise levels and the numbers of pedestrians passing by. All this information is pouring into what is known as a ‘data lake’. Analysing this lake helps authorities to take important decisions, such as deciding to charge higher tariffs to enter a city by car on days when pollution levels are high.

Meanwhile, companies like Google, Apple, Facebook and other internet firms are constantly collecting, using and selling data. Data-aggregator companies buy this data from multiple sources and use it to create detailed data sets and profiles that they then sell to insurers, airline companies, supermarkets and others.

For example, a supermarket may use big data in its decisions on what special offers to launch on which products. To help it choose, it may buy data to develop client profiles of shoppers in a particular geographical area. E-SIDES also considered the ethical dimension of using big data in this way.

‘We are building a framework to show people what they should do with this type of data to avoid it being ethically incorrect – for example, it could also end up racially profiling people. We are working in a grey space since there are not many international laws governing what companies are allowed to do with data,’ explains Stevens.

Expert guidance

The project brought together lawyers, ethics experts, developers, policymakers, small business representatives, industry representatives and civil society representatives to build a community of big-data stakeholders with many different perspectives. E-SIDES then developed sets of guidelines on the ethical and social implications of using big data for analytics and artificial intelligence product developers, private companies and public authorities.

‘One recommendation that applies to all users of big data is to invest in privacy. While this may appear to be an expensive process for smaller companies, the consequences and potential losses from not investing can be unmanageably huge,’ says Stevens.

The project recommends developers and operators using big data should: comply with any laws and corporate policies; create a high-level role within a company in charge of data; publish a declaration of their data ethics policies; carry out impact assessments of their use of big data; and protect privacy by default and implement regular data security checks.

For policymakers, the project guidelines propose improving public awareness of big data; implementing data-protection measures where necessary; creating mechanisms to review data practices; and ensuring that public-sector procurement meets ethical criteria such as respecting privacy rights.

E-SIDES guidelines for civil society organisations include recommendations to inform individuals about data risks; maintain dialogue with all stakeholders; foster adherence to professional standards and codes of conduct; and promote the idea of a data ethics oath.

The project held workshops, conferences and online debates to bring all communities on-board and produce a community position paper. All E-SIDES recommendations are available on the project website, and other projects are now implementing the guidelines.