Controlled by calculations

Download the report

This is the third of four papers exploring power and accountability in the data economy. These will set the stage for future interventions to ensure power becomes more evenly distributed. This paper explores the rise of algorithms to make sense of big data, while other papers examine: the impact of the mass collection of data; the companies built on data, that mediate our interface with the digital world; and the labour market dynamics that they are disrupting.

Our research so far has identified a range of overarching themes around how power and accountability is changing as a result of the rise of the digital economy. These can be summarised into four key points:

• Although the broader digital economy has both concentrated and dispersed power, data is very much a concentrating force.
• A mutually reinforcing government-corporation surveillance architecture – or data panopticon – is being built, that seeks to capture every data trail that we create.
• We are over-collecting and under-protecting data.
• The data economy is changing our approach to accountability from one based on direct causation to one based on correlation, with profound moral and political consequences.

This four-part series explores these areas by reviewing the existing literature and conducting interviews with respected experts from around the world.

Algorithms have become essential to managing our digital lives and navigating the digital world. Without them we could not make sense of the huge mass of digital information available. But now that algorithms wield such influence, we have a responsibility not to misuse them.

• Algorithms are actively shaping our lives: They have morphed from curating online content to curating and influencing our lives.
• We are entering the scored society: Access to services, both public and private, are increasingly being mediated through algorithms which analyse our data and decide whether or not our digital profile matches the requirements for access.
• Lazily programmed algorithms entrench discrimination and bias: This happens especially when data is used uncritically.
• We need to rethink accountability in age of algorithms: How we hold these systems accountable, as well as how we use these systems to hold people accountable is shifting.
• Data cannot be permanently anonymous: Although diligent preparation can make it harder to re-identify data, with the right tools and techniques it is always possible.

A new economy is emerging. And this new economy is powered by a new type of fuel: data. As the data economy becomes increasingly prominent, there are troubling signs that it is worsening existing power imbalances, and creating new problems of domination and lack of accountability. But it would be wrong simply to draw dystopian visions from our current situation. Technological change does not determine social change, and there is a whole range of potential futures – both emancipatory and discriminatory – open to us. We must decide for ourselves which one we want.