Big data to the rescue? Challenges in analysing granular household electricity consumption in the United Kingdom

Rapid growth in smart meter installations has given rise to vast collections of data at a high time-resolution and down to an individual level. However, to enable efficient policy interventions, we need to be able to appropriately segment the population of users. The aim of this paper is to consider challenges and opportunities associated with large highly-granular temporal datasets that describe residential electricity consumption. In particular, the focus is on experiments relating to aggregation of smart meter time-series data in the context of clustering and prediction tasks that are often used for customer targeting and to gain insight on energy-use about sub populations. To cluster energy use profiles, we propose a novel framework based on a set of Gaussian based models which we use to encode individuals’ energy consumption over time. The dataset consists of half hourly electricity consumption records from smart meters of households in the UK (2014–2015). The contribution of this paper comes from its investigation of how consumers or groups may be clustered according to model parameters in scenarios where additional data on consumers is not available to the researcher, or where anonymity preservation of the smart meter user is prioritised. A secondary aim is to invite greater awareness when data reduction is required to reduce the size of a large dataset for computational purposes. This may have implications for policy interventions acting at the individual or small group level, for instance, when designing incentives to encourage energy efficient behaviour or when identifying fuel poor customers.