Finkel Review - Data: Critical data needs in the national electricity market

In brief

In the fourth of our series analysing the Finkel Review, we look at Dr Finkel's assessment of the critical data needs in the National Electricity Market (NEM), including a key recommendation that by the end of 2018, the proposed Energy Security Board, in collaboration with the Australian Energy Regulator (AER), should develop a data strategy for the NEM. As new technology continues to develop and consumers are placed at the centre of the NEM, it is not surprising that considerable focus has been given to the collection and increased transparency of data. Partner Michael Park and Associate Jessica McCarthy report.

Key themes

Dr Finkel has identified the following critical issues in relation to data:

1. There is a lack of availability and transparency of data – improved access to a broad range of data and information to enable all consumers to share in the benefits of new technologies is required.
2. There is a lack of visibility over distributed energy resources (DER) – new data collection frameworks or mechanisms should be used to increase the visibility of all forms of DER.
3. There should be a continued focus on cyber security – as market systems contain increased volumes of valuable data, it will be essential to ensure that the integrity and security of the systems and the data within them are maintained at all times.
4. There is a recognised gap in data used for NEM forecasting and planning – information on energy consumption and the changing drivers of energy demand is limited.
5. There is currently no data strategy for the NEM – a data strategy should be developed that will, among other benefits, provide the ability to track the key performance indicators of the NEM and refine the plan as needed in the future.

Background

The Finkel Report is the product of an extensive examination of all aspects of the future of the NEM. (See our previous Client Updates Finkel: the solution for our energy future? for a broad overview, Finkel Review – Gas: A revolving door or real change? for a more specific look into gas and Finkel Review – Distributed Energy Resources: Unlocking Potential and future partnerships for a more detailed look into DER.)

The Finkel Report addresses critical concerns in relation to access to and use of data in the NEM. The uptake of new technologies and the promotion of innovation requires better access to energy data. The way in which regulators balance consumer protections, including requirements for customer consent, with a need for real time access in an appropriate format will of course require further investigation. From a whole of market perspective, with increases in DER, the security and stability of the market will largely rest on the overall visibility of data.

Importantly, Dr Finkel recommends that new data collection frameworks or mechanisms should be used to increase the visibility of all forms of DER and a new data strategy for the NEM should be established.

Key recommendations and opportunities

Set out below are the key recommendations outlined in the Finkel Report in relation to data.

• By mid-2020, the COAG Energy Council should facilitate measures to remove complexities and improve consumers' access to, and rights to share, their energy data.

A focus on access to energy data is not a new theme in energy reform. The Power of Choice reform package, introduced in 2012, proposed changes to enhance customers' ability to access consumption data and the ability to share that data with approved service providers. While subsequent rule changes have allowed consumers and their authorised representatives to obtain consumption data from their electricity retailer or distributor free of charge up to four times a year, a number of practical barriers still limit consumers’ ability to easily access and use this information. For example, under the current arrangement, it can take up to 10 days to receive the data and the format may vary. In an ideal world – and certainly one we should be aiming for in the future –, accurate data would be available in the format required to be used in an app or website immediately. New rules commence in December this year to facilitate competition in metering. Under the new rules, it will be the role of a 'Metering Coordinator' to facilitate access to data to approved parties. Both Metering Coordinators and retailers will have a role to play in improving access to energy data in line with Dr Finkel's recommendations.  

Dr Finkel states that consumers should have access to their electricity consumption data in real time and they should also have control over who, if anyone, can access that data. A key issue for policy makers will be the need to balance consumer protections with simple procedures for immediate access to accurate data to approved parties. At present, questions remain as to who owns metering data and what can be done with that data. As part of these proposed reforms, further clarity will be needed on this point.

These changes will provide software developers with opportunities to continue building solutions for retailers, metering coordinators and other service providers to facilitate access to energy data in various formats. As consumers have increased access to data, it will also drive decision making in the uptake of new technologies, presenting opportunities for retailers and other service providers to offer that technology and the associated services.

• The COAG Energy Council, in addition to its project on energy storage systems, should develop a data collection framework (or other mechanism) to provide static and real-time data for all forms of DER at a suitable level of aggregation. The project should be completed by mid-2018.

As we noted in our article on Distributed Energy Resources: Unlocking Potential and future partnerships, the visibility of DER is a high-priority challenge. The Australian Energy Market Operator (AEMO) is currently proceeding with an engagement process with key industry stakeholders to identify the viability of different options for a future data collection framework. Some stakeholders have suggested that a market approach to the wider challenge of DER orchestration will implicitly solve the visibility problem as it would, in theory, provide an incentive for DER owners to provide their data in order to participate in the market.

Policy makers will need to grapple with what the regulatory framework for hosting and sharing the data will look like. Issues such as who is responsible for data collection and maintenance and who has access to the data will be key considerations in designing the future framework.

As highlighted in our last update on DER, AEMO's increasing data collection needs will provide opportunities for those with telemetry technologies, data analytics, modelling and existing mechanisms and frameworks for data collection. Increased data collection needs will also provide opportunities for those developing advanced operating and communication systems, as well as software for stable and efficient operation.

• By end 2018, the Energy Security Board, in collaboration with the AER, should develop a data strategy for the NEM. The initial design of the data strategy must be developed in consultation with industry bodies and consumer bodies, and be consistent with open government data principles. The Energy Security Board must report to the COAG Energy Council on the completion of the first stage. This should include costs for design and implementation for initial set up, plus indicative costs for ongoing maintenance of the key deliverables under the data strategy. The first phase of the data strategy must be completed by end-2017, with the functionality of the components of the strategy reviewed annually to ensure that they continue to be fit-for-purpose.

A data strategy for the NEM has been flagged as a critical data need in the Finkel Report. The aim of the data strategy is broad. Dr Finkel notes the strategy should enhance the availability of data that can support changes in the market, including data that can assist the use of DER to reduce overall system costs and improve reliability and security. It should also address barriers to accessing information by balancing the needs for particular types of information (including to support emerging services and competition) with the open government data principles.

The strategy is broken down into a staged approach in the report, as follows:

Phase 1: The initial program of work should include:

1. a rigorous gap analysis, to be undertaken in consultation with industry and other relevant stakeholders. It should consider whether additional measures are needed in terms of market bodies' powers to collect and share information (for example, whether there is a need for a formalised agreement between market bodies and government agencies explicitly detailing information sharing arrangements); the effectiveness of data sharing in operational systems in terms of managing future challenges and supporting innovation; and the publication of other relevant data sets.
2. an initiative to develop a catalogue of energy market data and publications. This will help overcome the barrier of not knowing what information is available, who publishes it, and where to get it. Data.gov.au should be considered as the platform for this resource.

Phase 2: A medium-term program of work to enhance and consolidate existing data platforms and disseminate up-to-date information in three ways:

1. a dashboard that provides transparency on the performance of the NEM against security and reliability, affordability and emissions reduction objectives, allowing progress to be tracked. The dashboard would be a 'single source of truth' for this information. It should be easy to navigate, and combine high-level information with the ability to drill down to specific areas.
2. an ongoing Energy Use Data Model. This will support AEMO with demand forecasting, as well as helping new entrants to assess market opportunities.
3. a mapping platform that provides a single view into the NEM in terms of generation, networks and consumption, to allow new opportunities to be more easily understood. It should combine the features of the Energy Use Data Model, Australian Renewable Energy Mapping Infrastructure (AREMI) and AEMO’s interactive map of planning information to provide detailed spatial information regarding predicted generation opportunities, network constraints, expansion plans and support requirements. All network operators should be incentivised to maintain up-to-date information about constraints and opportunities that are forecast to occur in their network.

Phase 3: A longer-term process whereby stakeholders can request access to non-public data for research and development purposes. This process will ensure that unanticipated future needs can be met and future innovation supported. It must consider the benefits of the work that the data would enable, the potential risks to privacy, security, and any existing commercial confidentiality agreements, and the costs of making it available.

The benefits and opportunities coming from improved access to data

The Finkel Report highlights that improved confidence, understanding and management of the NEM will be largely achieved through increased availability and transparency of data. Significant challenges have been raised in relation to the operation of current data systems as a result of new technologies. The increasing development and uptake of  smart meters, smart grids, DER and the Internet of Things means electricity service providers and regulators are dealing with unprecedented volumes of data. Of course, there is significant value in this information but capturing it, understanding it and using it to offer services and to operate the NEM will take time and further reforms. There is no doubt that security will be strengthened when we achieve a greater overview of energy data and a clear regulatory framework for responsibility of, ownership and access to that data. As Dr Finkel highlights, the availability of timely and accurate data will enable improved decision making across all areas of the NEM, including market operation, investment, policy development and regulatory enforcement.

The proposed changes are not surprising and policy makers have been grappling with these issues for some time. While Dr Finkel promotes an accelerated approach to identifying and implementing solutions, this process will require in-depth industry consultation and time to properly develop, given the number of risks and complexities that will need to be assessed and balanced in the final approach.

The takeaway point for businesses is that the focus on increasing access to data will naturally promote the uptake of technology and ongoing innovation. This will bring significant opportunities for retailers, metering coordinators and service providers to develop and implement new technology and associated services for the benefit of consumers and the overall management of the NEM. To the extent that participants are not already doing so, it will be important to start thinking about the technology that will be required in order to offer services and to compete in a market that will soon have easy access to valuable data, creating opportunities that have not previously been available in the energy industry.

As Dr Finkel recommends throughout the report, in order to implement these reforms, consultation with industry and other relevant stakeholders will be required. As a next step, industry participants may consider their own views on these issues in preparation for consultation with policy makers.

We will be keeping clients up to date on the progress of these key data recommendations.