Landmark Productivity Commission report on data availability and use

In brief

The Productivity Commission has released a landmark draft report on the ability of individuals, businesses and government to access and use data in Australia. The report criticises Australia's historically conservative approach to data use and proposes a 'fundamental and systematic change' to the way that data is made available and linked. Partner Gavin Smith, Managing Associate Valeska Bloch, Associate Tom Griffin and Lawyer Claudia Hall report. 

Introduction

Background

Over the past two years, restrictions on access to, and availability of, data have been highlighted in multiple inquiries and reports across a number of different sectors, such as the Financial System Inquiry 2014 (the Murray Inquiry) and the Review of Competition Policy 2015 (the Harper Review).

On 21 March 2016, Scott Morrison issued terms of reference to the Productivity Commission to review the benefits and costs of increasing the availability of public and private sector data generally and to review options to improve individuals' access to data about themselves. In response to the terms of reference, the Commission has consulted with, and received submissions from, government agencies, industry and other stakeholders. It released a draft report on 3 November 2016.

The Commission will continue to accept written submissions on the draft report until 12 December 2016, and has scheduled public consultations in Melbourne on 21 November 2016 and in Sydney on 28 November 2016.

The final report is scheduled for release in March 2017.

The thesis

The draft report acknowledges that the 'extraordinary' growth in the generation and usability of data, coupled with increases in computing power and data analytics skills has enabled the emergence of a 'kaleidoscope of new business models, products and insights', to the benefit of both Australian businesses and consumers. However, the Commission observes that the current legal framework and the myriad of policy requirements and approval processes that govern the access to and use of data in Australia have created an entrenched culture of risk aversion.² As a consequence (and to the detriment of Australia's economy in a competitive global market) agencies and organisations generally deny access to data, as the easiest way to minimise their risk. This approach is out of step with competing economies like the United States, the United Kingdom and New Zealand.

The report argues that data is a valuable asset, not merely a risk or an overhead and should be better utilised by government, businesses and consumers. The Commission's recommended approach is underpinned by the following four key areas of focus:

• giving individuals more control over data held on them;
• enabling broad access to both public and private sector datasets that are of national interest;
• increasing the usefulness of publicly funded identifiable data amongst trusted users; and
• creating a culture in which non-personal and non-confidential data is released by default for widespread use.

These areas are consistent with the terms of reference, and are reflected in the Commission's 27 recommendations which include proposed changes to legislative and governance structures.

Regulatory framework

Legislative reform

The report proposes the introduction of 'umbrella' Commonwealth legislation – the Data Sharing and Release Act (the DSRA) – which would apply to all 'digital data' in Australia.

If introduced, the DSRA will, among other things:

• require government agencies to share and release data between government agencies and other sectors, except in limited circumstances (likely to be where the data includes identifiable or commercially sensitive information);
• grant individuals certain rights in relation to data held about them; and
• establish a framework to govern access to, and the sharing and linking of datasets, in Australia (with a particular, but not exclusive, focus on the public sector).³

As a consequence, more than 500 secrecy and privacy provisions in Commonwealth legislation, policies and guidelines which currently limit the availability and use of identifiable data, will need to be reviewed and potentially modified, revoked or overridden by the DSRA.

The report acknowledges that although it intends to retain the key protections within the existing Privacy Act 1988 (Cth),⁴ the provisions of the Privacy Act will only apply to the extent that they are not inconsistent with the DSRA.⁵

The report also proposes revising the penalties under the Privacy Act which relate to the misuse of data, to ensure that data custodians, such as ARAs and government agencies, will not be liable for the misuse, by third parties, of data shared or released by the custodian. While this may increase the number of datasets disclosed by reducing the potential risks for the disclosing custodian, it could also decrease the avenues of recourse for those whose information has been misused, especially where the misusing entity cannot be identified.⁶

These proposals would mark a significant addition to the current privacy law framework.

Government standards

The report proposes the development of sector-specific data management standards to support increased data availability and use among government agencies. The report states that the standards would be released in a draft form for consultation by the end of 2017 and would be implemented by 2020.⁷

Industry self-regulation

For the private sector, the Commission prefers industry developed sector-specific standards for data sharing between firms, proposing the areas of superannuation and retail banks as industries which would particularly benefit from such standards.⁸ However, if voluntary approaches are deemed to be insufficient, the Commission recommends that the government should facilitate the development of such standards when it is in the public interest to do so (which may, for example, be where regulation requiring the collection of data will provide an information advantage to incumbent businesses).⁹

We have already observed considerable progress in this space. Allens is a founder member of the newly established Data Governance Australia, the peak industry body for organisations involved with data collection, access and use.

Governance and oversight

The National Data Custodian

The report recommends that a new national organisation, the National Data Custodian (NDC), be established by the DSRA to oversee the operation of the new national data system, designate datasets of national interest and accredit release authorities and trusted users.¹⁰ Whereas the Australian Information Commissioner would have a continuing role in relation to data protection and privacy, the NDC would focus on data access and use.

The NDC's main functions would be to:

• select datasets of value to be nominated as NIDs;
• determine when datasets are too sensitive to release;¹¹
• implement processes to designate high-value datasets for broader access and use;
• determine funding and allocation to ARAs for the management, storage and access of datasets;
• set prices for organisations to access datasets; and
• facilitate the development of standards for dataset curation and storage.¹²

Accredited Release Authorities

Accredited Release Authorities (ARAs) with particular sectoral expertise will be accredited by the NDC to:

• determine whether a dataset should be made available for public release or limited sharing with trusted users; and
• collate, curate and ensure the timely updating and maintenance of NIDs.¹³

ARAs will also provide advice to both government and the broader community of data custodians and users in relation to technical matters. ARAs will largely be public sector agencies which already release data, such as the Australian Institute of Health and Welfare, and will receive funding to take on these additional responsibilities.¹⁴

Existing bodies with new roles

The following existing bodies will also be responsible for additional roles as follows:

• consumer access to data (and the levying of fees on consumers to access or transfer that data) and the entire administration of the 'comprehensive' right regime will be governed by the ACCC;¹⁵
• privacy and the handling of privacy complaints will be governed by the OAIC;
• the assessment of data sharing and release disputes and appeals will be undertaken by the Administrative Appeals Tribunal; and
• sector specific issues will be the responsibility of the relevant industry ombudsman or regulators, or, where no clear ombudsman or regulator exists in relation to the sector, the OAIC will effectively act as a 'backup ombudsman'.¹⁶

Consumer data

The report proposes a new concept of 'consumer data' to be introduced into applicable legislation, including the DSRA. Consumer data encompasses personal information (as defined in the Privacy Act), all files posted online by a consumer, all data derived from a consumer's online transactions or internet-connected activities and all other data associated with a consumer's transactions or activity that is relevant to the transfer of data to a nominated third party.¹⁷

The report distinguishes, in similar manner to the framework in the EU and UK, between 'personal information' (identifiable information subject to restrictions on collection, use and disclosure) and 'consumer data' (data subject to a transfer right, and the other rights comprising the 'comprehensive right').¹⁸ This distinction is necessary to exclude 'copyrighted data analytics works, parts of proprietary statistical models and data collected for law enforcement or national security purposes' from the ambit of consumer data, and accordingly the comprehensive right.¹⁹

The concept of consumer data includes 'other data derived from consumers' online transactions of internet-related activity'.²⁰ However, for practical reasons consumer data will not include information which has been so substantially altered that it can no longer be linked back to the relevant individual within the entity's systems. The report also suggests that information which has been de-identified will not fall within the definition of consumer data.²¹

A new 'comprehensive right'

The report proposes the introduction of a new 'comprehensive right' entitling individuals to access 'consumer data' about themselves that is digitally held by businesses, government agencies and government business enterprises.

Although certain elements of this right already exist under the Privacy Act (eg the right to access, request edits or correction under Australian Privacy Principles 12 and 13), new elements include a right for consumers to:

• access digitally held data about themselves (whether collected from the individual, a third party, or created, for example through re-identification);
• be informed of any intention to disclose or sell data about them. It is unclear from the report whether this right requires pro-active notification to the consumer each time there is a disclosure or sale, or whether it will be satisfied by the current process of notifying consumers through a privacy policy. The Commission is seeking views on what methods of disclosure would be most likely to result in consumers making a meaningful choice about how their personal information is being used. Our view is that a requirement to provide a notification each time a disclosure occurs would be overly burdensome on business and would have a deleterious rather than positive effect on innovation;
• obtain a copy of their data in a machine-readable form and transfer themselves, or require the entity to transfer, the copy to another service provider or third party; and
• opt-out of having their data collected, unless such collection is necessary for a public benefit purpose (eg public health and safety or taxes), necessary to satisfy legal obligations or claims, where the information forms part of an NID or where the collection is necessary for the continued delivery of a product or service to the individual. This right goes further than the existing right in Australian Privacy Principle 2 which allows individuals to use a pseudonym or choose not to identify themselves when their information is being collected.

The report proposes that entities be required to retain consumer data for a minimum period of one year, but sets out that where consumer data is retained for a longer period it will continue to be subject to the comprehensive right.²²

The comprehensive right is intended to apply equally to the private and public sector. Companies will therefore need to implement processes to identify what consumer data they hold in relation to particular consumers and to respond to requests from consumers seeking to exercise their comprehensive right.

The report proposes that guidance be provided to businesses on what is considered 'reasonable steps' to ascertain an individual's identity for the purpose of providing access to or transferring their consumer data. However, it acknowledges that there may be costs incurred in storing, retrieving and supplying data to consumers under this comprehensive right.²³

Management of HVDs

The report defines 'public sector data' as data held by government agencies (at all levels) and entities which receive public funding, for example government business enterprises, universities and research institutes.²⁴ In keeping with the focus on open access, the report suggests that all public sector information be released, except where national security or other compelling public interest considerations ought to prevent such release.²⁵

HVDs

However, so as not to overwhelm individuals and agencies, the report proposes prioritising the release of 'high value datasets'. HVDs are likely to be unique (or unable to be replicated), of high quality, to have a high degree of coverage in the relevant population, and/or be up to date or updated regularly.²⁶ While these datasets may not meet the significant public interest test to be designated as NIDs, they will nonetheless provide a public benefit and should be released for broader access and use. It is expected these datasets will largely consist of public sector data, for example data relating to health, natural hazards, education and welfare,²⁷ however certain private datasets may also constitute HVDs.²⁸

Access to HVDs by the public sector and Trusted Users

Where there is a risk of identification or re-identification of individuals or of confidential or government sensitive information being disclosed in relation to a dataset (as may be the case with a proportion of datasets created by publicly funded service providers such as private hospitals), the dataset should still be made available, but only within the public sector and only to accredited trusted users. Accredited trusted users may include federal, state and local government agencies, Australian universities, as well as other specific entities covered by the privacy legislation (including not-for-profits, corporates and research bodies) (Trusted Users).²⁹

Release of datasets

The report proposes that all data (including metadata) held by government agencies or publicly funded entities that is not confidential and not related to individual businesses or people, must be released in a catalogued and searchable format on a site such as data.gov.au by 1 October 2017.³⁰

In relation to HVDs, the report proposes a system whereby government agencies, researchers and the private sector can nominate, or vote on the nomination of, datasets or combinations of datasets for public release as HVDs.³¹ Where the request is approved, the relevant ARA should publish the dataset publicly on a site like data.gov.au, along with any conditions of access.

Where data is highly sensitive and disclosure would not be in the public interest, the ARA should, in the interest of transparency, publish a notice of their decision to refuse to release the dataset, or a notice indicating that an unspecified dataset has been assessed as 'not available'.³²

All datasets which are released via data.gov.au must be able to be accessed and used by all users, notwithstanding any restrictions set out in other pieces of legislation or policy. The report indicates that released data may include information which might identify individuals but which is already in the public domain in some form, for example, property ownership information.³³

Management of NIDs

The report also proposes the implementation of a process to enable the nomination and designation of public and private datasets that are of significant public interest as NIDs. Unlike HVDs, NIDs will not be subject to any of the restrictions on collection, use or disclosure set out in the Privacy Act.

NIDs are a subset of HVDs, as while many datasets are of a high value, only some of them may be used to generate substantial benefits across such a broad swathe of the Australian population that they can be characterised as being of 'national interest'.³⁴ Criteria which might signal that a dataset is of national interest are:

• the dataset is of interest to a broad range of users, and would generate broad economic and social benefits if access to it was extended;
• the dataset can be used as a basis for comparison between states and territories; and/or
• the dataset has a national focus.³⁵

Nomination and designation of NIDs

Although the process to designate NIDs will be open to state and territories, designation will ultimately occur via disallowable instrument on the recommendation of the NDC.

The Commission is seeking views on the establishment of a Parliamentary committee (or an alternative body or process) to receive community input on possible NIDs, to review nominations made, and to propose future designations.

Release of NIDs

Similar to HVDs, any designated NID which does not contain identifiable or commercially sensitive information will be released immediately. Where an NID contains identifiable information, it will initially be released to Trusted Users, with the intention to eventually release such data publicly on a de-identified basis.³⁶

Once data has been designated as being part of an NID, all restrictions on access to, and use of, the data, such as the requirement to obtain consent, under the Privacy Act or any other piece of Commonwealth or state legislation or policy will cease to apply, and all access and use will be dealt with under the new DSRA. The report states that this process is intended to 'cleanse valuable data of existing or future encumbrances on its broader use'.³⁷

Access to NIDs by Trusted Users

As outlined above, Trusted Users organisations will be accredited by the NDC to access NIDs which are not to be released publicly. Any access to an NID by a Trusted User will be subject to an ongoing access arrangement with the applicable accredited organisation. While not expressly stated in the report, we expect that not all Trusted Users will be provided with access to all NIDs. This will need to be considered on a project basis or on the basis of pre-approved categories.

Listing NIDs

All NIDs that have been publicly released or are potentially available to share, along with the relevant data custodian and ARA for that dataset, will (similar to other HVDs) be listed on a central website such as data.gov.au.

The private sector

Private sector NIDs

While the report expresses hesitance to intervene in the information flows between businesses, it notes that where the private sector has an incentive to restrict access to information which would improve the functioning of markets, or where there are significant public benefits which would arise from the disclosure of private sector data, the government may be justified in intervening to ensure broader access to data in the public interest.

In making this suggestion, the report distinguishes between two categories of private sector entities:

• commercial entities that are subject to regulation that permits or requires the collection of certain data in fulfilling certain public interest obligations (eg banks, health insurance funds and energy providers) (Regulated Entities); and
• entities that do not acquire data to comply with a regulatory requirement or as a result of public funding (Unregulated Entities).³⁸

The Commission claims that Regulated Entities have a competitive advantage because the regulatory requirements compelling their collection of certain data have allowed private 'data monopolies' to emerge. Regulated Entities also have a weaker incentive to meet the interests of their customers in making customer data available to them. It argues that because of the regulatory requirements imposed on them, Regulated Entities effectively enter into a regulatory contract with their customers and that 'in return for the benefits conferred by the regulation', Regulated Entities should be required to release or share data where there is a net public benefit from doing so. The result, the report claims, will be greater competition and innovation in the market, particularly where the absence of data is a barrier to entry. It will also give customers a greater opportunity to make informed consumer-related decisions based on their own consumer data.

This is the premise on which the report suggests that certain private sector datasets may be designated as NIDs, including, for example:

• data collected in the course of meeting requirements set by regulators (eg ACCC, ACMA, TIO);
• data collected through the course of the provision of services on behalf of the government;
• data collected in the course of outsourcing government operations (such as in the provision of transport or electricity services); and
• health insurance data.

It is also the premise on which the report proposes requiring certain Regulated Entities to provide datasets to, for example, a regulator, without that dataset being designated as an NID, where there is a net public benefit from the release of the data (taking into account the negative impact on, or cost for, the individual business).³⁹ It is not clear from the report whether the datasets which will be released to regulators will be HVDs or a separate third category of datasets. In a speech to the FINSIA regulators panel on 11 November 2016, the Chairman of ASIC, echoed the potential public benefits which arise from providing more data to, and enabling the greater use of data by, regulators. ASIC has recently established a Chief Data Office to govern ASIC's data assets as well as the third party data they access and use.⁴⁰ We expect that this trend of regulators making better use of the data they are provided by the private sector is only likely to increase over time.

The report acknowledges that the benefits and costs of designating private sector datasets as NIDs, or requiring them to otherwise be released, will need to be rigorously assessed and that further analysis of the interaction between the public interest and data sharing is likely to be conducted for the final report.

Contracting with government

The report suggests that all Federal government entities which enter into contracts with the private sector, for example in the delivery of public services, should assess the strategic significance and public interest value of the data which might be created before contracting. Where it is considered valuable, the agency should retain the right to access or purchase the data in a machine readable form, or to perform analysis on the data.⁴¹

The report also recommends that all new contracts or funding agreements entered into by the government should incorporate terms and conditions which allow the government to collect and access any data created in connection with such contract or agreement.

This will need to be considered by businesses when contracting with the government. It is important to note that regardless of the contractual position, data related to, or created by, government service providers could be designated as falling within an NID in which case it must be disclosed without restriction.

Comprehensive credit reporting

Although the Commission acknowledges that there has been a slow uptake of voluntary comprehensive credit reporting (CCR) in Australia, it considers that it is still too early to determine that the scheme in its existing voluntary form will fail. The report consequently proposes the adoption of a minimum target for voluntary participation in comprehensive credit reporting of 40 per cent of accounts by 30 June 2017. If this target is not achieved, the Commission suggests that the government should circulate draft legislation to impose mandatory reporting by 31 December 2017.⁴²

De-identification

The report proposes that the OAIC, in collaboration with the Australian Bureau of Statistics and other agencies with de-identification expertise, create and publish guidance for best practice de-identification processes, which would apply to both business and government agencies.⁴³ The report also proposes to give the OAIC the power to certify when an entity is using best practice de-identification processes.

Notwithstanding the benefit of certainty around de-identification obligations such a certification process would provide, this proposal raises significant practical questions for the OAIC, most notably if re-identification were to take place and given the rapid pace of technological developments in data science, how long the period of such a certification would be.

Interestingly, the report avoids addressing whether the new Commonwealth Bill to criminalise re-identification of datasets is consistent with the thesis that government agencies should avoid defaulting to the worst case scenario and avoid releasing the data altogether.

Valuing and pricing data

The report raises interesting questions around the pricing and funding of access to data.

One of the proposals is that for public sector datasets, agencies only undertake the minimum level of work to enable the dataset to be easily accessed by individuals, and that such datasets be made freely available or available at marginal cost.⁴⁴ Otherwise, the report recommends that an independent review consider the pricing of public sector datasets for publicly funded research and other public interest purposes.⁴⁵

Similarly, the report suggests that government business enterprises only undertake minimum value adding processes for social objectives, but undertake value adding for internal or commercial purposes on the basis of prevailing commercial considerations.

In relation to private sector data, the report acknowledges that the market will both provide a value and a price for datasets, which will be largely determined by:

• demand for the data;
• the existence of accessible alternatives;
• the extent to which it needs to be processed in order to be used;
• the potential uses to which the data can be put; and
• strategic leverage, that is, how willing the business will be to relinquish exclusive control over the data and its use.⁴⁶

The report proposes that where a demonstrable public interest is only capable of being served by the release of private sector data, the release should occur with the least practical cost and the maximum public benefit. As such, it suggests that in most scenarios government should not compensate the private sector for the release of their datasets, although it does not rule out compensation altogether.⁴⁷

Public interest test for researchers

The report proposes that data created as a result of publicly funded research should be made available beyond the initial researchers to, potentially, all researchers in the relevant field.⁴⁸ The Commission also proposes extending the exceptions to obtaining consent for use of identifiable information in health and medical research⁴⁹ to a more general exception for all research which is in the public interest.⁵⁰ The OAIC will publish guidance on what inputs are required to establish a public interest case.

Health data

Despite the potential benefits of using linked data, as outlined in the draft report, there is an inherent tension between the collection, use and disclosure of health information and privacy laws. Although this can usually be resolved by using de-identified information, the draft report acknowledges that risks of misuse and re-identification problems apply in relation to the management and use of high-value public interest datasets.

The recommendation to amend existing Commonwealth privacy legislation and create the new DSRA is interesting in relation to health information, taking into account:

• the recently announced Bill to amend the Privacy Act to make it a criminal offence to re-identify, or to counsel, procure, facilitate or encourage the re-identification of, de-identified government data or to publish or communicate a re-identified dataset; and
• the interaction that will be required between any such new legislation and existing health legislation (not just privacy legislation) such as the My Health Records Act 2012 (Cth).

For more detailed, sector-specific analysis of the report, such as the impact on the healthcare sector, see our subsequent releases in this series.

Looking forward

Notwithstanding that this is a draft report, the open and accessible approach championed by the Productivity Commission signifies a potential step change in the attitude towards the access and use of data in Australia.

The report proposes a framework which will dramatically shift the way in which data is thought about and managed by government, the private sector and individuals. While greater access to public and private datasets is likely to improve the insights that can be gained about population trends and may improve the setting of public policy, it is also likely to impose a cost on private entities, both in relation to compliance and through increased levels of competition. Government and legislators will also need to carefully consider how, in practice, various pieces of legislation, such as the Privacy Act, the recent Re-Identification Bill and state and federal health laws will work together under the new framework. Crucially, in attempting to implement this framework, the government must ensure that data is provided in a meaningful and understandable way to consumers and the broader public and is not released in quantities which are overwhelming, and that personal and commercial-in-confidence information is protected.

Further, in developing the final report, the Commission will need to clarify the overlap between consumer data and personal information, the kinds of, and the extent to which, private sector data will need to be shared or released, and the process by which NIDs will be designated.

Please let us know if we can assist you in preparing a submission to the Productivity Commission in response to the draft report.