Conformance Test Suite process
The Australian Competition and Consumer Commission (ACCC) manages the Conformance Test Suite, which is a key part of the Consumer Data Right on-boarding process.
The Consumer Data Right Conformance Test Suite confirms the technical conformance of your production-ready software using a range of test scenarios targeting specific areas.
The Conformance Test Suite has 2 test suites – one for each type of provider (also known as participant): data holders and data recipients. If you are a data recipient, the tests provide a simulated data holder and a mock Register to support the test scenarios. You can test in isolation against the simulated providers and the mock Consumer Data Right Register, so you don’t interfere with live consumer data.
The diagram below shows how data recipients each interact with the Conformance Test Suite.
The Conformance Test Suite is not a testing tool to assist you during the development of your software. Rather, it is available to you during on-boarding, once you are accredited as a data recipient before you become activated on the Register.
The Conformance Test Suite tests your conformance with the Consumer Data Standards before entering into the Consumer Data Right system. You should have a production-ready software product before undertaking the Conformance Test Suite.
Completing the Conformance Test Suite
The Conformance Test Suite is an automated testing suite and can be completed within a short period (for example, within an hour) providing all provider configuration is completed correctly and no errors are encountered. So, it is important for you to ensure your solution has been adequately tested before executing the Conformance Test Suite to minimise the test completion period.
The completion period can be lengthy (for example, days or weeks) if issues are encountered, as errors need to be diagnosed and resolved, and your solution made ready for retesting.
Conformance Test Suite tests for data recipients
There are 4 key tests included in the Conformance Test Suite for accredited data recipients.
- Dynamic client registration tests whether the data recipient can successfully obtain an access token, a list of data holder brands, and a Software Statement Assertion (SSA) from the Register for dynamic client registration, and could register its software product with the Conformance Test Suite data holder.
- Consent test tests for request authorisation for once-off data sharing request and data sharing request with duration. It also tests for Pushed Authorisation Requests (PAR) via the backchannel and request authorisation via reference_uri.
- Consent withdrawal test tests for initiating withdrawal of consent to data sharing, notifying a data holder and receiving withdrawal of consent (or authorisation for data holders) notifications from a data holder. It also tests for initiating withdrawal of consent arrangement and receiving withdrawal of consent arrangement from a data holder.
- API test tests for initiating calls to Get Accounts and Get Transactions For Account banking application programming interface (APIs).
Except for the dynamic client registration test, all tests can be run multiple times during the test run. The result of the last testing attempt will be included in the test run report for the Conformance Test Suite outcome assessment.
More information about the test cases will be made available to providers who have begun on-boarding with the ACCC.
Using the Conformance Test Suite to test as a data recipient
Conformance Test Suite accredited data recipient guidance material provides more information about how to prepare, execute and complete the Conformance Test Suite.
Data recipients need to follow these steps for testing using the Conformance Test Suite.
- pass accreditation
- complete and submit your Conformance Test Suite enrolment form
- sign and submit your Conformance Test Suite acknowledgement form, ACCC PKI Subscriber Agreement and Relying Party Agreement
- receive your test certificate and apply it to your software product
- review the technical instructions on how to start Conformance Test Suite tests.
- complete any IP whitelisting required to interact with the Conformance Test Suite
- add the Conformance Test Suite data holder to the data holder list your software product transacts with
- create a client of your software product to simulate a consumer
- any other configuration details the ACCC On-boarding Team communicated to you.
- access the Conformance Test Suite Register via the link provided in the guidance material, to begin discovery
- execute the test cases in the recommended order.
- pass all the tests specified on your Conformance Test Suite enrolment form
- request the ACCC On-boarding Team to send the Conformance Test Suite test run report
- send the results to the ACCC On-boarding Team for final assessment.
Timeline for future releases
The Conformance Test Suite is being developed over time and its functions will expand to cover critical points of failure. The Conformance Test Suite is already available for providers to begin testing, with future iterations to be delivered in 2021 to include wider testing, usability enhancements and other important technical aspects to support version 2 of the Consumer Data Right Rules.
Conformance Test Suite roadmap
The planned roadmap for the data recipient Conformance Test Suite is provided below.
Previous Conformance Test Suite release notes
The October 2020 release included:
- dynamic client registration
- single (once off, with duration)
- concurrent consents (arrangement_id)
- consent management (arrangement API)
- PAR (modify consents)
- revoke (token endpoint)
- revoke (arrangements)
- banking API.