The conformance suite is setup to run tests against a number of different environments nightly and before any code changes are merged into the master branch. The results are made public and viewable here:

https://staging.certification.openid.net/plans.html?public=true

More detailed results are show in the latest pipeline results:

https://gitlab.com/openid/conformance-suite/pipelines (look at the latest pipeline for master and click through into the 'deployment_test_staging' job log file).

We are very happy to add further environments to these tests. Vendors or banks make wish to take advantage of this in order to show "continuous conformance", and also because it helps prevent regressions in the conformance suite that may cause false failures when running in your tests. (Please note this it is still necessary to complete the certification process as per https://openid.net/certification/instructions/ to be listed as a certified implementation/deployment)

The requirements to be added to the automated tests are:

1. The environment must be available 24/7 and generally of close to production stability/quality. Fintechlabs reserves the right to remove any test environment that regularly fails, regresses or otherwise requires anything beyond very minimal effort from our engineers.
2. Any issues (regressions/downtime) in the environment should be resolved within 2 business days. Any non-conformances to the relevant specifications found should be resolved within 5 business days unless otherwise agreed.
3. The authorisation server must not require any complicated authentication - no SMS or OTP, no "enter letters 2 and 4 of your password", no push verification to a mobile device.
4. The vendor must provided a fully complete and passing configuration for the conformance suite with the redirect urls specified below.
5. The configuration must included full browser automation so that no user interaction is required and the test runs fully automatically. (TODO: add link here to the browser automation documentation)
6. A new issue should be raised in this project containing links to the conformance suite plan results page showing the relevant test plans passing and contact details for the person responsible for maintaining the environment.

Fintechlabs is happy to assist you in meeting these requirements or to help you understand the reason for any test failures; please email joseph.heenan@fintechlabs.io for details.

Required redirect urls

The provided clients must accept all these redirect urls as registered/valid.

Exceptions may be allowed if for overriding technical reasons it is not possible for the localhost:8443 or localhost.emobix.co.uk:8443 urls to be registered. This is at our discretion.

Please replace 'vendorname' in the urls with a unique identifier for your company / the test environment. The unique identifier (with 'openidinternal-' on the beginning) is set in the test configuration as the 'alias' field.

https://www.certification.openid.net/test/a/openidinternal-vendorname/callback
https://staging.certification.openid.net/test/a/openidinternal-vendorname/callback
https://demo.certification.openid.net/test/a/openidinternal-vendorname/callback
https://localhost:8443/test/a/openidinternal-vendorname/callback
https://localhost:8443/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://localhost.emobix.co.uk/test/a/openidinternal-vendorname/callback
https://localhost.emobix.co.uk/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://localhost.emobix.co.uk:8443/test/a/openidinternal-vendorname/callback
https://localhost.emobix.co.uk:8443/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://review-app-dev-branch-1.certification.openid.net/test/a/openidinternal-vendorname/callback
https://review-app-dev-branch-2.certification.openid.net/test/a/openidinternal-vendorname/callback
https://review-app-dev-branch-3.certification.openid.net/test/a/openidinternal-vendorname/callback
https://review-app-dev-branch-4.certification.openid.net/test/a/openidinternal-vendorname/callback
https://review-app-dev-branch-5.certification.openid.net/test/a/openidinternal-vendorname/callback
https://review-app-dev-branch-6.certification.openid.net/test/a/openidinternal-vendorname/callback
https://review-app-dev-branch-7.certification.openid.net/test/a/openidinternal-vendorname/callback
https://review-app-dev-branch-8.certification.openid.net/test/a/openidinternal-vendorname/callback
https://review-app-dev-branch-9.certification.openid.net/test/a/openidinternal-vendorname/callback
https://staging.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://www.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://demo.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://review-app-dev-branch-1.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://review-app-dev-branch-2.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://review-app-dev-branch-3.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://review-app-dev-branch-4.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://review-app-dev-branch-5.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://review-app-dev-branch-6.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://review-app-dev-branch-7.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://review-app-dev-branch-8.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum
https://review-app-dev-branch-9.certification.openid.net/test/a/openidinternal-vendorname/callback?dummy1=lorem&dummy2=ipsum