Package test coverage
This issue collects the current package test coverage information
Frontend
Frontend Test Coverage Details
The data is coming from the latest master pipeline: https://gitlab-org.gitlab.io/-/gitlab/-/jobs/855067204/artifacts/coverage-frontend/lcov-report/index.html
Note the links are broken in the following table
Statements | Branches | Functions | Lines | ||||||
app/assets/javascripts/packages/details |
|
26.67% | 4/15 | 0% | 0/2 | 0% | 0/3 | 26.67% | 4/15 |
app/assets/javascripts/packages/details/components |
|
99.13% | 114/115 | 89.29% | 25/28 | 97.44% | 38/39 | 99.13% | 114/115 |
app/assets/javascripts/packages/details/store |
|
96.72% | 59/61 | 52.17% | 12/23 | 96.43% | 27/28 | 96.43% | 54/56 |
app/assets/javascripts/packages/list |
|
80.49% | 33/41 | 87.5% | 7/8 | 60% | 3/5 | 80% | 32/40 |
app/assets/javascripts/packages/list/components |
|
93.75% | 90/96 | 73.08% | 19/26 | 93.18% | 41/44 | 93.75% | 90/96 |
app/assets/javascripts/packages/list/stores |
|
93.33% | 56/60 | 100% | 9/9 | 91.67% | 22/24 | 94.44% | 51/54 |
app/assets/javascripts/packages/shared |
|
100% | 22/22 | 92.31% | 12/13 | 100% | 4/4 | 100% | 21/21 |
app/assets/javascripts/packages/shared/components |
|
95.24% | 40/42 | 92.86% | 13/14 | 96% | 24/25 | 95.24% | 40/42 |
app/assets/javascripts/registry/explorer |
|
25.81% | 8/31 | 71.43% | 5/7 | 20% | 2/10 | 25.81% | 8/31 |
app/assets/javascripts/registry/explorer/components |
|
100% | 13/13 | 100% | 2/2 | 100% | 9/9 | 100% | 13/13 |
app/assets/javascripts/registry/explorer/components/details_page |
|
98.46% | 64/65 | 93.75% | 30/32 | 96.67% | 29/30 | 98.46% | 64/65 |
app/assets/javascripts/registry/explorer/components/list_page |
|
100% | 51/51 | 100% | 21/21 | 100% | 18/18 | 100% | 51/51 |
app/assets/javascripts/registry/explorer/constants |
|
100% | 63/63 | 100% | 0/0 | 100% | 0/0 | 100% | 63/63 |
app/assets/javascripts/registry/explorer/pages |
|
93.06% | 67/72 | 100% | 23/23 | 87.88% | 29/33 | 93.06% | 67/72 |
app/assets/javascripts/registry/explorer/stores |
|
100% | 85/85 | 83.33% | 10/12 | 100% | 41/41 | 100% | 82/82 |
app/assets/javascripts/registry/settings |
|
42.86% | 6/14 | 0% | 0/2 | 0% | 0/2 | 42.86% | 6/14 |
app/assets/javascripts/registry/settings/components |
|
98.15% | 53/54 | 100% | 25/25 | 96.55% | 28/29 | 98.15% | 53/54 |
app/assets/javascripts/registry/settings/graphql |
|
0% | 0/2 | 100% | 0/0 | 100% | 0/0 | 0% | 0/2 |
app/assets/javascripts/registry/settings/graphql/utils |
|
100% | 7/7 | 100% | 0/0 | 100% | 3/3 | 100% | 6/6 |
app/assets/javascripts/registry/shared |
|
92.68% | 38/41 | 0% | 0/2 | 81.82% | 9/11 | 94.87% | 37/39 |
app/assets/javascripts/registry/shared/components |
|
90.48% | 19/21 | 100% | 20/20 | 83.33% | 10/12 | 90.48% | 19/21 |
app/assets/javascripts/vue_shared/components/registry |
|
100% | 32/32 | 100% | 18/18 | 100% | 22/22 | 100% | 32/32 |
Folders containing bootstrapping functions and small utilities that we typically do not test in the frontend
Conclusions
- We have identified the following underpeformers:
- app/assets/javascripts/packages/details
- app/assets/javascripts/registry/explorer
- app/assets/javascripts/registry/settings
- app/assets/javascripts/registry/settings/graphql
Ruby
Ruby Test Coverage Details
```$ ./bin/ptcs backend -oConclusions
- We can see that the less covered files are mainly in models.
- The previous result of "some apis are not covered" doesn't appear which proves that the previous approach can lead to inaccurate results.
- The worst file has a coverage of 85% which is excellent.
GoLang
GoLang Test Coverage Details
Integration Tests
Each of these tests are ran against the three most recent versions of go, but we'll only report for the latest version for brevity.
Job Name | Coverage | Link |
---|---|---|
api Go 1.15 | 44.7% | https://gitlab.com/gitlab-org/container-registry/-/jobs/855728220 |
database:api Go 1.15 | 64.6% | https://gitlab.com/gitlab-org/container-registry/-/jobs/855728217 |
database:datastore Go 1.15 | 78.8% | https://gitlab.com/gitlab-org/container-registry/-/jobs/855728214 |
database:migration Go 1.15 | 90.3% | https://gitlab.com/gitlab-org/container-registry/-/jobs/855728211 |
filesystem storage driver Go 1.15 | 75.4% | https://gitlab.com/gitlab-org/container-registry/-/jobs/855728224 |
inmemory storage driver Go 1.15 | 82.4% | https://gitlab.com/gitlab-org/container-registry/-/jobs/855737095 |
s3 storage driver Go 1.15 | 61.9% | https://gitlab.com/gitlab-org/container-registry/-/jobs/855728236 |
swift storage driver Go 1.15 | 62.7% | https://gitlab.com/gitlab-org/container-registry/-/jobs/855728232 |
API Integration Tests Total Coverage
These tests were ran with coverpkg=all
meaning that the percentage reported is for the coverage that these tests have for all packages. These should better represent the depth of coverage achieved by these tests.
Job Name | Coverage | Link |
---|---|---|
api Go 1.15 | 20.5% | https://gitlab.com/gitlab-org/container-registry/-/jobs/855829072 |
database:api Go 1.15 | 22.4% | https://gitlab.com/gitlab-org/container-registry/-/jobs/855829067 |
Unit Tests
? github.com/docker/distribution [no test files]
? github.com/docker/distribution/cmd/digest [no test files]
? github.com/docker/distribution/cmd/registry [no test files]
? github.com/docker/distribution/cmd/registry-api-descriptor-template [no test files]
ok github.com/docker/distribution/configuration 0.074s coverage: 84.8% of statements
ok github.com/docker/distribution/context 2.022s coverage: 70.6% of statements
ok github.com/docker/distribution/contrib/token-server 0.059s coverage: 11.8% of statements
ok github.com/docker/distribution/digestset 0.019s coverage: 88.9% of statements
ok github.com/docker/distribution/health 0.021s coverage: 51.1% of statements
ok github.com/docker/distribution/health/api 0.008s coverage: 88.9% of statements
ok github.com/docker/distribution/health/checks 0.157s coverage: 60.0% of statements
? github.com/docker/distribution/manifest [no test files]
ok github.com/docker/distribution/manifest/manifestlist 0.013s coverage: 77.2% of statements
ok github.com/docker/distribution/manifest/ocischema 0.026s coverage: 75.4% of statements
ok github.com/docker/distribution/manifest/schema1 0.041s coverage: 66.0% of statements
ok github.com/docker/distribution/manifest/schema2 0.022s coverage: 81.1% of statements
? github.com/docker/distribution/metrics [no test files]
? github.com/docker/distribution/migrations [no test files]
ok github.com/docker/distribution/notifications 0.949s coverage: 80.5% of statements
ok github.com/docker/distribution/reference 0.026s coverage: 80.0% of statements
ok github.com/docker/distribution/registry 9.326s coverage: 24.5% of statements
ok github.com/docker/distribution/registry/api/errcode 0.058s coverage: 46.4% of statements
ok github.com/docker/distribution/registry/api/v2 0.388s coverage: 79.4% of statements
? github.com/docker/distribution/registry/auth [no test files]
ok github.com/docker/distribution/registry/auth/htpasswd 0.102s coverage: 78.5% of statements
ok github.com/docker/distribution/registry/auth/silly 0.040s coverage: 48.1% of statements
ok github.com/docker/distribution/registry/auth/token 0.054s coverage: 72.0% of statements
ok github.com/docker/distribution/registry/client 0.129s coverage: 65.3% of statements
ok github.com/docker/distribution/registry/client/auth 0.036s coverage: 74.9% of statements
ok github.com/docker/distribution/registry/client/auth/challenge 0.046s coverage: 91.4% of statements
? github.com/docker/distribution/registry/client/transport [no test files]
ok github.com/docker/distribution/registry/datastore 0.064s coverage: 5.0% of statements
? github.com/docker/distribution/registry/datastore/models [no test files]
? github.com/docker/distribution/registry/datastore/testutil [no test files]
ok github.com/docker/distribution/registry/handlers 15.047s coverage: 14.7% of statements
? github.com/docker/distribution/registry/listener [no test files]
? github.com/docker/distribution/registry/middleware/registry [no test files]
? github.com/docker/distribution/registry/middleware/repository [no test files]
ok github.com/docker/distribution/registry/proxy 9.575s coverage: 45.8% of statements
ok github.com/docker/distribution/registry/proxy/scheduler 0.575s coverage: 60.6% of statements
ok github.com/docker/distribution/registry/storage 14.914s coverage: 73.7% of statements
ok github.com/docker/distribution/registry/storage/cache 0.005s coverage: 47.4% of statements
? github.com/docker/distribution/registry/storage/cache/cachecheck [no test files]
ok github.com/docker/distribution/registry/storage/cache/memory 0.012s coverage: 93.4% of statements
? github.com/docker/distribution/registry/storage/cache/metrics [no test files]
? github.com/docker/distribution/registry/storage/cache/redis [no test files]
? github.com/docker/distribution/testutil [no test files]
ok github.com/docker/distribution/uuid 0.010s coverage: 68.6% of statements
? github.com/docker/distribution/version [no test files]
Conclusions
- We have identified the following underperformers:
- github.com/docker/distribution/contrib/token-server
- github.com/docker/distribution/health
- github.com/docker/distribution/health/checks
- github.com/docker/distribution/registry
- github.com/docker/distribution/registry/api/errcode
- github.com/docker/distribution/registry/auth/silly
- github.com/docker/distribution/registry/proxy
- github.com/docker/distribution/registry/storage/cache
Unit Test Summary by Functional Area
Functional Area | Lines of Coverage |
---|---|
javascript | 90.13% |
ruby | 99.33 % |
golang | 63.53% |
Upcoming Testing Focus
- Create
real world
test cases for the Package stage - Package QA Test Suite
- Create QA tests for the Container Registry
- Expand Testing of the Container Registry Authentication Service to Include Write-Only Deploy Tokens
- Test group/subgroup support for the NuGet repository
- Review and determine a plan to address noted underperformers across functional areas
- Review and determine a plan to improve GoLang Integration Test Coverage
Edited by John Hampton