GitLab issueshttps://gitlab.com/gitlab-org/gitlab/-/issues2023-09-12T18:02:24Zhttps://gitlab.com/gitlab-org/gitlab/-/issues/362494Maven sync worker fails with invalid metadata.2023-09-12T18:02:24ZDavid FernandezMaven sync worker fails with invalid metadata.## :fire: Problem
We got an alarm on the Maven sync worker due to too many failures.
On [inspection](https://log.gprd.gitlab.net/goto/46908940-d4fb-11ec-aade-19e9974a7229) (internal), it seems that all failures are
```
metadata_conten...## :fire: Problem
We got an alarm on the Maven sync worker due to too many failures.
On [inspection](https://log.gprd.gitlab.net/goto/46908940-d4fb-11ec-aade-19e9974a7229) (internal), it seems that all failures are
```
metadata_content is invalid
Packages::Maven::Metadata::SyncWorker::SyncError
app/workers/packages/maven/metadata/sync_worker.rb:37:in `perform'
```
This indicates that the [`SyncService`](https://gitlab.com/gitlab-org/gitlab/-/blob/ca2318f4fa7c973c6b9ece259d763834d1d33355/app/services/packages/maven/metadata/sync_service.rb) is not successful in its execution.
We assume that the error comes from creating the versions xml file, given that the errors we see target android applications and we really doubt that this error comes from the plugin versions xml file.
So this [function](https://gitlab.com/gitlab-org/gitlab/-/blob/7baa6a66fe82c5ab49762a7a95ca2fc903922eb6/app/services/packages/maven/metadata/create_versions_xml_service.rb#L40-42) seems to fail = the expected xml nodes are not present in the metadata file.
[Kibana search](https://log.gprd.gitlab.net/goto/82b564a0-d518-11ec-aade-19e9974a7229).
## :fire_engine: Solution
* Get a metadata file content from an existing package that produces the error.
* Investigate how we can handle this situation instead of making the work fail.Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/424238Database race condition when uploading maven packages2024-03-14T14:00:55ZCaleb WilliamsonDatabase race condition when uploading maven packages<!---
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "type::bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://g...<!---
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "type::bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=type::bug
and verify the issue you're about to submit isn't a duplicate.
--->
### Summary
When using official maven images tagged with `eclipse-temurin`(e.g. `maven:3.9.3-eclipse-temurin-11`) it is possible to encounter a race condition when uploading packages, ultimately resulting in the following output within the job and a `{"message":"Validation failed: Name has already been taken"}` error within our logs:
```
Could not transfer artifact calebw:maven-test2:pom:0.59.40 from/to gitlab-maven (https://gitlab.com/api/v4/projects/12345678/packages/maven): status code: 400, reason phrase: Bad Request (400)
```
This seems to **only occur** when **both** of the following conditions are true:
- An official maven image tagged with `eclipse-temurin` is used
- The flag `-DdeployAtEnd=true` is passed as a Maven argument, which configures all the packages to be pushed at the end.
After speaking with @10io, it seems like this is occurring due to how these specific images send the requests when the above conditions are met. It looks like we receive two authorize requests sequentially to the `/authorize` endpoint (one for the pom, one for the jar) before a request to upload either one of these files is made.
### Steps to reproduce
Fork [this project](https://gitlab.com/calebw/mvn-400-bad-request) and run a pipeline. This project utilizes predefined CI variables and should need no configuration in order to build and attempt to publish packages to it's relevant package registry.
This project has a parent/child pom configuration that publishes multiple packages. You will see the job fail with the above error on a random package, and it will fail when attempting to publish either the POM or JAR file. Removing the `-DdeployAtEnd=true` flag from the `.gitlab-ci.ym` or changing the image to `maven:3.9.3` will allow the job to succeed.
Relevant logs can be found in Kibana after a failed attempt is made by searching on the following: `json.path: /api/v4/projects/PROJECT_ID/packages/maven/NAMESPACE/maven-test/*`.
### Example Project
https://gitlab.com/calebw/mvn-400-bad-request
### What is the current *bug* behavior?
Deploy attempts will fail with a `400 bad request` output in the job, and a `{"message":"Validation failed: Name has already been taken"}` error on the backend.
### What is the expected *correct* behavior?
All packages are deployed without issue.
### Relevant logs and/or screenshots
<!-- Paste any relevant logs - please use code blocks (```) to format console output, logs, and code
as it's tough to read otherwise. -->
### Output of checks
<!-- If you are reporting a bug on GitLab.com, uncomment below -->
This bug happens on GitLab.com
### Possible fixes
Implement support for parallel uploads. This could be challenging but we could have some leads:
Use upsert this way, the database itself will take care of the race condition. The challenge here is that we need to have the proper unique constraints in place.
Use an exclusive lease on the backend so that second threads wait for the first one to finish. Not ideal but could work.Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/394813Maven package registry: move checks from upload to authorize endpoint2024-03-19T00:23:05ZDavid FernandezMaven package registry: move checks from upload to authorize endpoint## :fire: Problem
The ~"Maven Repository" uses direct uploads for its uploads. We're not going to detail all the aspects here but basically:
1. workhorse intercepts the upload.
2. workhorse asks rails: hey where do you want me to put th...## :fire: Problem
The ~"Maven Repository" uses direct uploads for its uploads. We're not going to detail all the aspects here but basically:
1. workhorse intercepts the upload.
2. workhorse asks rails: hey where do you want me to put this (request on the `/authorize` endpoint).
3. workhorse uploads the file to the location pointed by rails.
4. workhorse call the rails upload endpoint with a "file" parameter which is a "pointer" to the location where the file has been uploaded (request on the upload endpoint).
We see that rails receives `2` requests during the upload (step (2.) and (4.)).
Currently, step (2.) checks are fairly simple:
* check that the user has the right permissions.
* check that the workhorse header is present.
Then on step (4.), we have additional checks:
* `.md5` file + `FIPS`
* duplicates check (users can control if duplicates are allowed or not).
The issue is that those additional checks could be done in step (2.). By doing so, we would save step (3.) which is an interaction with object storage. Also that would mean one less thing to do for workhorse.
## :fire_engine: Solution
* Move the additional checks from steps (2.) and (4.)
* For step (4.), extract the duplication check into its own service.
* :warning: Use a ~"feature flag".Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/374571Cloning project over SSH fails when using gemnasium-python image2024-03-07T22:37:04ZDuncandrharris@gitlab.comCloning project over SSH fails when using gemnasium-python image### Summary
SSH checkout fails when using the Gemnasium-Python image
### Steps to reproduce
1. Configure a CI/CD job to check out a project via SSH using the `gemnasium-python:3` image
### Example Project
https://gitlab.com/gitlab-g...### Summary
SSH checkout fails when using the Gemnasium-Python image
### Steps to reproduce
1. Configure a CI/CD job to check out a project via SSH using the `gemnasium-python:3` image
### Example Project
https://gitlab.com/gitlab-gold/duncan/sshclone/
This project uses the same `before_script`, `script`, and project variables. The only change is job image.
### What is the current *bug* behavior?
SSH authentication fails with error `debug1: read_passphrase: can't open /dev/tty: No such device or address`
### What is the expected *correct* behavior?
Authentication should succeed.
### Relevant logs and/or screenshots
https://gitlab.com/gitlab-gold/duncan/sshclone/-/jobs/3042046163
```
$ ssh -Tvvv git@gitlab.com
OpenSSH_8.4p1 Debian-5+deb11u1, OpenSSL 1.1.1n 15 Mar 2022
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'
debug2: resolving "gitlab.com" port 22
debug2: ssh_connect_direct
debug1: Connecting to gitlab.com [172.65.251.78] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.4p1 Debian-5+deb11u1
debug1: Remote protocol version 2.0, remote software version GitLab-SSHD
debug1: no match: GitLab-SSHD
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to gitlab.com:22 as 'git'
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,ssh-ed25519,sk-ssh-ed25519@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-s
debug2: host key algorithms: ssh-dss,ecdsa-sha2-nistp256,ssh-ed25519,rsa-sha2-256,rsa-sha2-512,ssh-rsa
debug2: ciphers ctos: aes128-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr
debug2: ciphers stoc: aes128-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none
debug2: compression stoc: none
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw
debug1: read_passphrase: can't open /dev/tty: No such device or address
Host key verification failed.
```
### Output of checks
This bug happens on GitLab.com
#### Results of GitLab environment info
GitLab Enterprise Edition 15.4.0-pre 8ef12c3857d
#### Results of GitLab application Check
### Possible fixes
<!-- If you can, link to the line of code that might be responsible for the problem. -->BacklogYasha RiseYasha Risehttps://gitlab.com/gitlab-org/gitlab/-/issues/343530Intermittent issues with publishing maven packages to the registry2023-10-24T01:47:50ZKate GrechishkinaIntermittent issues with publishing maven packages to the registry<!---
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://gitlab....<!---
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=bug
and verify the issue you're about to submit isn't a duplicate.
--->
### Summary
Sometimes, when publishing maven packages to GitLab maven package registry, the `PUT` event results in 500 error which fails the whole pipeline. This can be quite frustrating because a deploy job needs to be manually restarted to complete the deployment. When retrying the same pipeline with zero changes, the same package can be successfully deployed to the maven registry.
Searching through the logs, I have found ~200 events of 500 errors for the past 7 days:
```
Rack::Timeout::RequestTimeoutException
Request ran for longer than 60000ms
```
### Steps to reproduce
The issue is transient, no steps to reproduce, unfortunately, but customers that heavily use maven registry encounter it quite often. One of such customers has contacted us in [this ticket](https://gitlab.zendesk.com/agent/tickets/245891) to raise our awareness.
### Example Project
You can find the projects that experience this issue by running this query in Kibana:
https://log.gprd.gitlab.net/goto/90677fb8197acadf455753ed2de07e22
### What is the current *bug* behavior?
Intermittent 500 errors when publishing packages to maven repository
### What is the expected *correct* behavior?
Users don't receive intermittent 500 errors when publishing packages to maven repos.
### Relevant logs and/or screenshots
```
app/uploaders/object_storage.rb:404:in `block in cache_remote_file!',
app/uploaders/object_storage.rb:403:in `tap',
app/uploaders/object_storage.rb:403:in `cache_remote_file!',
app/uploaders/object_storage.rb:367:in `cache!',
app/services/packages/create_package_file_service.rb:12:in `execute',
lib/api/maven_packages.rb:276:in `block (2 levels) in <class:MavenPackages>',
ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call',
ee/lib/gitlab/ip_address_state.rb:10:in `with',
ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call',
lib/api/api_guard.rb:213:in `call',
ee/lib/omni_auth/strategies/group_saml.rb:41:in `other_phase',
lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call',
lib/gitlab/middleware/rails_queue_duration.rb:33:in `call',
lib/gitlab/middleware/speedscope.rb:13:in `call',
lib/gitlab/request_profiler/middleware.rb:17:in `call',
lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call',
lib/gitlab/metrics/rack_middleware.rb:16:in `block in call',
lib/gitlab/metrics/web_transaction.rb:46:in `run',
lib/gitlab/metrics/rack_middleware.rb:16:in `call',
lib/gitlab/jira/middleware.rb:19:in `call',
lib/gitlab/middleware/go.rb:20:in `call',
lib/gitlab/etag_caching/middleware.rb:21:in `call',
lib/gitlab/middleware/multipart.rb:178:in `block in call',
lib/gitlab/middleware/multipart.rb:63:in `with_open_files',
lib/gitlab/middleware/multipart.rb:177:in `call',
lib/gitlab/middleware/read_only/controller.rb:50:in `call',
lib/gitlab/middleware/read_only.rb:18:in `call',
lib/gitlab/middleware/same_site_cookies.rb:27:in `call',
lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call',
lib/gitlab/middleware/basic_health_check.rb:25:in `call',
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call',
lib/gitlab/middleware/request_context.rb:21:in `call',
config/initializers/fix_local_cache_middleware.rb:11:in `call',
lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call',
lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call',
lib/gitlab/metrics/requests_rack_middleware.rb:75:in `call',
lib/gitlab/middleware/release_env.rb:12:in `call'
```
### Output of checks
This bug happens on GitLab.com
### Possible fixes
<!-- If you can, link to the line of code that might be responsible for the problem. -->Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/288045Maven package registry access with Gradle using Personal Access Tokens is inc...2023-10-24T02:03:02ZDirk HasselbalchMaven package registry access with Gradle using Personal Access Tokens is inconsistent.### Summary
Maven package registry access is inconsistent.
1. Personal Access Token with `read_api` scope *does* give access if building via Gradle with empty Gradle cache using project/group-ids.
2. Personal Access Token with `read_ap...### Summary
Maven package registry access is inconsistent.
1. Personal Access Token with `read_api` scope *does* give access if building via Gradle with empty Gradle cache using project/group-ids.
2. Personal Access Token with `read_api` scope *does not* give access if building via Gradle with primed Gradle cache using project/group-ids. Gradle fails during https HEAD call.
Documentation https://docs.gitlab.com/ee/user/packages/maven_repository/index.html is not precise regarding authentication for a) using a maven registry for dependencies and b) publishing. In a), read-only should suffice. In b), you should need write permission. See https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/packages/maven_repository/index.md#L278.
Also, documentation for `read_api` under https://gitlab.com/-/profile/personal_access_tokens means that it should be possible to use `read_api` tokens only when using maven registry for dependencies. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/locales/doorkeeper.en.yml#L80.
### Steps to reproduce
1. Place a maven package in a Gitlab Maven package registry.
2. Create a gradle project with a dependency to that maven package.
3. Authenticate with a personal access token with `read_api` scope, and use the group/project id in the registry path.
4. The build will succeed if there is no prior Gradle cache.
5. The build will fail if there is a primed Gradle cache.
### Example Project
https://gitlab.com/maven-registry-test/libraryconsumer
### What is the current *bug* behavior?
When gradle cache is empty, gradle succeeds.
When gradle cache is primed, and refresh cache is triggered, gradle fails with the error (example):
```
Could not HEAD 'https://gitlab.com/api/v4/ ... pom'. Received status code 403 from server: Forbidden
```
### What is the expected *correct* behavior?
The gradle build should succeed in both cases (if it is the intended behaviour that `read_api` should indeed give read access to the maven package registry, as stated in https://gitlab.com/-/profile/personal_access_tokens).
### Current work-around
Use personal access tokens with `api` (i.e. read/write) scope, or use deploy or job tokens.
### Relevant logs and/or screenshots
See example project for full logs.
```
$ ./gradlew classes
Downloading https://services.gradle.org/distributions/gradle-6.6.1-bin.zip
.........10%..........20%..........30%..........40%.........50%..........60%..........70%..........80%..........90%.........100%
Welcome to Gradle 6.6.1!
Here are the highlights of this release:
- Experimental build configuration caching
- Built-in conventions for handling credentials
- Java compilation supports --release flag
For more details see https://docs.gradle.org/6.6.1/release-notes.html
Starting a Gradle Daemon (subsequent builds will be faster)
> Task :compileJava
> Task :processResources
> Task :classes
BUILD SUCCESSFUL in 58s
2 actionable tasks: 2 executed
$ ./gradlew clean classes --refresh-dependencies
> Task :clean
> Task :compileJava FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':compileJava'.
> Could not resolve all files for configuration ':compileClasspath'.
> Could not resolve com.trifork.sandbox:library:0.0.1.
Required by:
project :
> Could not resolve com.trifork.sandbox:library:0.0.1.
> Could not get resource 'https://gitlab.com/api/v4/groups/10202943/-/packages/maven/com/trifork/sandbox/library/0.0.1/library-0.0.1.pom'.
> Could not HEAD 'https://gitlab.com/api/v4/groups/10202943/-/packages/maven/com/trifork/sandbox/library/0.0.1/library-0.0.1.pom'. Received status code 403 from server: Forbidden
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 34s
```
### Output of checks
This bug happens on Gitlab.com.
### Possible fixes
1. Make `HEAD` http(s) calls to maven registries respect the same access control policies as `GET` calls.
2. Update documentation on https://docs.gitlab.com/ee/user/packages/maven_repository/index.html to make it clear that different permissions are required for using a maven registry as a dependency (read-only) and publishing to a maven registry (read/write).Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/443358Maven metadata.xml corruption2024-03-04T21:01:13ZCleveland Bledsoe JrMaven metadata.xml corruption### Summary
A customer encountered an issue with a corrupt `maven-metadata.xml` file during a build. It was unclear on why or how the file became corrupted, so it would be challenging to attempt to reproduce. The intention of the issue ...### Summary
A customer encountered an issue with a corrupt `maven-metadata.xml` file during a build. It was unclear on why or how the file became corrupted, so it would be challenging to attempt to reproduce. The intention of the issue is to document it to determine if others are experiencing similar issues and determine how it may have ended up in this state.
Error in the build for a project:
```
Could not read metadata /.m2/repository/path/to/file/maven-metadata-xx.xml: no more data available - expected end tags </lastUpdated></versioning></metadata> to close start tag <lastUpdated> from line 79 and start tag <versioning> from line 5 and start tag <metadata> from line 2, parser stopped on TEXT seen ...<lastUpdated>20240213044202</la... @79:36
```
Note the missing closing tag below:
```
104 <version>1.1.1-RELEASE</version>
105 <version>2.2.1-RELEASE</version>
106 <version>3.3.1-RELEASE</version>
107 <version>4.4.1-RELEASE</version>
108 <version>5.5.1-RELEASE</version>
109 </versions>
110 <lastUpdated>20240213044202</la
```
### Steps to reproduce
No reproduction
### Example Project
See [ZD Ticket](https://gitlab.zendesk.com/agent/tickets/503680) (Internal) for reference
### What is the current *bug* behavior?
Corrupt Maven metadata files can prevent builds and proper usage of maven commands
### What is the expected *correct* behavior?
Maven metadata files should not become corrupt or have a mechanism to regenerate under certain conditions.
### Relevant logs and/or screenshots
<!-- Paste any relevant logs - please use code blocks (```) to format console output, logs, and code
as it's tough to read otherwise. -->
### Output of checks
This bug happens on GitLab.com 16.10.0-pre 64f5301ea4f
### Workarounds
Delete a relevant package from the package registry. This seemingly re-creates the metadata (sync_worker?)
### Possible fixes
<!-- If you can, link to the line of code that might be responsible for the problem. -->Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/404733Parallel Maven uploads lead to 409 Conflict responses2023-10-24T01:20:46ZDavid FernandezParallel Maven uploads lead to 409 Conflict responses## :fire: Problem
In https://gitlab.com/gitlab-org/gitlab/-/issues/367356#note_1341809603, we discovered that the ~"Maven Repository" could receive parallel publications for the exact same package name and version.
That could lead to u...## :fire: Problem
In https://gitlab.com/gitlab-org/gitlab/-/issues/367356#note_1341809603, we discovered that the ~"Maven Repository" could receive parallel publications for the exact same package name and version.
That could lead to upload the same file (eg. same filename) at the same time. The problem is that maven clients will upload a `.sha1` digest after upload a file.
We have these events:
1. Client A: uploads file `foobar.txt` for package `test`, version `1.2.3`.
1. Client B: uploads file `foobar.txt` (different contents) for package `test`, version `1.2.3`. This is now the most recent `foobar.txt` file. This is the file that will considered for checking `.sha1` digests.
1. Client B: uploads file `foobar.txt.sha1`. It matches the `sha1` of (2.), the backend will reply `200 Ok`
1. Client A: uploads file `foobar.txt.sha1`. This is the `sha1` of (1.). It doesn't match the `sha1` of (2.), the backend will return `409 Conflict`. :boom:
This issue is to discuss if the backend should handle (4.) in a different way or not.
### :stadium: Additional considerations
* Parallel maven publications is not a common thing to do but I can see it happening with 2 CI pipelines running and publishing the exact same package name+version.
* Maven already has a workaround for this situation: snapshot versions. Snapshot versions will add a suffix that is set by using a timestamp. This means that two parallel snapshot uploads can perfectly be handled.
* Is it technically wrong to answer `409` in 4? Well, the ~backend always work with the most recent files, _always_. If it receives a `sha1` from an older file, then there is something fishy (parallel upload) about the upload. It could be ok to keep it as it is.
* Measuring impact. At the time of this writing, in the last 24 hours, ~"gitlab.com" received:
* `72 538` successful `sha1` uploads (`204 Created`)
* `48` conflict errors in `sha1` uploads (`409 Conflict`)
* `409 Conflict` represents about `0.07%` of all uploads.
* Please note that the upload of the `.sha1` file will fail _but_ the publication maven command will still go on and even be successful. In other words, a failing `sha1` upload does not interrupt the maven package publication.
* Given the challenges on the possible solutions (see below), I'm not sure that this issue is worth solving.
## :fire_engine: Solution
Things are not simple as maven clients will not send an identifier to link all uploads of the same package publication. In fact, they do this but _only_ for snapshot versions.
Here are some leads for the solution:
* Inspect older files to see of the uploaded `sha1` exists.
* Use something like a lease so that parallel uploads can't happen.
The concerns I have with the above solutions is that we could have edge cases. Verifying a `.sha1` is a useful feature. When the backend returns an error for a `.sha1`, it indicates a very precise situation: my most recent file doesn't match the signature you sent. This means that we could have an issue with the most recent file.Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/369059Maven Packages: The response body should be a String, but it is of type NilClass2024-03-19T00:23:17ZBotMaven Packages: The response body should be a String, but it is of type NilClasshttps://gitlab.com/gitlab-org/gitlab/-/blob/39d35cb97853f4b5dc5e7b46a9d911ec2f34b9d9/lib/api/api_guard.rb#L227 is catching this: the API endpoint is returning a `nil` body instead of a string.
https://sentry.gitlab.net/gitlab/gitlabcom/...https://gitlab.com/gitlab-org/gitlab/-/blob/39d35cb97853f4b5dc5e7b46a9d911ec2f34b9d9/lib/api/api_guard.rb#L227 is catching this: the API endpoint is returning a `nil` body instead of a string.
https://sentry.gitlab.net/gitlab/gitlabcom/issues/3352847/?referrer=gitlab_plugin
```
ArgumentError: The response body should be a String, but it is of type NilClass
```Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/327179Publishing fails, but publish2023-10-24T02:00:30ZAndret2344Publishing fails, but publishWhen trying to publish jar using gradlew publish, the job fails, in log I can see HTTP code 400, but... artifact is correctly published and available for download using maven/gradle. About a week ago it worked well.
Failing pipeline: ht...When trying to publish jar using gradlew publish, the job fails, in log I can see HTTP code 400, but... artifact is correctly published and available for download using maven/gradle. About a week ago it worked well.
Failing pipeline: https://gitlab.com/andret-tools-system/ats-parkour/-/jobs/1148279494Awaiting further demandhttps://gitlab.com/gitlab-org/gitlab/-/issues/300022Unauthorized (401) : Mule Maven packages in the Package Repository2023-10-24T02:02:02ZGandeev kumarUnauthorized (401) : Mule Maven packages in the Package Repository### Summary
<!-- Summarize the bug encountered concisely. -->
Until now we have used Jfrog Repo for publishing artifacts. However, now we have decided to use the Maven Package registry for publishing artifacts. But while we are using th...### Summary
<!-- Summarize the bug encountered concisely. -->
Until now we have used Jfrog Repo for publishing artifacts. However, now we have decided to use the Maven Package registry for publishing artifacts. But while we are using the Mule Maven plugin to create and deploy the packages to the GitLab maven repository, the GitLab CI job is throwing a 401 Unauthorized error. I have tried using Deploy-Token, Job-Token and private token and the result are the same. Then I have tried plain maven projects, I'm able to publish them to the maven package registry.
### Steps to reproduce
I have followed the steps as part of the document link below:
https://docs.gitlab.com/ee/user/packages/maven_repository/
### Example Project
I have created a sample mule project:
* **Sample Project**: https://gitlab.com/ranaboddeti/mulesoft-sample-project
* **Failed Jobs**:
* https://gitlab.com/ranaboddeti/mulesoft-sample-project/-/pipelines/246984857
### What is the current *bug* behavior?
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy (default-deploy) on project mulesoft-cicd-sample1: Failed to deploy artifacts: Could not transfer artifact com.mycompany:mulesoft-cicd-sample1:pom:1.0.0-20210126.131829-1 from/to gitlab-maven (https://gitlab.com/api/v4/projects/23934631/packages/maven): Unauthorized (401)
### What is the expected *correct* behavior?
Jar file should be published in the maven package registry similar to how it is getting published from the plain maven project
### Relevant logs and/or screenshots
```64161 [INFO]
64161 [INFO] --- maven-deploy-plugin:2.8.2:deploy (default-deploy) @ mulesoft-cicd-sample1 ---
64216 [INFO] No primary artifact to deploy, deploying attached artifacts instead.
64610 [INFO] ------------------------------------------------------------------------
64610 [INFO] BUILD FAILURE
64610 [INFO] ------------------------------------------------------------------------
64611 [INFO] Total time: 01:02 min
64611 [INFO] Finished at: 2021-01-26T13:18:29+00:00
64698 [INFO] Final Memory: 21M/90M
64698 [INFO] ------------------------------------------------------------------------
64699 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy (default-deploy) on project mulesoft-cicd-sample1: Failed to deploy artifacts: Could not transfer artifact com.mycompany:mulesoft-cicd-sample1:pom:1.0.0-20210126.131829-1 from/to gitlab-maven (https://gitlab.com/api/v4/projects/23934631/packages/maven): Unauthorized (401) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy (default-deploy) on project mulesoft-cicd-sample1: Failed to deploy artifacts: Could not transfer artifact com.mycompany:mulesoft-cicd-sample1:pom:1.0.0-20210126.131829-1 from/to gitlab-maven (https://gitlab.com/api/v4/projects/23934631/packages/maven): Unauthorized (401)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Failed to deploy artifacts: Could not transfer artifact com.mycompany:mulesoft-cicd-sample1:pom:1.0.0-20210126.131829-1 from/to gitlab-maven (https://gitlab.com/api/v4/projects/23934631/packages/maven): Unauthorized (401)
at org.apache.maven.plugin.deploy.DeployMojo.deployProject(DeployMojo.java:284)
at org.apache.maven.plugin.deploy.DeployMojo.execute(DeployMojo.java:185)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: org.apache.maven.artifact.deployer.ArtifactDeploymentException: Failed to deploy artifacts: Could not transfer artifact com.mycompany:mulesoft-cicd-sample1:pom:1.0.0-20210126.131829-1 from/to gitlab-maven (https://gitlab.com/api/v4/projects/23934631/packages/maven): Unauthorized (401)```Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/222365Unauthorized (401): While deploying mule maven artifacts to GitLab maven repo...2023-10-24T02:06:25Zsameer shaikUnauthorized (401): While deploying mule maven artifacts to GitLab maven repository<!---
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://gitlab....<!---
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=bug
and verify the issue you're about to submit isn't a duplicate.
--->
### Summary
While using the [Mule maven plugin](https://docs.mulesoft.com/mule-runtime/4.3/mmp-concept) to create and deploy the packages to the GitLab maven repository, the GitLab CI job is throwing a 401 Unauthorized error.
### Steps to reproduce
```
deploy:
script:
- 'cp ci_settings.xml /root/.m2/settings.xml'
- cat /root/.m2/settings.xml
- 'mvn deploy:deploy-file -Dfile=mule-template-app-1.0.3-SNAPSHOT-mule-xxx.jar -Durl=https://gitlab.com/api/v4/projects/xxxxx/packages/maven -DrepositoryId=gitlab-maven -DgroupId=com.xxx.xxx.xxx.xxx -DartifactId=mule-template-app -Dpackaging=mule-application -Dversion=1.0.3-SNAPSHOT'
only:
- master
image: maven:latest
```
```
<settings>
<servers>
<server>
<id>gitlab-maven</id>
<configuration>
<httpHeaders>
<property>
<name>Job-Token</name>
<value>${env.CI_JOB_TOKEN}</value>
</property>
</httpHeaders>
</configuration>
</server>
</servers>
</settings>
```
### Example Project
- **Sample Repo**: https://gitlab.com/s_shaik/maven-401
- **Failed Jobs**:
- * https://gitlab.com/s_shaik/maven-401/-/jobs/714013145
- * https://gitlab.com/s_shaik/cmule/-/jobs/714061994
### What is the current *bug* behavior?
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy-file (default-cli) on project mule-4-template-project-F2: Failed to deploy artifacts: Could not transfer artifact 0dd19ae0743567:xxxxx-project-F2:jar:1.0.3 from/to gitlab-maven (https://gitlab.com/api/v4/projects/xxxxx/packages/maven): Unauthorized (401) -> [Help 1]
### What is the expected *correct* behavior?
Instead of 401 - the package should deploy
### Relevant logs and/or screenshots
```
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.142 s
[INFO] Finished at: 2020-05-13T16:56:00+00:00
[INFO] Final Memory: 15M/56M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy-file (default-cli) on project mule-4-template-project-F2: Failed to deploy artifacts: Could not transfer artifact 0dd19ae0743567:xxxxx-project-F2:jar:1.0.3 from/to gitlab-maven (https://gitlab.com/api/v4/projects/xxxxx/packages/maven): Unauthorized (401) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
```
### Output of checks
Happens on GitLab.comBackloghttps://gitlab.com/gitlab-org/gitlab/-/issues/354820Discussion: should the maven-metadata.xml be shown on the package UI2022-10-17T00:27:14ZKatie MacoyDiscussion: should the maven-metadata.xml be shown on the package UI## Context
When the user allows Maven duplicates, each publish to the registry adds
* the `maven-metadata.xml` to the versionless package (package with *no* version)
* the .pom and .jar files to the package with a version.
When consumi...## Context
When the user allows Maven duplicates, each publish to the registry adds
* the `maven-metadata.xml` to the versionless package (package with *no* version)
* the .pom and .jar files to the package with a version.
When consuming the package, only the latest `maven-metadata.xml` is served. The user cannot access the older `maven-metadata.xml` files on the UI or API.
## A note about snapshots
Currently Maven snapshots do show the `maven-metadata.xml` in the UI.
Snapshot packages don't have `1` but `2` `maven-metadata.xml` files:
1. One in the versionless package (<- this one exists for *all* maven packages)
1. One in the package with the version (<- this one only exists for snapshot versions)
(1.) is not visible in the UI.
(2.) being in the package with the version, you can see it in the UI.
## Question
Should we show the (versionless package) `maven-metadata.xml` on the UI?
## User should not be able to delete the `maven-metadata.xml`
Note: if we show the file the UI, we should not allow the user to delete it.
This file is technically the *state* of the package registry for a given package. This is used during the requests :ping_pong: with `$ mvn` when `$ mvn` asks: `Hey third party registry, which versions of package foobar are available?`. This is when the ~backend returns the most recent `maven-metadata.xml` file.
Because of the above, if we allow the delete operation on that file, this can have a pretty deep impact on the registry behavior for that package (eg. `$ mvn` could complain that a specific version doesn't exist on the registry).
## Options
1. we show the file the UI but do not allow delete operation - consistent with competitors, maybe more clear what is being stored as part of the package.
1. we continue to not show it. - I have not seen any customer complaints about not showing it.Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/354665Overwrite maven-metadata.xml file2023-04-10T17:32:54ZKatie MacoyOverwrite maven-metadata.xml file## Problem
When the user allows Maven duplicates, each publish to the registry adds
* the `maven-metadata.xml` to the versionless package (package with _no_ version)
* the .pom and .jar files to the package with a version.
_Versionles...## Problem
When the user allows Maven duplicates, each publish to the registry adds
* the `maven-metadata.xml` to the versionless package (package with _no_ version)
* the .pom and .jar files to the package with a version.
_Versionless_ packages are not returned by APIs so the .pom and .jar are displayed in the UI but the `maven-metadata.xml` is not.
When consuming the package, only the latest `maven-metadata.xml` is served. The user cannot access the older `maven-metadata.xml` files.
With time, a versionless maven package can accumulate _many_ `maven-metadata.xml` files. This consumes object storage space.
## Proposal
Overwriting the `maven-metadata.xml` with each publish of a duplicate Maven package to the registry would reduce storage space, since the older versions of the file are not accessible anyway. This is consistent with what Artifactory does.
## Other considerations
With cleanup policies for packages, the _versionless_ package will be taken into account.
Example, let's say that users want to keep only 10 of the duplicated package files. The _versionless_ package will be cleaned up as any other package.
Still, we know for sure that only the most recent file is used in the operations with `$ mvn` or `$ gradle`.Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/299555Discussion: Rework the Maven UI to better describe the package structure2023-11-07T18:32:59ZSteve Abramssabrams@gitlab.comDiscussion: Rework the Maven UI to better describe the package structure## Problem
As is described in https://gitlab.com/gitlab-org/gitlab/-/issues/11424#note_485471051, Maven packages consists of a set of files, but there exists one or more `maven-metadata.xml` files that belongs to all versions of a given...## Problem
As is described in https://gitlab.com/gitlab-org/gitlab/-/issues/11424#note_485471051, Maven packages consists of a set of files, but there exists one or more `maven-metadata.xml` files that belongs to all versions of a given package. So we have something that wants to look like:
```mermaid
classDiagram
MavenPackage <|-- Version1
MavenPackage <|-- Version2
MavenPackage <|-- Version3
MavenPackage : maven-metadata.xml
Version1 : .jar file
Version1: .pom file
Version2 : .jar file
Version2: .pom file
Version3 : .jar file
Version3: .pom file
```
But in GitLab, each Version is a single `Packages::Package` record, and the `maven-metadata.xml` file is associated with it's own `Packages::Package` file that has no version:
```mermaid
classDiagram
MavenPackageVersion1 <|-- PackageFiles1
MavenPackageVersion2 <|-- PackageFiles2
MavenPackageVersion3 <|-- PackageFiles3
MavenPackageVersionless <|-- PackageFilesVersionless
MavenPackageVersion1 : name foo
MavenPackageVersion1 : version 1.0.0
MavenPackageVersion2 : name foo
MavenPackageVersion2 : version 2.0.0
MavenPackageVersion3 : name foo
MavenPackageVersion3 : version 3.0.0
MavenPackageVersionless : name foo
MavenPackageVersionless : version NULL
PackageFiles1 : .jar file
PackageFiles1: .pom file
PackageFiles2 : .jar file
PackageFiles2: .pom file
PackageFiles3 : .jar file
PackageFiles3: .pom file
PackageFilesVersionless: maven-metadata.xml
```
In the GitLab UI, we simply don't show the versionless package, so the user only sees the three actual packages available for use.
## Proposal
Group packages with the same name together, perhaps in an expandable/accordion type of display. When expanded, users will see each version of the package, but also see the metadata files that are shared among all three of them. There are often many of these metadata files, so we would only want to show the most recent one because all previous ones can be considered stale.
## Considerations
There has also been discussion around reworking the backend structure of these objects to better fit with the first diagram. If we were to move forward with that, we should wait to adjust the UI.Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/8280Make maven URLs browseable when popped into a browser2022-01-13T00:45:33ZThiago PresaMake maven URLs browseable when popped into a browserCurrently if users wished to browse for the packages they would have to convert from `api/v4/projects/${env.CI_PROJECT_ID}/packages/maven` to `/<group>/<project-name>/-/packages by knowing how to translate project id into group and proje...Currently if users wished to browse for the packages they would have to convert from `api/v4/projects/${env.CI_PROJECT_ID}/packages/maven` to `/<group>/<project-name>/-/packages by knowing how to translate project id into group and project name. Could we perhaps make that relationship clearer, or even use the same URL for both cases?
ZD https://gitlab.zendesk.com/agent/tickets/107334Awaiting further demandhttps://gitlab.com/gitlab-org/gitlab/-/issues/427151Gradle: can't use the URL-encoded project path2023-10-06T05:54:10ZDavid FernandezGradle: can't use the URL-encoded project path## :fire: Problem
When using `$ gradle`, we can't use the URL-encoded project path as the id of the project.
`$ gradle` will interpret those `%2F` into `/` and thus, it will not properly work.
`build.gradle` example:
```groovy
plugin...## :fire: Problem
When using `$ gradle`, we can't use the URL-encoded project path as the id of the project.
`$ gradle` will interpret those `%2F` into `/` and thus, it will not properly work.
`build.gradle` example:
```groovy
plugins {
id 'java'
id 'application'
}
repositories {
maven {
url "https://gitlab.com/api/v4/projects/issue-reproduce%2Fpackages%2Fmaven%2F332028-test/packages/maven"
name "GitLab"
credentials(HttpHeaderCredentials) {
name = 'Private-Token'
value = '<token>'
}
authentication {
header(HttpHeaderAuthentication)
}
}
}
dependencies {
implementation 'foo.bar.app:something-works:3.1'
}
```
## :crystal_ball: The available workaround
The available workaround here is to use the project `id` directly instead of its full path.
## :fire_engine: Solutions
There are different ways to solve this:
* Properly document that URL-encoded project full paths are not supported with `$ gradle`.
* Check if we could do something about it in the Grape API router.Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/409676Document basic auth for mvn and gradle2023-05-04T11:07:41ZDavid FernandezDocument basic auth for mvn and gradle## :fire: Problem
In https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118692#note_1370366017, we added basic auth support to the download endpoints of the ~"Maven Repository".
This was to support downloads for the `$ sbt` client.
...## :fire: Problem
In https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118692#note_1370366017, we added basic auth support to the download endpoints of the ~"Maven Repository".
This was to support downloads for the `$ sbt` client.
In that MR, we updated the ~documentation for `$ sbt` but not `$ mvn` or `$ gradle`.
## :fire_engine: Solution
Update the ~"Maven Repository" ~documentation so that it is clearly noted that basic auth is support for pulling packages (not pushing).Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/396661Improve Snapshot dynamic parts extraction2023-03-15T07:49:08ZDavid FernandezImprove Snapshot dynamic parts extraction## :fire: Problem
For the fix of https://gitlab.com/gitlab-org/gitlab/-/issues/325749, we had to build a way to extract the dynamic part that ~"Maven Repository" clients put on file names.
Example: `my-package-11.0-20230315.073119-2.po...## :fire: Problem
For the fix of https://gitlab.com/gitlab-org/gitlab/-/issues/325749, we had to build a way to extract the dynamic part that ~"Maven Repository" clients put on file names.
Example: `my-package-11.0-20230315.073119-2.pom`, `20230315.073119` is the dynamic part. It changes each time that `$ mvn deploy` runs.
To detect that part, we used a [regex](https://gitlab.com/gitlab-org/gitlab/-/blob/177c7a54a54ae643b289e2590af99724331769c2/lib/gitlab/regex.rb#L12). That one will match the very last match.
The problem is that users can use _classifiers_ to generate additional artifacts. They can use any string for the _classifier_ part.
Example: `my-package-11.0-20230315.073119-2-fancy-classifier.jar`, `20230315.073119` is the dynamic part and `fancy-classifier` is the classifier.
The regex will not work if the classifier follows the same structure as the dynamic part. For example, `my-package-11.0-20230315.073119-2-55555555.444444-3.jar`, here the backend will wrongly extract `55555555.444444-3` as the classifier.
## :fire_engine: Solution
Improve the classifier extraction:
* Remove the package name and version from the file name and match the first group with the regex.Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/396660Document Maven version warning2023-03-15T07:49:08ZDavid FernandezDocument Maven version warning## :fire: Problem
From https://gitlab.com/gitlab-org/gitlab/-/issues/325749#note_1308829837: Maven doesn't allow specific characters in the version string of maven packages.
However, the build itself will not fail. This means that arti...## :fire: Problem
From https://gitlab.com/gitlab-org/gitlab/-/issues/325749#note_1308829837: Maven doesn't allow specific characters in the version string of maven packages.
However, the build itself will not fail. This means that artifacts with malformed versions can be pushed.
The GitLab ~"Maven Repository" [relies](https://gitlab.com/gitlab-org/gitlab/-/blob/92312944bb8a827b5d00509a5d07d0fe8ee4955d/app/services/packages/maven/find_or_create_package_service.rb#L39) on a structure format to extract the package name and the package version. Malformed versions will end up in a buggy behavior for that extraction.
## :fire_engine: Solution
* Document that the GitLab ~"Maven Repository" doesn't support such malformed versionsBacklog