Backend: Gitlab ignores new manual job variable values when you run a job the second time

Current Status

This was reproduced after Frontend: Surface the used custom variables for... (#361237 - closed) was fixed.

Summary

After the first launch of a manual job with the input of variables - they are read. But if you abort the job and enter the variables again - the original overridden variables are used, not the updated ones from the second run.

Steps to reproduce

1. Update the .gitlab-ci.yml file to include a manual job
2. Create MR
3. Once the pipeline finished running, I go to view the jobs and I see that one is manual and didn't run. One that was not manual did run.
4. By clicking the "manual" status label, I am able to set manual job variables that I already set in the .gitlab-ci.yml so I am basically overwriting the variable.
5. When click play, it runs the job. I get an echo with my variable that I updated above.
6. If I clear the job my pressing the bin icon, it shows me variable form again.
7. When I set same variable with different value and run the pipeline, I see in the stack trace that my old value I set in point 4 is still there.

What is the current bug behavior?

The variables first used for overriding are used on subsequent runs, even if different values are entered.

What is the expected behavior?

The values entered when running the job are the ones used.

Proposed Changes

Update the /jobs//play request to update the variables appropriately

Request URL: http://127.0.0.1:3000/root/<project name>/-/jobs/<id>/play

Payload: {"job_variables_attributes":[{"key":"TEST_VAR_JOB","secret_value":"new"}]}

Technical notes

The steps I followed:

1. Update the .gitlab-ci.yml file to include a manual job
2. Create MR
3. Once the pipeline finished running, I go to view the jobs and I see that one is manual and didn't run. One that was not manual did run.
4. By clicking the "manual" status label, I am able to set manual job variables that I already set in the .gitlab-ci.yml so I am basically overwriting the variable.
5. When click play, it runs the job. I get an echo with my variable that I updated above.
6. If I clear the job my pressing the bin icon, it shows me variable form again.
7. When I set same variable with different value and run the pipeline, I see in the stack trace that my old value I set in point 4 is still there.

Bottom line: looks like the manual job variables unfortunately don't updated.
Looks like it might be this POST request that is not reassigning the variable correctly:
Request URL: http://127.0.0.1:3000/root/<project name>/-/jobs/<id>/play
Payload: {"job_variables_attributes":[{"key":"TEST_VAR_JOB","secret_value":"new"}]}

Additional note: Seems like I need to bin the previous job to be able to set new variables again on the job, not sure if that is intentional.