Coverage-guided fuzzing results download - Follow up AJAX API/File Path cleanup
As part of the MVC for fuzzing artifacts download we have the following Merge Request: !36676 (comment 378690409)
In order to generate the URL for any given pipeline job we construct it in the frontend with
artifactDownloadUrl(job) {
return `/api/v4/projects/${this.projectId}/jobs/artifacts/${job.ref}/download?job=${job.name}`;
Problem
Having client side path generation is not ideal. This makes paths fragile because they are not bound to a single source of truth. (i.e. rails path helper).
We can't pass in a static route like we usually do because they are dynamic and different based on ref
and name
.
We need the routes included in the API response or generated client side. Client side generation has been difficult because libraries like the ruby gem js-routes
don't generate routes for applications mounted as Rack middleware/Rails engine. The Grape API gem is mounted as rack middleware. I was able to generate dynamic route helper for the main rails app, but none for the Grape API endpoints.
See: #20183 (comment 215082892)
How we ended up with this less than ideal solution
The goal was to provide a dropdown with multiple downloads available for download.
We have the following API endpoints
https://docs.gitlab.com/ee/api/jobs.html#list-pipeline-jobs
https://docs.gitlab.com/ee/api/jobs.html#download-a-single-artifact-file-by-job-id
Neither of these endpoints provide the artifacts file path, or give us enough information to correctly hit the https://docs.gitlab.com/ee/api/jobs.html#download-a-single-artifact-file-by-job-id endpoint.
Additional Context
The pipeline jobs endpoint does not provide this artifact download path. See: https://docs.gitlab.com/ee/api/jobs.html#list-pipeline-jobs
The following endpoint requires us to have the artifact_path
which we don't have from the pipeline jobs endpoint either.
https://docs.gitlab.com/ee/api/jobs.html#download-a-single-artifact-file-by-job-id
Attribute | Type | Required | Description |
---|---|---|---|
id | integer/string | yes | ID or URL-encoded path of the project owned by the authenticated user. |
job_id | integer | yes | The unique job identifier. |
artifact_path | string | yes | Path to a file inside the artifacts archive. |
Things to Consider
We should factor in the future iterations of the Fuzzing UX, to better plan out future API endpoints that relate to fuzzing artifacts. It may change our approach on how we clean up this ~"technical debt"
Possible Solutions
Client side helper
Go with the approach suggested by @pgascouvaillancourt in !36676 (comment 378731617)
@winh and @jamedjo had mentioned a similar approach in #20183 (comment 215082874) and #20183 (comment 215082877)
We should startout defining these slugs
on an as needed basis for version 1, this differs from previous attempts where the main blocker was figuring out how to dynamically generate slugs for all routes. (Rails app and Grape)
There might be a more naive approach though:
- Expose the path as we usually would from the backend, but instead of providing actual parameters for the job's ref and name, pass some placeholders. It might look like this:
expose_path(api_v4_projects_pipelines_jobs_fuzing_artifacts_path(id: project.id, job_ref: ':job_ref'))
The output might look like this:
/api/v4/projects/123/jobs/artifacts/:job_ref/download
- In the frontend, create some helper that would accept a path, and an object for replacing the placeholders with actual parameters, e.g.:
const getApiPath = (path, params) => {
let outputPath = path;
Object.entries(params).forEach(([key, value]) => {
outputPath.replace(key, value);
});
return outputPath;
};
- Use this to generate the actual path in the component:
artifactDownloadUrl(job) {
return getApiPath(this.fuzzingArtifactsPath, { ':job_ref': job.ref }) + '?job=${job.name}';
}
Client Side helper variation 2
const getApiPath = (path, {keys, params}) => {
let outputPath = path;
Object.entries(keys).forEach(([key, value]) => {
outputPath.replace(key, value);
});
return params ? outputPath : outputPath + '?' + Object.keys(params).map(key => key + '=' + params[key]).join('&');
};
artifactDownloadUrl(job) {
return getApiPath(this.fuzzingArtifactsPath, {
keys: { ':job_ref': job.ref },
params: {job: job.name}
}
)}
Add data to existing or new endpoint.
TBD