### Problem When running API Security, it is a common configuration to add a `service` definition to the job with the target app, database, etc. Sometimes the port numbers conflict with those we use by default, such as port 5000. This issue has impacted two customers at the time of writing. https://gitlab.zendesk.com/agent/tickets/304578 ### Proposal - Default to special port 0, but allow override with new variable - Special Port: "0" (means find an available port and use it) - https://andrewlock.net/how-to-automatically-choose-a-free-port-in-asp-net-core/ - Engine writes out URL to file on disk - worker entry Read the `_API` variable from a file. **Example Place to Put Write File** ```csharp public class Startup { public void Configure(IApplicationBuilder app, ILogger<Startup> log) { // IApplicationBuilder exposes an IFeatureCollection property, ServerFeatures var addressFeature = app.ServerFeatures.Get<IServerAddressesFeature>(); foreach(var address in addressFeature.Addresses) { _log.LogInformation("Listing on address: " + address); } } // ... other configuration } ``` ~Allow the user to change the port using a new variable.~ ~1. [ ] Pick an unused port for default (5001)(https://docs.gitlab.com/ee/administration/package_information/defaults.html#ports)~ ~2. [ ] Add a new variable to allow the port to be configured by the user at runtime.~ ~3. [ ] ...~ ### Implementation Proposal 1. [x] Set up API port file to let worker-entry know current port in USE 1. [x] Add new variable `_API_PORT_FILE` (e.g. `FUZZAPI_API_PORT_FILE` , `DAST_API_API_PORT_FILE`) as non configurable variable. Default to `gl-api-security-port.log` 1. [x] `web\SDK\entrypoints\analyzer-dast-api` 1. [x] `web\SDK\entrypoints\analyzer-fuzz-api` 1. [x] Update `scanner` to write the file based on `_API_PORT_FILE` file path. See [how to automatically choose a free port in asp net core](https://andrewlock.net/how-to-automatically-choose-a-free-port-in-asp-net-core/). If possible, limit file content only to the port number. 1. [x] Update `worker-entry` to consume file before calling `wait_for_api_fuzzer` (which internally calls to `glapifuzzing.set_api(self.api)`) 1. [x] Load new configuration variable `_API_PORT_FILE` (new worker-entry property, print out information in logs) 1. [x] After printing configuration handle the file pointed by `_API_PORT_FILE` 1. [x] Wait for the file to be created. Exponential waits up to N retries. Log process status, until wait finished (found file o give up) 1. [x] On found file: 1. [x] read port from a file, then update `api` property of the `worker-entry` 1. [x] On give up: 1. [x] report failure, suggest actionable items 1. [x] Update `worker-entry` tests 1. [x] Set `_PORT_FILE` to `/output/` 1. [x] `build/jobs/tests_int_openapi` 1. [x] `build/jobs/tests_int_postman` 1. [x] `build/jobs/tests_int_worker-entry_py36` 1. [x] `build/jobs/tests_int_worker-entry_py39` 1. ~Expose `_PORT_FILE` to test the environment ~ NOTE: Better to add when needed 1. ~[ ] `web/Test/docker-compose.gitlab.dast.yml`~ 1. ~[ ] `web/Test/docker-compose.gitlab.har.yml`~ 1. ~[ ] `web/Test/docker-compose.gitlab.openapi.yml`~ 1. ~[ ] `web/Test/docker-compose.gitlab.postman.yml`~ 1. ~[ ] `web/Test/docker-compose.python.py36.yml`~ 1. ~[ ] `web/Test/docker-compose.python.py39.yml`~ 1. ~[ ] `web/Test/docker-compose.worker-entry_py36.yml`~ 1. ~[ ] `web/Test/docker-compose.worker-entry_py39.yml`~ 1. ~[ ] `web/SDK/worker-entry/tox.ini`~ 1. [x] Update `web/SDK/entrypoints/tests/01-fuzz-general-test.sh` / `web/SDK/entrypoints/tests/02-dast-general-test.sh` to test new variable `_API_PORT` 1. [x] Update `WorkerEntryTest` class to allow specifying `_API_PORT_FILE` parameter. 1. [x] Add test for new backoff functionality 1. [x] Add a new variable to specify `_API_PORT` as a non-configurable variable. Default to `0` 1. Set `ASPNETCORE_URLS` in the entry point 1. [x] Move declaration from `web/PeachWeb/Dockerfile`, `web/PeachWeb/Dockerfile-fips` to entry points scripts 1. [x] Update entry points (`web\SDK\entrypoints\analyzer-dast-api`, `web\SDK\entrypoints\analyzer-fuzz-api`) 1. [x] Add new variable: `_API_PORT` (e.g. `FUZZAPI_API_PORT` , `DAST_API_API_PORT`) default to `0` 1. [x] Set `_API` based on `_API_PORT` 1. [x] Set `ASPNETCORE_URLS` based on `_API_PORT` 1. Worker-Entry 1. [x] Load new configuration variable `_API_PORT` (new worker-entry property, print out information in logs) 1. [x] Fail to start if `_API_PORT` lower or equals 1024 1. Update `worker-entry` tests 1. ~Expose `_PORT_FILE` to test the environment~ NOTE: Better to add when needed 1. ~[ ] `web/Test/docker-compose.gitlab.dast.yml`~ 1. ~[ ]`web/Test/docker-compose.gitlab.har.yml`~ 1. ~[ ] `web/Test/docker-compose.gitlab.openapi.yml`~ 1. ~[ ] `web/Test/docker-compose.gitlab.postman.yml`~ 1. ~[ ] `web/Test/docker-compose.python.py36.yml`~ 1. ~[ ] `web/Test/docker-compose.python.py39.yml`~ 1. ~[ ] `web/Test/docker-compose.worker-entry_py36.yml`~ 1. ~[ ] `web/Test/docker-compose.worker-entry_py39.yml`~ 1. ~[ ] `web/SDK/worker-entry/tox.ini`~ 1. [x] Update `web/SDK/entrypoints/tests/01-fuzz-general-test.sh` / `web/SDK/entrypoints/tests/02-dast-general-test.sh` to test new variable `_API_PORT` 1. [x] ~Update `WorkerEntryTest` class to allow specifying `_API_PORT` parameter~ Instead allowing specifying `_API_PORT_FILE` content to point o a different port. 1. [x] Add test to check `_API_PORT` fails when set lower or equals to 1024 1. [x] Document new variable 1. [x] Add `_API_PORT` to the list of configuration variables 1. [x] Add a troubleshoot entry section about when to use this new variable.