SDMX Faceted Search to Support (or avoid) API Keys
The ABS' externally-available NSI services have an API Gateway sitting in front of them that limits the rate at which users can call the services. At the moment it is possible to call the service with or without an API Key attached to the request. Requests with an API Key are throttled according to that key's usage plan. Requests without an API Key are considered to be all on the same plan, with a relatively low acceptable rate of calls.
The SDMX Faceted Search calling the services in order to index them without an API Key will quickly reach our rate limit and fail. The ability to specify an API Key for the service to use would therefore be required. Or the ability for Faceted Search to avoid calling the endpoints that require an API key.
Options:
- Provide the ability to specify an API Key for the SDMX Faceted Search Service via:
- 1a) An environment variable X-API-KEY_HEADER. This would be the simplest implementation, but would mean all datasources receive the same API Key. This is fine for the ABS at this stage.
- 1b) A series of environment variables X-API-KEY_HEADER_{DataSourceId} (or similar). This would mean each datasource could have a separate API Key, or no key. This would also work for the ABS, but provides a hardcoded link between the datasources.json file and environment variables.
-
Provide the ability to specify arbitrary headers for the SDMX Faceted Search Service via either a series of environment variables or a configuration file bundled with the SDMX Faceted Search Service (as distinct to being served by the Config service).
-
Allow the SDMX Faceted Search Service to have different URLs for datasources than DE in the browser. At the moment both applications retrieve the same URL from the datasources.json file (which causes some issues too with the docker-compose repository). This would allow the SDMX Faceted Search in the ABS to connect directly to the services, circumventing the API Gateway and throttling considerations.