Optimise request for DE home page
Context
Data Explorer (DE) landing page requests SFS to get facet domain values for a subset of all facets.
SFS does not offer a specific request to cover needs for this page, so DE requests https://sfs-demo.siscc.org/api/search?tenant=[tenantId] with a body like:
{"lang":"en","facets":{"datasourceId":["ds:staging:SIS-CC-stable","ds:staging:SIS-CC-reset","ds:UNSD-test"]}}
This request is very similar to SELECT *
, with the same performance problems to come as facets cardinality increases!
Solution
Enhance existing search request to allow:
- to limit searched and returned fields
- to limit searched and returned facets
Solr facets and fields are all defined as SFS fields, so only one new property is required to be added to the search api: ‘fieldList’
DE gets landing page required facets from its settings.json (served by DE backend) and will fill fieldList in search api.
Todo
-
update POST /api/search to add fieldList property (will throw a 400 error if a field name is unknown) -
update solr module to update SOLR query with new values for ‘field’ and ‘facet’ fields from fieldList values -
write tests