search endpoints: certain flow values not available via API

Context & Problem Description

So far, we've used the provided raw data version. Now, we want to migrate to the new API with the /search/batch endpoint.

Via the API, we do not receive data for all flow values that we previously had access to via the raw data extract. Below is the list of satellite IDs (equivalent to flow values in the API) we are using and have access to via the raw data extract. Of these, some flows are not available and not documented via the API /flow endpoint.

Satellite IDs used via raw data extract

[
  "land_use:annual_crops",
  "land_use:permanent_crops",
  "land_use:pastures",
  "land_use:urban",
  "land_use:extensive_forestry",
  "land_use:intensive_forestry",
  "emissions_(edgar):ghg_total_edgar_consistent",
  "blue_water_consumption:agriculture_blue_water_consumption",
  "blue_water_consumption:non-agriculture_blue_water_consumption",
  "water_stress:agriculture_water_stress",
  "water_stress:non-agriculture_water_stress",
  "emissions_(edgar):nox_total_edgar_consistent",
  "emissions_(edgar):nh3_total_edgar_consistent",
  "emissions_(edgar):so2_total_edgar_consistent",    "emissions_(edgar):co2_excl_short_cycle_org_c_total_edgar_consistent",
  "emissions_(edgar):hcfc_141b_total_edgar_consistent",
  "emissions_(edgar):hcfc_142b_total_edgar_consistent",
  "emissions_(edgar):hfc_23_total_edgar_consistent",
  "emissions_(edgar):hfc_32_total_edgar_consistent",
  "emissions_(edgar):hfc_41_total_edgar_consistent",
  "emissions_(edgar):hfc_43_10_mee_total_edgar_consistent",
  "emissions_(edgar):hfc_125_total_edgar_consistent",
  "emissions_(edgar):hfc_134a_total_edgar_consistent",
  "emissions_(edgar):hfc_143a_total_edgar_consistent",
  "emissions_(edgar):hfc_152a_total_edgar_consistent",
  "emissions_(edgar):hfc_227ea_total_edgar_consistent",
  "emissions_(edgar):hfc_236fa_total_edgar_consistent",
  "emissions_(edgar):hfc_245fa_total_edgar_consistent",
  "emissions_(edgar):hfc_365mfc_total_edgar_consistent",
  "emissions_(edgar):nf3_total_edgar_consistent",
  "emissions_(edgar):sf6_total_edgar_consistent",
  "emissions_(edgar):ch4_total_edgar_consistent",
]

Satellites/flows not available via API or not documented

When using the /flow endpoint, the following satellites from above are missing (translated to their respective flow names, hopefully correctly):

Undocumented flows/satellites

climate_change:hcfc_141b
climate_change:hcfc_142b
climate_change:hfc_41
climate_change:hfc_43_10_mee
climate_change:hfc_23
climate_change:hfc_236fa
climate_change:nf3
climate_change:sf6

Based on testing of the /search/endpoint, it turns out that only the following flows are missing via the API /search/batch endpoint:

Missing flows/satellites

climate_change:hfc_23
climate_change:hfc_236fa
climate_change:nf3
climate_change:sf6

Indicators not available via API

In addition, the urban indicator for land_use is missing, compared to the raw data extract.

Expected Behavior & Documentation

  1. Provide access to the missing Missing flows/satellites and missing indicators via the API
  2. Ensure that the /flow endpoint returns all values available via the API, including the Undocumented flows/satellites
  3. Ensure that the api_field_values.json of the FootprintLab API Access page is consistent with what the API returns
  4. Document the impact category/indicator/flow combinations available via the API — either as a dedicated endpoint or in the documentation — at least for the satellites above. Currently, it is somewhat guesswork on our end as to how to combine them correctly. These are present as query parameters in the request, so the client should be aware of the available combinations to enable proper filtering.

Impact

This issue is blocking us from migrating to the new API.

Edited by Markus Jura