Table of content:

• Introduction
  • Issues & error reporting
• Installation of nFIESTA
  • Automated tests checking installation
• Data used in Minimal Working Example
  • Sampling design
    • Strata and NFI plots (grids)
    • Panels
  • Local densities
    • Target local densities
    • Auxiliary local densities
  • Estimation cells and parametrization areas
  • Automated tests importing data
• Configuration of estimates
  • Estimation cell and target variable
  • Single-phase configuration
  • Regression estimates configuration
    • Model configuration
  • Automated tests creating and showing configurations
  • Automated tests selecting data for calculation
• Calculation of estimates
  • Single-phase estimates calculation
  • Computation of G_beta
  • Regression estimates calculation
  • Automated tests performing calculations of estimates

Introduction

This part of documentation describes the Minimal Working Example showing how forest inventory estimates can be produced by the nFIESTA system. Estimates of Forest area according to FAO's definition based on NFI data from Czech NFI2 (2011 - 2014 / 2015) are demonstrated in this example. Single phase and regression estimate of total is calculated together with its variability. Internally derived map of forest (OLIL version 2013) is used as a auxiliary data.

It is highly advisable to go through document Functional overview of nFIESTA and following Data storage, Estimates configuration and Estimates calculation before going through Minimal Working Example to get conceptual overview of the whole system. Alternatively, that part can be found also in Appendix B of technical report.

Following three chapters shows procedure how to get single estimate (total of forest area) using nFIESTA system from the very beginning:

• Installation of nFIESTA,
• Data used in Minimal Working Example,
• Configuration of estimates,
• Calculation of estimates.

The Minimal Working Example is tightly connected to automated tests (in PostgreSQL terminology called regression tests) designed to alert when nFIESTA system starts to behave differently unintentionally (regression testing in computing terminology). Because in Statistics word regression has different meaning (regression estimator implemented in nFIESTA) we will call nFIESTA regression test automated tests later in documentation. Automated tests can be seen as a kind of 'live documentation'. They are triggered each time the nFIESTA source code is changed. Results calculated in automated tests (using nFIESTA) were verified by independent implementation of estimators in R. At the end of each chapter you will find links to corresponding automated tests.

It is recommended to check the contrib_regression database (created by automated tests and containing automated test data) using e.g. QGIS. In particular to connect to the DB, look at the database objects (tables, views, functions) and to visualise test data (geometries).

Issues & error reporting

In case of troubles, please see a list of issues with label MinimalWorkingExample. If your are facing new issue, please do not hesitate to create new one.

Installation of nFIESTA

For installation instruction see Installation page.

Automated tests checking installation

• install_test

Data used in Minimal Working Example

For conceptual details see page Data storage especially ERD diagram.

Sampling design

Strata and NFI plots (grids)

Data used in automated tests covers eastern part of Czech Republic. Area can be split into 5 strata (so called 'Natural Forest Regions Districts' – NFRD – in Czech OPLO) which are aggregated into 2 strata sets. For this experiment, you can imagine strata set as independent country with its own design of sampling grid:

• Czechia-Moravia border (NFRD 7 and 12) with sampling grid from Czech NFI1 (clusters) with degraded coordinates of plot positions
• Moravian and Silesian region (NFRD 13, 11 and 14) with sampling grid from Czech NFI2 with degraded coordinates of plot positions.

Moreover strata inside strata sets can differ in density of the grid: NFRD 14 has lower density compared to 11 and 13. See overview map for graphical representation.

You can see strata with their attributes in following table:

select
	t_stratum.id, stratum, area_m2, buffered_area_m2,
	t_strata_set.country, t_strata_set.strata_set, t_strata_set.comment
from
nfiesta_test.t_stratum
inner join nfiesta_test.t_strata_set on (t_stratum.strata_set = t_strata_set.id)
order by strata_set, stratum;

Panels

List of panels and its connection to reference year sets can be seen in following table.

select
	t_panel.stratum as t_panel__stratum, t_panel.panel as t_panel__panel,
	t_reference_year_set.label as t_reference_year_set__label
from nfiesta_test.t_panel
inner join nfiesta_test.cm_refyearset2panel_mapping on (t_panel.id = cm_refyearset2panel_mapping.panel)
inner join nfiesta_test.t_reference_year_set on (cm_refyearset2panel_mapping.reference_year_set = t_reference_year_set.id)
order by stratum, t_panel.panel
;

Local densities

Target local densities

with w_distinct_ldsities as (
	select distinct target_variable, sub_population_category, area_domain_category, reference_year_set
	from nfiesta_test.t_target_data
)
select
	--w_distinct_ldsities.target_variable as w_distinct_ldsities__target_variable,
	c_target_variable.description as c_target_variable__description,
	--c_variable_type.description as c_variable_type__description,
	--c_state_or_change.description as c_state_or_change__description,
	--sub_population_category,
	coalesce(c_area_domain_category.description, 'Without distinction') as c_area_domain_category__description,
	t_reference_year_set.label
from
w_distinct_ldsities
inner join nfiesta_test.c_target_variable on (w_distinct_ldsities.target_variable = c_target_variable.id)
inner join nfiesta_test.c_variable_type on (c_target_variable.variable_type = c_variable_type.id)
inner join nfiesta_test.c_state_or_change on (c_target_variable.state_or_change = c_state_or_change.id)
left join nfiesta_test.c_area_domain_category on (w_distinct_ldsities.area_domain_category = c_area_domain_category.id)
left join nfiesta_test.t_reference_year_set on (w_distinct_ldsities.reference_year_set = t_reference_year_set.id)
order by c_target_variable.description, c_area_domain_category.description, t_reference_year_set.label;

Particular local densities can be obtained by following query:

select
	plot,
	c_target_variable.label as target_variable,
	coalesce(c_area_domain_category.label, 'without distiction') as area_domain_category,
	t_reference_year_set.label as reference_year_set,
	value as local_density
from nfiesta_test.t_target_data
inner join nfiesta_test.c_target_variable ON c_target_variable.id = t_target_data.target_variable
left join nfiesta_test.c_area_domain_category ON c_area_domain_category.id = t_target_data.area_domain_category
left join nfiesta_test.t_reference_year_set ON t_reference_year_set.id = t_target_data.reference_year_set
where plot = 1
order by plot, target_variable, area_domain_category, reference_year_set
;

Auxiliary local densities

with w_distinct_ldsities as (
        select distinct auxiliary_variable_category from nfiesta_test.t_auxiliary_data
)
select
	c_auxiliary_variable_category.description as c_auxiliary_variable_category__description,
	c_auxiliary_variable.description as c_auxiliary_variable__description
from w_distinct_ldsities
inner join nfiesta_test.c_auxiliary_variable_category on (w_distinct_ldsities.auxiliary_variable_category = c_auxiliary_variable_category.id)
inner join nfiesta_test.c_auxiliary_variable on (c_auxiliary_variable_category.auxiliary_variable = c_auxiliary_variable.id);

Particular local densities can be obtained by following query:

select
	plot,
	c_auxiliary_variable_category.label as auxiliary_variable_category,
	value as local_density
from nfiesta_test.t_auxiliary_data
inner join nfiesta_test.c_auxiliary_variable_category ON c_auxiliary_variable_category.id = t_auxiliary_data.auxiliary_variable_category
where plot = 1
order by plot, auxiliary_variable_category
;

Estimation cells and parametrization areas

Each stratum is also configured to act as estimation cell and parametrization area. This way you can compute estimates for whole stratum or strata set, parametrizing model on such territory. Beside of this, geographically independent system of rectangular cells is defined: 25x25km and 50x50km EEA reference / INSPIRE grid.

Automated tests importing data

• fst_data
• fst_conf

Configuration of estimates

For conceptual details see page Estimates configuration.

Estimation cell and target variable

We will limit this documentation to single estimation cell: 25kmE4800N2900. The cell covers three different strata, varying in cluster utilization and density of the grid. See detailed map for graphical representation.

We will need ID of estimation cell for configuration:

select
	f_a_cell.estimation_cell,
	c_estimation_cell.estimation_cell_collection, c_estimation_cell.label
from nfiesta_test.f_a_cell
inner join nfiesta_test.c_estimation_cell on (f_a_cell.estimation_cell = c_estimation_cell.id)
where c_estimation_cell.label = '25kmE4800N2900';

and we will need to select target variable:

select
	t_variable.id,
	c_target_variable.label as c_target_variable__label,
	c_sub_population_category.label as c_sub_population_category__label,
	c_area_domain_category.label as c_area_domain_category__label
from nfiesta_test.t_variable
inner join nfiesta_test.c_target_variable on (t_variable.target_variable = c_target_variable.id)
left join nfiesta_test.c_sub_population_category on c_sub_population_category.id = t_variable.sub_population_category
left join nfiesta_test.c_area_domain_category ON c_area_domain_category.id = t_variable.area_domain_category;

Single-phase configuration

Final step is to call function: fn_1p_est_configuration performing configuration passing id of estimation cell, begin and end dates to be used for estimate, note and id of target variable:

select nfiesta_test.fn_1p_est_configuration(81::integer, '2011-01-01'::date, '2015-12-31'::date, 'note: this is our first estimate'::varchar, 3::integer);

after this step we can see created configuration in view v_conf_overview:

select *
from nfiesta_test.v_conf_overview
where estimation_cell = 81
	and target_variable = 1
	and area_domain_category is null
	and estimate_type_str = '1p_total'
order by estimate_conf;

It is possible to select and visualize data prepared for calculation by calling function: fn_1p_data

select * from nfiesta_test.fn_1p_data(143);

Regression estimates configuration

We can extend further the example to regression estimate. Here we will need three more parameters:

• auxiliary configuration
  • parametrization area (we will use 50kmE480N290)
  • model (set of auxiliary variables) & totals of auxiliary variable for each estimation cell
• whether to use force_synthetic switch (default False)

Model configuration

Model in Minimal Working Example has following structure:

forest = forest_{OLIL} * p_1 + nonforest_{OLIL} * p_2

where forest is prediction, forest_{OLIL} is auxiliary variable indicating plot is inside OLIL, nonforest_{OLIL} is auxiliary variable indicating plot is outside OLIL, p_1 is mean of ground truth inside OLIL and p_2 is mean of ground truth outside OLIL.

We can check, what combinations of parameters are already available for parametrization area:

select
	t_aux_conf.id as t_aux_conf__id, t_aux_conf.param_area,
	f_a_param_area.param_area_code as f_a_param_area__param_area_code,
	t_model.id as t_model__id,
	c_auxiliary_variable_category.description as c_auxiliary_variable_category__description,
	t_aux_conf.sigma as t_aux_conf__sigma
from nfiesta_test.t_aux_conf
inner join nfiesta_test.f_a_param_area ON f_a_param_area.gid = t_aux_conf.param_area
inner join nfiesta_test.t_model on t_model.id = t_aux_conf.model
inner join nfiesta_test.t_model_variables ON t_model_variables.model = t_model.id
inner join nfiesta_test.t_variable ON t_variable.id = t_model_variables.variable
inner join nfiesta_test.c_auxiliary_variable_category ON c_auxiliary_variable_category.id = t_variable.auxiliary_variable_category
where f_a_param_area.param_area_code = '50kmE480N290';

In case we would like to configure different estimate, it is necessary to insert relevant values in tables (see ETL page for examples):

• c_auxiliary_variable
• c_auxiliary_variable_category
• t_aux_total
• t_variable
• t_model
• t_model_variables (set of variables used by model)
• f_a_param_area
• t_aux_conf (id to be passed to function bellow)
• t_panel2aux_conf

and run function fn_2p_est_configuration performing configuration passing id of estimation cell, begin and end dates to be used for estimate, note, id of target variable, id of auxiliary configuration, force synthetic switch:

select nfiesta_test.fn_2p_est_configuration(81::integer, '2011-01-01'::date, '2015-12-31'::date, ''::varchar, 3::integer, 31::integer, false::boolean);

after this we can see configuration in view v_conf_overview:

select *
from nfiesta_test.v_conf_overview
where estimation_cell = 81
	and target_variable = 1
	and area_domain_category is null
	and param_area = 31
	and force_synthetic = False
	and estimate_type_str = '2p_total'
order by estimate_conf;

It is possible to select and visualize data prepared for calculation by calling function: fn_2p_data:

select * from nfiesta_test.fn_2p_data(400);

Automated tests creating and showing configurations

• fst_1p_est_conf
• fst_1p_v_conf_overview
• fst_2p_est_conf
• fst_2p_v_conf_overview
• fst_remove_param_area

Automated tests selecting data for calculation

• fst_1p_data
• fst_2p_data

Calculation of estimates

For conceptual details see page Estimates calculation.

Single-phase estimates calculation

Function fn_1p_total_var contains implementation of single-phase estimator. Function argument is id from table t_total_estimate_conf.

select * from nfiesta_test.fn_1p_total_var(143);

Function fn_make_estimate_table is more user friendly wrapper printing results. Function argument is id from table t_estimate_conf.

select * from nfiesta_test.fn_make_estimate_table(143);

Function fn_make_estimate is more user friendly wrapper saving results into database. Function argument is id from table t_estimate_conf.

select * from nfiesta_test.fn_make_estimate(143);

Computation of G_beta

Coefficients \boldsymbol{\tilde{G}_{\beta_{t+}}}. can be computed using function fn_g_beta as is shown in test fst_2p_gbeta.

Regression estimates calculation

Function fn_2p_total_var contains implementation of regression estimator. Function argument is id from table t_total_estimate_conf.

select * from nfiesta_test.fn_2p_total_var(400);

Function fn_make_estimate_table is more user friendly wrapper printing results. Function argument is id from table t_estimate_conf.

select * from nfiesta_test.fn_make_estimate_table(628);

Function fn_make_estimate is more user friendly wrapper saving results into database. Function argument is id from table t_estimate_conf.

select * from nfiesta_test.fn_make_estimate(628);

It is also possible to connect computed estimates (here it is single phase total area of forest) with geometries and visualize resulting map.

select
	row_number() over() as gid,
	v_conf_overview.estimation_cell_label,
	t_result.*,
	f_a_cell.geom::geometry(MultiPolygon, 5221)
from nfiesta_test.t_result
inner join nfiesta_test.v_conf_overview on (t_result.estimate_conf = v_conf_overview.estimate_conf)
inner join nfiesta_test.f_a_cell on (v_conf_overview.estimation_cell = f_a_cell.gid)
where
	v_conf_overview.estimation_cell_label like '25km%'
	and estimate_type_str = '1p_total'
	and target_variable = 1
	and area_domain_category is null;

Automated tests performing calculations of estimates

• fst_1p_total
• fst_1p_ratio
• fst_2p_gbeta
• fst_2p_total
• fst_2p_ratio