journey_doc.md 36.6 KB
Newer Older
Julien Cloud's avatar
Julien Cloud committed
1
2
<img src="https://gitlab.com/fastplatform/website/-/raw/master/assets/images/logo.svg" height="48" />
<img src="https://gitlab.com/fastplatform/website/-/raw/master/assets/images/european_commission.svg" style="float:right" height="48" />
Julien Cloud's avatar
Julien Cloud committed
3
4
5

# FaST Journey Documentation

Julien Cloud's avatar
Julien Cloud committed
6
> This document can be downloaded as a [pdf file](https://gitlab.com/fastplatform/docs/-/raw/master/journey_doc/journey_doc.pdf?inline=false)
Julien Cloud's avatar
Julien Cloud committed
7
8
9

The goal of the Journey Documentation is to **support Member States' Paying Agencies (MS PA) in their understanding of what is involved in joining FaST**, through a description of what they can expect to experience during on-boarding and throughout the project.

Julien Cloud's avatar
Julien Cloud committed
10
**Table of contents**
Julien Cloud's avatar
Julien Cloud committed
11

Julien Cloud's avatar
Julien Cloud committed
12
- [Part 1 - Preparation activities](#part-1-preparation-activities). This section summarizes the activities that PAs are expected to conduct, resources PAs should provide, project reference documents and assets, meetings/workshops that will be organized
Julien Cloud's avatar
Julien Cloud committed
13

Julien Cloud's avatar
Julien Cloud committed
14
- [Part 2 - Project startup](#part-2-project-startup). This phase corresponds to the first weeks of the project, when PA are onboarding. This phase will most specifically make an introduction to the FaST platform (features, what it is, etc.). This onboarding phase is also an opportunity to connect with the EC, ask questions about the project during the kick-off meeting. This phase will enable you to clearly understand your involvement in the project, present what is expected from you, and what you can expect from the delivery team and the EC. During this phase, all the technical streams will be kickstarted: interlocutors will be identified and technical discussions will get going
Julien Cloud's avatar
Julien Cloud committed
15

Julien Cloud's avatar
Julien Cloud committed
16
- [Part 3 - Iterative development & testing](#part-3-iterative-development-testing). During the (main) phase of the implementation, the PwC delivery team will implement the various customizations and algorithms while the PA will develop the necessary data access points (if necessary). The customizations will be delivered iteratively, and incrementally tested by test users designated by the PA.
Julien Cloud's avatar
Julien Cloud committed
17

Julien Cloud's avatar
Julien Cloud committed
18
- [Technical annex](#technical-annex)
Julien Cloud's avatar
Julien Cloud committed
19

Julien Cloud's avatar
Julien Cloud committed
20
This section provides an overview of the expected activites and a timeline of the different events that will happen during the entire project lifecycle. The timeline deals with development, support and delivery aspects, together with the main meetings/workshops that will be organized all along the project.
Julien Cloud's avatar
Julien Cloud committed
21

Julien Cloud's avatar
Julien Cloud committed
22
23
24
25
26
27
28
<p align="center">
<img src="img/app_screenshot_small.png">
</p>

Below is theoretical timeline of the project over a single year (note that the starting date is purely indicative). 

<p align="center">
Julien Cloud's avatar
Julien Cloud committed
29
<img src="https://gitlab.com/fastplatform/docs/-/raw/master/journey_doc/img/timeline.png" height="200" />
Julien Cloud's avatar
Julien Cloud committed
30
</p>
Julien Cloud's avatar
Julien Cloud committed
31
32
33
34
35
36
37
38
39

## Part 1 - Preparation activities

Before the official kick-off-meeting, all participating regions/countries should prepare themselves in order to be as efficient as possible in the first weeks of the project. This section aims at giving you tips to better prepare yourself, with a list of activities you can start performing. Please keep in mind that the more ready you are before the official kick-start of the project, the smoother the project will be.

During this phase, we advise you focus on familiarizing yourself with the necessary datasets, APIs and design choices mentioned in this document.

The delivery team can support you by providing any additional detail you need to prepare the key activities listed below.

Julien Cloud's avatar
Julien Cloud committed
40
### The FaST platform
Julien Cloud's avatar
Julien Cloud committed
41
42
43

FaST is a digital service platform where capabilities for agriculture, environment and agriculture sustainability are made available to farmers, farm advisors, EU Member States’ Paying Agencies (MS/PA) and researchers through a user-friendly experience.

Julien Cloud's avatar
Julien Cloud committed
44
For farmers and farm advisors, access to FaST is via a mobile phone application or a web portal: the FaST App, and also using the FaST Web Portal. MS/PA will also connect to the FaST through a customised Web Portal. Researchers and third parties will be able to access FaST through APIs.
Julien Cloud's avatar
Julien Cloud committed
45
46
47
48
49

The FaST platform will also expose capabilities required for the administration of the platform and to permit the integration of external products and services.

A high level list of proposed FaST platform capabilities appears below:

Julien Cloud's avatar
Julien Cloud committed
50
51
52
53
54
55
56
- Maps overlaying farm data on GIS layers including custom government-published maps
- Copernicus/Sentinel imagery: RGB+NDVI
- Campaign management with import of IACS/GSAA farmer data
- Fertilization recommendation
- Geo-tagged photos
- Two-way communications between Paying Agency and farmer
- Basic weather/climate
Julien Cloud's avatar
Julien Cloud committed
57
58
59
60
- User management and reporting

You should read about the project and from the following links, to better understand the objectives and features of FaST:

Julien Cloud's avatar
Julien Cloud committed
61
- FaST Platform website: https://fastplatform.eu/
Julien Cloud's avatar
Julien Cloud committed
62
63
64
65
66
67
68
69
- Mockup (visual) developed during the FaST feasibility study: https://www.figma.com/proto/WYUoKmrxS5ahE8VkGomwns8X/Farmer-app?node-id=1%3A86&scaling=min-zoom
- Context about FaST:
    - https://ec.europa.eu/info/news/new-tool-increase-sustainable-use-nutrients-across-eu-2019-feb-19_en
    - https://ec.europa.eu/jrc/sites/jrcsh/files/03-fast_final.pdf
    - https://www.slideshare.net/EU_GNSS/farm-sustainability-tool-for-nutrients-by-isidro-campos-ec

### Project Management

Julien Cloud's avatar
Julien Cloud committed
70
You should identify individuals able to coordinate some aspects of the project collaboration, from the Paying Agency side. This includes profiles such as (but not limited to):
Julien Cloud's avatar
Julien Cloud committed
71

Julien Cloud's avatar
Julien Cloud committed
72
73
74
- **Paying Agency staff** supporting use of FaST and able to make design decisions with regards to the FaST customizations and provide the necessary data access authorizations when required
- **Technical referents** for IACS API development, authentication systems and agronomic algorithms
- **Motivated farmers** - ideally mixing young/older, digital/not digital, for a representative sample of the farmers in the region/country
Julien Cloud's avatar
Julien Cloud committed
75
76
77
        
The people selected at the beginning of the project should be willing to commit from project start to finish.

Julien Cloud's avatar
Julien Cloud committed
78
### Project Assets & Reference Documentation
Julien Cloud's avatar
Julien Cloud committed
79

Julien Cloud's avatar
Julien Cloud committed
80
During the preparation phase, you can gather more detailed knowledge about FaST by checking the repositories of project assets built during the phase 1 of FaST:
Julien Cloud's avatar
Julien Cloud committed
81

Julien Cloud's avatar
Julien Cloud committed
82
83
84
85
86
- [Source code](https://gitlab.com/fastplatform) (95% open access)
- [Technical documentation](https://gitlab.com/fastplatform/docs) (open access)
- [User documentation (Administration Portal)](https://gitlab.com/fastplatform/core/-/blob/master/services/web/backend/docs/en/index.md) (open access)
- [User quickstart (mobile app)](https://gitlab.com/fastplatform/docs/-/blob/master/quickstart/app-quickstart-en.pdf) (open access)
- [Project wiki](https://fastplatform.atlassian.net/wiki/spaces/PF) (credentials needed)
Julien Cloud's avatar
Julien Cloud committed
87

Julien Cloud's avatar
Julien Cloud committed
88
If you have any question about those assets, reach out to the [delivery team](mailto:tech@fastplatform.eu).
Julien Cloud's avatar
Julien Cloud committed
89

Julien Cloud's avatar
Julien Cloud committed
90
## Part 2 - Project startup
Julien Cloud's avatar
Julien Cloud committed
91

Julien Cloud's avatar
Julien Cloud committed
92
This phase corresponds to the first weeks of the project, when the Paying Agencies are onboarding. This phase will most specifically make an introduction to the FaST platform (features, what it is, etc). This onboarding phase is also an opportunity to connect with the EC, ask questions about the project during the first meeting in Brussels.
Julien Cloud's avatar
Julien Cloud committed
93
94
95

This phase will enable you to clearly understand your involvement in the project, present what is expected from you, and what you can expect from PwC and the EC.

Julien Cloud's avatar
Julien Cloud committed
96
During the first weeks, the delivery team will accompany you to setup the necessary coordination structure that will be used during the course of the project, and kickstart all the technical streams.
Julien Cloud's avatar
Julien Cloud committed
97

Julien Cloud's avatar
Julien Cloud committed
98
### Setup coordination and management
Julien Cloud's avatar
Julien Cloud committed
99
100
101
102
103

The coordination and management effort aims at harmonizing all the other activity streams and takes the following form:

- **Bi-weekly calls** (every two weeks) to review the current open issues with regards to customizations, prioritize actions and effort:
  - Attendance:
Julien Cloud's avatar
Julien Cloud committed
104
105
106
107
    - *Paying Agency:* the project sponsors, as well as technical experts (fertilization, authentication, IACS connectivity) as necessary
    - *Delivery team:* project managers as well as technical experts will be present on an ad hoc basis
    - *European Commission* stakeholders might attend some of those calls, as necessary
  - Timing: the timing of these meetings will be agreed between the delivery team and the Paying Agency. They usually last between 30 minutes and 1 hour every two weeks
Julien Cloud's avatar
Julien Cloud committed
108
109
110
111
112
113
  - Logistics: those meeting will be held over video conferencing

- **Email exchanges** (as required) are frequent between the delivery team and the Paying Agency experts. During this onboarding phase, email communications will be intense as both teams get to know each other and the first technical aspects are clarified.

- **Wiki**: FaST stores some common documents on a (private) wiki, and you should identify the people that will need to be granted access

Julien Cloud's avatar
Julien Cloud committed
114
115
- **Source code**: you should identify people that need access to the full source code/documentation.

Julien Cloud's avatar
Julien Cloud committed
116
117
118
119
120
121
122
123
124
125
> **What should you expect in terms of effort?**
>
> - Video conferences:
>   - Coordination: 1h every 2 weeks
>   - Technical activities: approx 1h per activity per week (IACS, authentication, algorithm)
> - Workshops: 1 to 4 workshops of 1 to 2 days each
> - Emails: 5 to 10 emails per week, covering coordination activities and technical discussions
> - API development: the development of APIs for connecting to the IACS system / farm register will greatly depend on your current situation. If these APIs already exist and are suitable for FaST interconnection, you will have nothing to do and the FaST delivery team will develop the necessary connector on the FaST backend. If you do not have APIs already, or if the APIs you have do not provide the information that FaST needs, then you will need to develop those.
> 
> The effort will be greater at the start of the project, when most of the information / clarifications have to be exchanged. You can count approximatey 2 to 3 times the above estimate during the firsy 2 months of the project.
Julien Cloud's avatar
Julien Cloud committed
126
127

### Workshops
Julien Cloud's avatar
Julien Cloud committed
128

Julien Cloud's avatar
Julien Cloud committed
129
User workshops will be organized to gather feedback, suggestions, improve the overall FaST experience and understand your local specificities.
Julien Cloud's avatar
Julien Cloud committed
130

Julien Cloud's avatar
Julien Cloud committed
131
These 1 to 4 workshops need to be organised to ensure their efficiency. Depending on the Covid-19 situation, these workshops will be either held in situ (in your country/region) or through video conferencing.
Julien Cloud's avatar
Julien Cloud committed
132

Julien Cloud's avatar
Julien Cloud committed
133
The workshops organization includes:
Julien Cloud's avatar
Julien Cloud committed
134

Julien Cloud's avatar
Julien Cloud committed
135
136
137
138
139
140
141
142
- selecting a suitable date
- identifying a point of contact for organisation of workshop logistics
- organizing a preliminary meeting and dedicated workshops in collaboration with PwC
- involving a relevant set of interlocutors / main stakeholders of agricultural sector in the region/country relevant to the project
- Each user review session will last about 1,5 hours and will be conducted in a group of 1 to 3 people. The three target reviewing groups are:
  - Farmers and Farm Advisors
  - MS/PA personnel
  - Researchers / other relevant stakeholders
Julien Cloud's avatar
Julien Cloud committed
143

Julien Cloud's avatar
Julien Cloud committed
144
**Workshop Groups Definition**
Julien Cloud's avatar
Julien Cloud committed
145

Julien Cloud's avatar
Julien Cloud committed
146
A review workshop consists of three different groups participating in different sessions of 1.5 hours each during 1 day. You should solicit participation for each group for a minimum of 1 person per group, and a maximum of 3 or 4 people. Ideally these same people will be able to participate in all future workshops and testing sessions.
Julien Cloud's avatar
Julien Cloud committed
147

Julien Cloud's avatar
Julien Cloud committed
148
**Workshop Organisation**
Julien Cloud's avatar
Julien Cloud committed
149

Julien Cloud's avatar
Julien Cloud committed
150
Each workshop will consist of 1 session per user group. Each session should last 1h30 with 15 to 30 minutes between any back to back sessions. Sessions must be successive but can be organised how you wish, spreading over the 2 days if necessary.
Julien Cloud's avatar
Julien Cloud committed
151
- Plan a lunch break for at least 1.5h
Julien Cloud's avatar
Julien Cloud committed
152
- Plan 1h30 at the end of the day to debrief if possible
Julien Cloud's avatar
Julien Cloud committed
153

Julien Cloud's avatar
Julien Cloud committed
154
**Example Agenda**
Julien Cloud's avatar
Julien Cloud committed
155

Julien Cloud's avatar
Julien Cloud committed
156
*Day 1*
Julien Cloud's avatar
Julien Cloud committed
157

Julien Cloud's avatar
Julien Cloud committed
158
159
160
161
- User Group Session 1: T0 -> T0+1h30 : PA/MS users and user admins
- User Group Session 2: T0+1h45 -> T0+3h15 : Farmers & Farm Advisors
- User Group Session 3: T0+3h45 -> T0+5h15 : Researchers, data users
- End of day: 1h30 debrief meeting
Julien Cloud's avatar
Julien Cloud committed
162

Julien Cloud's avatar
Julien Cloud committed
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
*Day 2*

Meetings with IT people, stakeholders, data users, MS/PA ... whomever is necessary for extra precision on user session subjects or other FaST subjects including technical

**Physical Arrangements** (in person workshops only)

Please assemble the following to the best of your abilities:
- Meeting room for 6 to 9 people, with room to move and walls for posting...
- Post-it board with adhesive sheets (63.5 cm x 77.5 cm)
- 12 markers
- Colored post-it notes
- Monitor and / or projector
- Wifi & connection information
- Power strips
- Mobile phones for each participant & laptops if available

### Technical activities
Julien Cloud's avatar
Julien Cloud committed
180

Julien Cloud's avatar
Julien Cloud committed
181
182
The technical activities will be carried out during the implementation phase, and need to be started as soon as possible. Some of them involve a significant effort from you, from the PwC delivery team, or both.

183
```plantuml
Julien Cloud's avatar
Julien Cloud committed
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
left to right direction
skinparam defaultFontSize 12

title Overview of technical activities\n

card "PwC delivery team" as pwc #lightpink {
  agent "Management" as pwc_management
  agent "Developers" as pwc_dev
}

card "MS Paying Agency" as ms_pa #lightyellow {
  agent "Management" as ms_pa_management
  agent "Fertilization\nalgorithm\nexpert(s)" as ms_pa_fertilization
  agent "Data\nexpert(s)" as ms_pa_data
  agent "IT\nexpert(s)" as ms_pa_it
  agent "IACS IT\nexpert(s)" as ms_pa_iacs_it
}

package "Project Management" as project_management #lightblue {
  usecase "Project coordination" as coordination
  usecase "User workshops\n& tests" as tests
}

package "Customizations" as customizations #lightblue {
  usecase "Fertilization\nalgorithm" as fertilization
  usecase "Integration\nof data sources" as data
  usecase "Authentication" as authentication
  usecase "Weather provider\nintegration" as weather
  usecase "IACS/GSAA\nintegration" as iacs
}

pwc_management --> coordination: Manages\nthe project
pwc_management --> tests: Sets up\ntests
pwc_dev --> fertilization: Implements\nalgorithm\nandUI
pwc_dev --> data: Integrates\ndata sources
pwc_dev --> authentication: Implements\nauthentication\nconnector
pwc_dev --> weather: Implements\nweather\nconnector
pwc_dev --> iacs: Implements\nIACS/GSAA\nconnector

coordination <-- ms_pa_management: Advises and\narbitrates
tests <-- ms_pa_management: Collects\nuser feedback
fertilization <-- ms_pa_fertilization: Provides algorithm
data <-- ms_pa_data: Indentify data sources
authentication <-- ms_pa_it: Identify/implement\nauthentication\nmechanism
weather <-- ms_pa_it: Identify\nweather\ndata source(s)
iacs <-- ms_pa_iacs_it: Indentify/implement\naccess APIs
```
Julien Cloud's avatar
Julien Cloud committed
231

Julien Cloud's avatar
Julien Cloud committed
232
#### Integration of (static) data sources
Julien Cloud's avatar
Julien Cloud committed
233

Julien Cloud's avatar
Julien Cloud committed
234
235
236
FaST builds on several data sources, which are either connected (live sources) or imported (static sources) in the platform. 
  
The necessary data sources can be summarized as:
Julien Cloud's avatar
Julien Cloud committed
237

Julien Cloud's avatar
Julien Cloud committed
238
239
240
241
242
243
244
- **Soil data:** necessary to provide default soil information to users and to be used as default values in the fertilization algorithms
- **Surface waters and water courses:** necessary for display, to compute constraints and nitrogen limitations that depend on water body proximity
- **Nitrate Vulnerables Zones:** necessary for display, to compute constraints and nitrogen limitations that depend on being within an NVZ
- **Natura2000 areas:** mostly for display and to comute constraints
- **List of plant species and varieties:** this is the list of crop that will be used by all users in your region/country. This list can be different from the IACS crop nomenclature or from the way crops are described within your fertilization algorithm: however, in both cases, you will need to provide a mapping table between each nomenclature.
- **List of fertilizer products:** if your fertilization algorithm produces actual *fertilizer product recommendations* (in addition to *chemical element recommendation*, such as N, P & K), then you should provide a base list of fertilizer products, either organic or mineral
- **Legal limits for nitrogen** and conditions that apply to each limit, such as crop yield or NVZ
Julien Cloud's avatar
Julien Cloud committed
245
- **Your own WMS/TMS layers** that you wish to display to FaST users, for example orthophotos, custom government maps or other agronomic layers of interest for the farmers
Julien Cloud's avatar
Julien Cloud committed
246

Julien Cloud's avatar
Julien Cloud committed
247
248
249
<p align="center">
<img src="img/admin_map.png" height="400">
</p>
Julien Cloud's avatar
Julien Cloud committed
250

Julien Cloud's avatar
Julien Cloud committed
251
252
253
These data sources are dubbed *"static"* because they are not expected to change often and do not depend on user input. It does not mean that they will be fixed forever. You will be able to update them, through the Administration Portal.

> **Activities:**
Julien Cloud's avatar
Julien Cloud committed
254
255
256
>
> - Identify the existence and location of the above data sources in your GIS management systems or in public data repositories
> - Start identifying the potential WMS/TMS layers that you like your users to see in the app (in addition to the FaST standard layers)
Julien Cloud's avatar
Julien Cloud committed
257

Julien Cloud's avatar
Julien Cloud committed
258
### Authentication provider
Julien Cloud's avatar
Julien Cloud committed
259

Julien Cloud's avatar
Julien Cloud committed
260
The users of FaST, either using the mobile application or the administration portal, have to be *authenticated* to determine their user account and give them access to their data.
Julien Cloud's avatar
Julien Cloud committed
261

Julien Cloud's avatar
Julien Cloud committed
262
To make the life of farmers easier, FaST will always try to build up on systems they already know and use ; for authentication, this means that FaST will endeavour to reuse existing Paying Agency login systems or national login systems, to avoid avoid double inputs and farmers having to remember additional credentials.
Julien Cloud's avatar
Julien Cloud committed
263

Julien Cloud's avatar
Julien Cloud committed
264
FaST provides its own way to authenticate users, through a login+password combination. However, this basic authentication system has several drawback when dealing with a lot of users like farmers:
Julien Cloud's avatar
Julien Cloud committed
265

Julien Cloud's avatar
Julien Cloud committed
266
- login+password combinations have to be created in advance for each and every user
Julien Cloud's avatar
Julien Cloud committed
267
- passwords have to be communicated to each user *while verifying the legal identity of the user*
Julien Cloud's avatar
Julien Cloud committed
268

Julien Cloud's avatar
Julien Cloud committed
269
To solve the above, FaST provides a mechanism to *delegate* this authentication to other systems such as **national identity providers** or **authentication systems from the Paying Agency**.
Julien Cloud's avatar
Julien Cloud committed
270

Julien Cloud's avatar
Julien Cloud committed
271
Note that each identity provider does not necessarily cover the full spectrum of FaST users for a given region. For example, some identity providers provide only authentication for farmers (like the providers from Paying Agencies, usually), while some other providers cover all the individuals within a country (like national systems such as [SPID](https://www.spid.gov.it/) or [TARA](https://www.ria.ee/en/state-information-system/eid/partners.html#tara)).
Julien Cloud's avatar
Julien Cloud committed
272

Julien Cloud's avatar
Julien Cloud committed
273
```plantuml
Julien Cloud's avatar
Julien Cloud committed
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
skinparam defaultFontSize 12

actor "             User\n(mobile / Admin Portal)" as user

rectangle idp_select as "
FaST IdP selection
--
- FaST Internal Identity Provider
- Federated Identity Provider #1 (e.g. TARA)
- Federated Identity Provider #2 (e.g. OPCyL)
..."

rectangle fast_idp as "
FaST login screen
--
Username: ______________
Password: ______________
"

cloud fed_idp_1 #cyan as "
IdP #1 login screen
--[outside FaST]--
Custom mechanism that
depends on the IdP
"

cloud fed_idp_2 #cyan as "
IdP #2 login screen
--[outside FaST]--
Custom mechanism that
depends on the IdP
"

usecase return as "
IdPs return the
**username** to FaST
upon successful
authentication"

usecase logged_in as "User is logged in to FaST"

user --> idp_select
idp_select --> fast_idp : Selects\nFaST IdP
idp_select --> fed_idp_1 : Selects\nFederated IdP #1
idp_select --> fed_idp_2 : Selects\nFederated IdP #2

fast_idp --> return
fed_idp_1 --> return
fed_idp_2 --> return
Julien Cloud's avatar
Julien Cloud committed
323

Julien Cloud's avatar
Julien Cloud committed
324
325
return --> logged_in
```
Julien Cloud's avatar
Julien Cloud committed
326

Julien Cloud's avatar
Julien Cloud committed
327
328
329
330

You must survey the potential systems that exist in your region/country and identify which one(s) would be suitable to authenticate farmers. Depending on the outcome of this survey, we will liaise with you to determine the most suitable way forward.

> **Activities:**
Julien Cloud's avatar
Julien Cloud committed
331
332
333
334
>
> - Identify an authentication system for farmers for your region, that FaST can delegate authentication to
> - If such a system does not exist or is not suitable for interconnection with FaST, start the developments necessary to build such a system
> - Name the corresponding experts that will assist the delivery team during the implementation phase (for clarifications, tests, validation)
Julien Cloud's avatar
Julien Cloud committed
335

Julien Cloud's avatar
Julien Cloud committed
336
### Fertilization algorithm
Julien Cloud's avatar
Julien Cloud committed
337

Julien Cloud's avatar
Julien Cloud committed
338
The first agronomic use case implemented on FaST is *fertilization advice*. This advice is proposed to the farmer and computed on the basis of:
Julien Cloud's avatar
Julien Cloud committed
339

Julien Cloud's avatar
Julien Cloud committed
340
341
342
343
- data stored on FaST (such as soil data)
- farmer data stored on FaST (such as parcel geometries or custom soil samples)
- custom forms where additional information is requested to the farmer
- a computational algorithm that returns the fertilization advice
Julien Cloud's avatar
Julien Cloud committed
344

Julien Cloud's avatar
Julien Cloud committed
345
As per the Terms of Reference of FaST, it is the responsibility of the Payging Agency / participating MS (you) to provide such a *"operational algorithm for fertilization"* to the delivery team.
Julien Cloud's avatar
Julien Cloud committed
346

Julien Cloud's avatar
Julien Cloud committed
347
As part of the preparation activities, you should survey the relevant universities and agronomic institutes to identify the desired algorithm to be implemented into FaST for your users.
Julien Cloud's avatar
Julien Cloud committed
348

Julien Cloud's avatar
Julien Cloud committed
349
> **Activities:**
Julien Cloud's avatar
Julien Cloud committed
350
351
352
353
354
>
> - Identify an operational fertilization algorithm to be implemented into FaST
> - Confirm the licensing situation of the proposed algorithm is legally suitable to be included into FaST
> - Provide to the delivery team all the source code, documentation, databases and test cases required to port the algorithm to the FaST platform
> - Name the corresponding experts that will assist the delivery team during the implementation phase (for clarifications, tests, validation)
Julien Cloud's avatar
Julien Cloud committed
355

Julien Cloud's avatar
Julien Cloud committed
356
### IACS connection
Julien Cloud's avatar
Julien Cloud committed
357

Julien Cloud's avatar
Julien Cloud committed
358
In order to provide access to his own data to the farmer, FaST connects to the regional/national IACS system (or equivalent farm registry), where the data of the farmer is stored. This data is usually derived from the GSAA (geospatial aid application) of the farmer from the previous year.
Julien Cloud's avatar
Julien Cloud committed
359

Julien Cloud's avatar
Julien Cloud committed
360
The data that FAST will need to be able to propose services to the farmer only includes agricultural data:
Julien Cloud's avatar
Julien Cloud committed
361

Julien Cloud's avatar
Julien Cloud committed
362
363
- parcel geometries (polygons)
- previous crops and varieties
Julien Cloud's avatar
Julien Cloud committed
364

Julien Cloud's avatar
Julien Cloud committed
365
This data is "pulled" into FaST but never pushed back into the IACS system ; the process is purely one way. Once the data is inside FaST, the farmer has the possibility to edit it in FaST without any impact his IACS/GSAA data.
Julien Cloud's avatar
Julien Cloud committed
366

Julien Cloud's avatar
Julien Cloud committed
367
> **Activities:**
Julien Cloud's avatar
Julien Cloud committed
368
369
370
371
372
>
> - Identify which IT system contains the relevant farmer's data
> - Identity the existing interconnection mechanisms (APIs) that exists for FaST to query the farmer's data
> - If no such interconnection mechanism exists, start the implementation as soon as possible
> - Provide all the required security credentials, documentation, test accounts and examples necessary for the delivery team to build the corresponding FaST connector. To speed up the process, the delivery team might start implementing the connector on the basis of the documentation alone.
Julien Cloud's avatar
Julien Cloud committed
373

Julien Cloud's avatar
Julien Cloud committed
374
### Weather/climate provider(s)
Julien Cloud's avatar
Julien Cloud committed
375

Julien Cloud's avatar
Julien Cloud committed
376
FaST provides a basic weather/climate widget on the mobile application. The data used to generate this widget should be provided by a weather/climate API that you provide. The delivery team will then build the necessary connector and user interface.
Julien Cloud's avatar
Julien Cloud committed
377

Julien Cloud's avatar
Julien Cloud committed
378
As a minimum, the weather provider should expose:
Julien Cloud's avatar
Julien Cloud committed
379

Julien Cloud's avatar
Julien Cloud committed
380
381
- a forecasts API with temperature, wind and rain predictions
- (optionally) an observations API with past temperatures, wind and rain values
Julien Cloud's avatar
Julien Cloud committed
382

Julien Cloud's avatar
Julien Cloud committed
383
384
385
386
387
388
389
390
391
392
393
<p align="center">
<img src="img/weather.png">
</p>

For example, the following providers have already been connected to FaST:

- Spain: [AEMET OpenData](https://opendata.aemet.es/) & [SiAR](http://eportal.mapa.gob.es/websiar/Inicio.aspx)
- Estonia: [Ilmateenistus](https://www.ilmateenistus.ee/)
- Piemonte: [ARPA Bolletino Meteorologico](http://www.arpa.piemonte.it/news/il-bollettino-meteorologico-in-formato-xml-xtensible-markup-language)

> **Activities:**
Julien Cloud's avatar
Julien Cloud committed
394
395
396
>
> - Identify which weather provider would be suitable for FaST users
> - If the weather provider data is not free/open, make the necessary arrangements to get access to the data
Julien Cloud's avatar
Julien Cloud committed
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
> - Provide the provider details and credentials to the delivery team for developing the connector


## Part 3 - Iterative development & testing

During this phase, which will last until the end of the project, the delivery team will deliver increments of added value on a cadenced basis: the development cycles of the delivery team are *sprints* of 2 weeks, and the release schedule is approximately aligned with the sprints: you can expect a new version of the application or of the Administration Portal on a regular basis.

If developments are needed o the Paying Agency side (like IACS APIs or authentication APIs), the delivery team will always try to work in parallel, on the basis of commonly agreed interface specifications.

<p align="center">
<img src="img/agile.svg">
</p>

### Agile development cycles

The development of the customizations is follows an agile methodology and the backlog is prioritized at the end of every sprint. The team is therefore able to react quickly to incoming changes (such as APIs becoming available or changing).

**Small autonomous and cross functional teams**: Teams consist of members representing both users and producers of the solution. Teams are small in order to facilitate communication and minimize overhead introduced by large team size. Because users review the solution at regular intervals, they provide immediate feedback. Having a close relationship with the users of the system at regular intervals allows the delivery team to be completely autonomous since no delay in advancement is introduced by missing or untimely feedback. 

**Limit “work in progress” and focus on incremental delivery**: In order to prevent dissipation of team members across too many subjects, functionality is developed incrementally. This ensures that the producers of the solution can deliver often. Each delivery adds incremental value, and each cycle delivers a working system.

**Work without interruption and with daily exchanges**: The delivery team is protected from outside distractions by the person leading the Scrum team. Communication between team members and users of the system occurs through the regularly spaced exchanges (sometimes called a “weekly”). This allows each member of the team (both users and developers) to have access to necessary information, and allows the developers of the solution to work uninterrupted between these exchanges.

**Collect customer feedback for each cycle or iteration and adjust**: The iteratively delivered system becomes a ‘living’ set of specifications. Having a living specification does not remove the need for any required written specifications for understanding and supporting the system, which can be developed in parallel or at the appropriate time. The iterative delivery means that users see the developing system often, and feedback is collected. This allows necessary changes to be incorporated. Frequent, early feedback reduces the risk of not meeting the expectations of the users.

### Technical activities


### Testing

-	Beta users will also participate in the UI/design workshops to iteratively validate and refine the user experience and interfaces of the system
-	Our Continuous Deployment methodology implies that end users will be presented live versions of each system very early on and very frequently, to gather their feedback
-	The user groups envisaged for the beta-testing will be discussed with the MS/MA/PA at the start of the project. A first proposal for the composition of these groups appears below

<p align="center">
<img src="img/testing_process.svg">
</p>

The user feedback is collected under a ticketing system for the whole project ; from there, the issues are tagged and prioritized with the rest of the backlog. Clarifications or reproducing instructions might be requested to the testing user.

## Lessons learnt




## Technical annex

This purpose of this short annex is to give you a quick technical primer on the requirements with regards to data and APIs. We now have the experience of the first year of the project, which involved 4 regions/countries, and will here extract the main requirements

### "Static" data sources

#### Soil data

> - Must be georeferenced
> - Should be of suitable density to for meaningful interpolation
> - Should contain at least measurements of chemical properties that are of interest for the fertilization algorithm being proposed: for example, if the algorithm needs the soil value for C/N ratio, it is better if the soil data being loaded provides this value. If the value is not part of the soil dataset, or if the geospatial density is not suitable, the farmers will have to provide the value themselves in the app.
> - Preferred format: an OGC vector format such as WFS, or a flat file (e.g. shapefile)

##### Examples from FaST v1

- Castilla y Leon: <https://servicios.itacyl.es/arcgis/services/Visor_Suelos/MapServer/WCSServer> (WCS server)
- Piemonte: <https://www.regione.piemonte.it/web/temi/agricoltura/agroambiente-meteo-suoli/suoli-paesaggi-agrari-piemonte> (flat files)

#### Nitrate Vulnerable Zones, Natura2000, Hydrography

> - Must be georeferenced
> - Should have attributes following the INSPIRE semantics, ideally
> - Preferred format: an OGC vector format such as WFS, or a flat file (e.g. shapefile)

<p align="center">
<img src="img/admin_nvz.png" height="350"><br>
Nitrate Vulnerable Zones
</p>

<p align="center">
<img src="img/admin_natura2000.png" height="350"><br>
Natura2000 areas
</p>

<p align="center">
<img src="img/admin_hydro.png" height="350"><br>
Hydrography
</p>

##### Examples from FaST v1

- Castilla y Leon:
  - Natura2000 areas: <https://idecyl.jcyl.es/geonetwork/srv/eng/catalog.search#/metadata/SPAGOBCYLMNADTSPSZEC> (IDECyL)
  - Hydrography: <https://idecyl.jcyl.es/geonetwork/srv/eng/catalog.search#/search?facet.q=inspireThemeURI%2Fhttp%253A%252F%252Finspire.ec.europa.eu%252Ftheme%252Fhy> (IDECyL)
- Estonia:
  - Nitrate Vulnerable Zones and Natura2000 areas: <https://gsavalik.envir.ee/geoserver/eelis/ows?service=WFS&version=1.3.0&request=GetCapabilities> (Minstry of Environment)
  - Hydrography: <https://geoportaal.maaamet.ee/index.php?lang_id=2&page_id=618> (Maamet)
- Piemonte:
  - Hydrography: <http://webgis.arpa.piemonte.it/w-metadoc/Download/WFD2000_60_CE_BDTRE.zip> (ARPA)
  - Geoportale: <http://www.geoportale.piemonte.it/geonetworkrp/srv/ita/metadata.show?id=6999&currTab=rndt>

#### Layer of all agricultural parcels (LPIS)

> - Must be georeferenced
> - Should be polygons that correspond to a real continuous field where the same crop is usually cultivated. This may or may not correspond exactly to the LPIS layer. For example, for Castilla y Leon, this layer is computed based on a merge of adjacent parcels extracted from the GSAA polygons. For Estonia, it is the layer of GSAA polygons.
> - Preferred format: an OGC vector format such as WFS, or a flat file (e.g. shapefile)

Note that this layer is necessary in FaST only if you activate the feature that allows farmers to add new parcels to their farm in FaST.

<p align="center">
<img src="img/admin_plots.png" height="350"><br>
All agricultural plots
</p>

##### Examples from FaST v1

- Castilla y Leon: <https://servicios.itacyl.es/arcgis/rest/services/mirand/Fois/MapServer/0> (ArcGIS layer)
- Estonia: <https://kls.pria.ee/geoserver/pria_avalik/ows> (WFS layer)

### Authentication

- Can be based on a known standard (SAML, OIDC) or be a custom development
- Must be idempotent, i.e. always return the same user identifier for the same person
- Must return an identifier that is compatible with the IACS API (see below). If the user identifier is different, there must exist an unambiguous method to map the user identifier returned by the authentication system user identifier to the IACS API user identifier
- In case of a custom development, there must by a way for FaST to guarantee the origin of the callback (signature, etc)

Julien Cloud's avatar
Julien Cloud committed
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
```mermaid
sequenceDiagram
  Note over FaST App: App starts
  alt If no region & token stored in app
    FaST App->>FaST Backend: Get list of regions/endpoints
    FaST Backend->>FaST App: 
    Note over FaST App: User selects region
    FaST App->>FaST Backend: Get list of identity providers for region
    FaST Backend->>FaST App: 
    Note over FaST App: User selects IdP
    FaST App->>Identity Provider: Redirect to identity provider login page
    Note over Identity Provider: User logs in with IdP
    Identity Provider->>FaST Backend: Redirect with user IdP ID, info and authorization code
    FaST Backend->>FaST Backend: Get FaST user for user IdP ID
    alt User does not exist
      FaST Backend->>FaST Backend: Create user
      FaST Backend->>Authorization Provider: Request initial access rights for user
      Authorization Provider->>FaST Backend: 
      FaST Backend->>FaST Backend: Assign rights/roles
    end
    FaST Backend->>FaST App: Return user details and FaST token
    FaST App->>FaST App: Persist region & FaST token in app
  else Else if region & token already in app
    FaST App->>+FaST Backend: Check token valid
    FaST Backend->>FaST Backend: Refresh token if necessary/allowed
    alt If token is valid
      FaST Backend->>-FaST App: Return (refreshed) token
      FaST App->>FaST App: Persist token in app
    else If token not valid
      FaST Backend->>FaST App: Return login error
      FaST App->>FaST App: Delete local token
      Note over FaST App: Redirect to login page
    end
  end
```

Julien Cloud's avatar
Julien Cloud committed
554
555
##### Examples from FaST v1

Julien Cloud's avatar
Julien Cloud committed
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
- Castilla y Leon developed a custom authentication API based on their IACS user database. The API proposes an additional endpoint to check the validity of the token that FaST receives.
- Andalucia expanded their already existing platform [DAT](https://www.juntadeandalucia.es/organismos/agriculturaganaderiapescaydesarrollosostenible/areas/agricultura/produccion-agricola/paginas/dat.html) to allow it to return a signed user identity to an external service. The API exposes a JWKS endpoint that serves the public key corresponding to the signature of the token that FaST receives.
- Estonia uses the [TARA](https://e-gov.github.io/TARA-Doku/Arikirjeldus) national authentication system. The TARA system is based on the OIDC protocol.
- Piemonte uses the [SPID](https://developers.italia.it/it/spid/) national authentication system. The SPID system is based on the SAML protocol.

### IACS/GSAA access

> - Must be served over an HTTP API
> - Credentials / security requirements must be provided to the delivery team
> - Must provide at least the 2 following capabilities:
>   1. Given a user identifier as input, return the list of farms/holdings that this user is allowed to access. Note that, in the case of a farm advisor, this list might contain more than 1 farm.
>   2. Given a farm identifier (and optionally, a year) return the list of parcels for this farm (for this IACS year), including but not limited to:
>       - parcel geometry
>       - the crop that was cultivated/declared
>       - (optional) the irrigation practice

Security-wise, the delivery team will adapt to your requirements, using for example:

- IP whitelisting
- Custom authentication
- API keys
- Custom network (e.g. X-Road)
- ...

```mermaid
sequenceDiagram
    FaST App ->> FaST Backend: User logs first time
    FaST Backend ->> Identity Provider: Delegate authentication
    Identity Provider ->> FaST Backend: Authenticated user U
    FaST Backend ->> IACS API: Get farms and roles for user U
    Note over IACS API: API GetFarms
    IACS API ->> FaST Backend: List of farms (F1, F2..) and role for each farm (owner, advisor...)
    Note over FaST Backend: Get or create<br/>corresponding farms
    loop For each farm
        FaST Backend ->> IACS API: Get farm F details from user U
        Note over IACS API: API GetFarmDetails
        rect rgba(255, 0, 0, 0.2)
            IACS API ->> IACS API: Check user U<br>permission for<br/>farm F
        end
        IACS API ->> FaST Backend: Farm details
    end
    FaST Backend ->> FaST App: List of farms with details
    Note over FaST App: Display farms
    Note over FaST App: [later...]<br>User requests<br>farm F1 update
    FaST App ->> FaST Backend: User U requests update farm F1
    FaST Backend ->> IACS API: Get farm F1 details from user U
    Note over IACS API: API GetFarmDetails
    rect rgba(255, 0, 0, 0.2)
        IACS API ->> IACS API: Check user U<br>permission for<br/>farm F1
    end
    IACS API ->> FaST Backend: Farm details
    Note over FaST Backend: Update farm F1
    FaST Backend ->> FaST App: Updated farm F1
```

##### Examples from FaST v1

Julien Cloud's avatar
Julien Cloud committed
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
- Castilla y Leon developed a custom HTTP API that connects to the regional IACS system
- Estonia developed a custom SOAP service that exposes the necessary Paying Agency objects through the Estonian X-Road network
- Piemonte developed a custom HTTP API that connects to the Paying Agency system
- Andalucia developed APIs to access their existing farm registry / management system

### Weather data provider

> - Must provide weather forecast based on geographical location
> - Must provide at least temperature, wind and rain prediction
> - Must be served over an HTTP API
> - If authentication is needed, credentials must be provided to the delivery team
> - Ideally, should provide those forecasts on an hourly or half-day step
> - Ideally, should provide past measurements as well

##### Examples from FaST v1

- Castilla y Leon and Andalucia:
  - <https://opendata.aemet.es/centrodedescargas/AEMETApi> (AEMET)
  - <http://eportal.mapa.gob.es/websiar/Inicio.aspx> (SiAR)
- Estonia: <https://www.ilmateenistus.ee/teenused/ilmainfo/ilmatikker/> (Ilmateenistus)
- Piemonte: <http://www.arpa.piemonte.it/export/meteoxml/meteo.xml> (ARPA)