Tags

Release 2.9.0

### Added

- Add a new configuration `PICTURE_PROCESS_NB_RETRIES` to tell the number of times a job will be retried if there is a `RecoverableProcessException` during process (like if the blurring api is not reachable). Defaults to 5 like before.
- Add a way to search on semantic tags in `/api/search` endpoint with a CQL2 filter like `filter="semantics.some_key"='some_value'`.
- Add a `API_REGISTRATION_IS_OPEN` setting in the API to tell the federation that account can (or not) be created directly on the instance. Defaults to `false`.
- Added field `geovisio:rank_in_collection` to pictures metadata to get the rank of the picture in the collection in `/api/collections/:cid/items/:id` / `/api/collections/:cid/items` and `/api/search` responses. This makes it easier to use the `startAfterRank` parameter to paginate through pictures in collections..
- Handle semantics returned by the blurring API, and automatically add annotations on the pictures with semantic tags on the detected objects..

### Fixed

- Fix the pagination links when many sequences had the exact same creation date on `/api/collections`.
- Migration `20240409_01_jnhra-pictures-grid.sql` was not working from empty database.
- When asking for collections list as CSV format, API was sending `Content-Type: text/html` instead of `text/csv` in response.
- Fix a sql error on `/api/users/me/collection` when using PostgreSQL <= 13.
- Fix a bug causing a 500 when filtering collections with no updates
- Fix a bug where the collection geometry was not recomputed after a sort.

### Changed

- Change the way the coefficients are computed for the grid, compute the mean only over squares with flat (or 360°) pictures.
- Update lot of dependencies (Flask, psycopg, pillow, sentry, ...)

Release 2.8.1

### Added

- Add a way to know the role and permission of the logged user in `/api/users/me`.
- Add the ability to update an upload set parameters through a `PATCH` on `/api/upload_sets/:id`.
- Add a debug endpoint `/api/debug_headers` that can be handy when setting a new instance to check if the proxies are correctly setting the headers.

### Fixed

- Fix a bug where dispatching several time the same upload set would not cleanup unused sequences.

### Changed

- Only display `tos_accepted` field in `/api/users/me` if ToS are mandatory.

Release 2.8.0

### Added

- Add the ability to export a user's collections in a CSV file by either providing a `format` query parameter or with the `Accept: text/csv` header. The CSV export is not limited to the first 1000 collections as the JSON API.
- More metadata can be defined for the API in `API_SUMMARY` environment variable: contact email and geographical coverage
- Add the ability to blur again a picture/sequence, by calling the `/api/collections/:id/prepare`/`/api/collections/:id/items/:id/prepare` endpoints.
- Add the ability to change a picture's capture time, latitude and longitude using the PATCH `/api/collections/:id/items/:id` route.
- Add the number of collection to the endpoint `/api/users/:id/collection`
- Add _pages_ management to handle legal mentions and terms of service storage in database. New routes `/api/pages` allow to create, list, delete these pages in several languages.
- In vector tiles, pictures have a new `first_sequence` property for an easier access to sequence UUID.
- Add a way to add tags on pictures and sequences, using the PATCH `/api/collections/:id/items/:id` and `/api/collections/:id` routes.
- Add a way for a user to restrict the collaborative editing of its pictures' metadata. This is done by setting the `collaborative_metadata` field of the user in the authenticated API endpoint `/api/users/me` to `false`. If not set, the instance's `collaborative_metadata` default value is used. If set to `true`, anyone with an account can update the heading / position of a pictures, and the `relative_heading` of a whole sequence.
- Add a CLI command to set the role of an account: `flask user <ACCOUNT_ID_OR_NAME> --set-role <ROLE>` (and the user can be created if not exist with the `--create` flag).
- Add a way to accept the terms of service for a user.

### Changed

- We can now view the files of an upload set as anonymous, without needing to be authenticated. It's because this does not contain any sensitive information (file name, md5, size, ...), not the picture in itself.
- Cameras generic metadata (sensor width, GPS accuracy) are now stored in GeoPicture Tag Reader code instead of a `cameras` table in API database.
- Mark the `/api/users/<uuid:userId>/catalog` route as deprecated, in favor of `/api/users/<uuid:userId>/collection` that supports more filtering/sorting parameters and return more information.
- `/api/users/me/collection` is no longer a HTTP 302 redirect to `/api/users/:userId/collection`, but returns directly the collection of the user.
- The user `elysee` in the example docker compose `docker-compose-full.yml` is now created with the role `admin`.
- If the instance is configured without an explicit `API_DEFAULT_COLLABORATIVE_METADATA_EDITING` settigs, pictures' metadata will be editable by any one with an account.
- If the instance is configured with an explicit `API_ENFORCE_TOS_ACCEPTANCE` setting, the instance will not accept pictures if the user has not accepted the terms of service.

### Fixed

- Fix the upload set dispatch in cases where a picture was not linked to a file (due to a bug at upload time, also fixed). Also retry the dispatch job if it fails.
- List of user collection CSV export was failing for some collections.
- Null Unicode character in EXIF fields was breaking picture insertion in database (commonly found in XPComment field).
- Handle limit parameter in `/api/users/me/catalog` route.
- Doc for vector tiles was not up-to-date (missing properties and old zoom level values).

Release 2.7.1

### Added

- Routes returning items/pictures have the `pers:interior_orientation/sensor_array_dimensions` property set, letting users know about original picture dimensions (width/height).
- Add number and coefficient of 360° pictures to the pictures grid. This will makes it possible to display a grid of only 360° pictures.
- Add `missing_fields` list to the `/api/upload_sets/:id/files` response detailling the missing fields from a rejected uploaded picture.
- Details about picture quality are now offered for display and filtering:
  - In sequences and pictures routes: `quality:horizontal_accuracy` for GPS position precision (in meters), `panoramax:horizontal_pixel_density` for pixel density on horizon (px/FOV degree).
  - In vector tiles: `gps_accuracy` and `h_pixel_density` (for __sequences and pictures__ layers, not available on grid layer)
- In vector tiles style, a new metadata property `panoramax:fields` lists all available properties in each layer (pictures, sequences, grid). This will allow easier compatibility checks in web viewer.
- Translations in Spanish 🇪🇸 and Hungarian 🇭🇺.

### Changed

- In vector tiles, grid layer is now returning circles instead of polygons. Map style is also adapted for a smoother transition on low zooms (for a heatmap-like effect).
- More EXIF tags are stripped out of database, to reduce used disk space: all keys containing hexadecimal sub-keys are removed (like `Exif.Sony.0x1234`). These fields are still available in original pictures files. A database migration removes them from existing pictures stock in your database (and could possibly take a bit of time to run...). After migration, you may want to run a little `VACUUM` to reclaim disk space.
- Add number and coefficient of 360° pictures to the pictures grid. This will makes it possible to display a grid of only 360° pictures.
- Change the way deleted pictures/upload_sets are handled, objects are removed from the database sooner than before, without needing a flag to tell that the underlying files needed deletion.

### Fixed

- The rejection reason of duplicate files is now `file_duplicate` instead of `invalid_metadata`.
- Fix a bug when loading a picture with a focal length of 0.

Release 2.7.0

⚠️ The minimal supported Python version is now 3.10. Note that you can run it with newer python versions as some might bring better performance. ⚠

### Added

- The API and picture workers now use connection pool to reduce the load on the database and improve the performances. The connection pool can be configured using `DB_MIN_CNX`/`DB_MAX_CNX`.
- Add a way to use Gunicorn instead of waitress as WSGI server. Gunicorn can use processes instead of threads, which can result in better performance for high load usage.
- The ID of the associated accounts are now returned in the STAC response in an `id` field in the `providers` field.
- Add a whole new way of uploading pictures, using the new `/api/upload_sets` APIs. The pictures added to the uploaded will be dispatched to one or more collection on completion.
- You can create and manage _Reports_ (issues with pictures or sequences) through `/api/reports` routes. Reports can be created anonymously or by authenticated users, can automatically hide faulty pictures or sequences, and be tracked over time for statistics. **Note** that reports management (beyond creation, which can be done through _web viewer_) is only available through HTTP API as now, no CLI or front-end is offered as now.
- You can create and manage _Excluded areas_ (areas where people should not upload pictures) through `/api/configuration/excluded_areas` and `/api/users/me/excluded_areas` routes. Excluded areas can be defined for everyone, or by user, for fine management. **Note** that this is only manageable through HTTP API, no CLI or front-end is offered as now.
- An user account can be marked with an `admin` role in database to enable some features (access to all reports, edit excluded areas).
- Support of cropped panorama, with new values in `pers:interior_orientation` in picture properties: `visible_area`, `sensor_array_dimensions` following [proposed definition in STAC](https://github.com/stac-extensions/perspective-imagery/issues/10).
- Routes `/api` and `/api/configuration` return API version in a `geovisio_version` attribute (example: `2.6.0-12-ab12cd34`).
- Routes returning collections display a `geovisio:upload-software` property showing which client has been used to create the collection (for meta-catalog statistics).
- handle user agent `GeoVisioCli` and `PanoramaxCli` as the same `cli` software.

### Changed

- Flask session has been set to `permanent`, thus the session cookie will have the lifetime defined by `FLASK_PERMANENT_SESSION_LIFETIME` (default to 7 days).
- Reject duplicates pictures (same md5). A configuration `API_ACCEPT_DUPLICATE` can make the instance accepts duplicates as before.
- The background jobs have been changed, it should be transparent for the administrator, but they now rely on the new `job_queue` table, and can handle more types of async jobs.
- More metadata can be set on API (name, description, logo and main color) in `API_SUMMARY` setting. This is served through `/api/configuration` and `/api` routes for client-side display.
- Maximum authorized distance between two consecutive pictures in a same segment of a sequence is now 75 meters (distance at 135km/h during two seconds). This only changes map rendering, to avoid sequences cuts over motorways.
- Sequences and Upload Sets creation now stores `User-Agent` HTTP requests headers.
- Minimal required PostGIS version is now 3.4 with PostgreSQL 12.
- Almost all database queries now have a statement timeout (default to 5mn).
- Updated Geopic Tag Reader to 1.3.0 to solve various issues related to pictures timestamps.
- In vector tiles style, `interpolate-hcl` expression for grid colouring has been replaced into `interpolate` for broader compatibility with QGIS versions.
- A new `geovisio:length_km` property is available on `/api/collections/:id`, `/api/users/:id/collection` and `/api/users/me/collection` route, giving the length of sequence in kilometers.
- ⚠️ The docker images are now `panoramax/api` instead of `geovisio/api`.

### Fixed

- RSS feed was producing invalid viewer links (missing coordinates) since sequences are represented as MultiLineString in database.
- Fix a bug where we could add pictures to an already deleted sequence.
- Fix a bug where the `pictures_grid` view was refreshed too frequently (and the computation can be expensive for the database).

### Removed

- The `test-api-conformance.sh` are now regular python tests
- Removed on-the-fly JPEG to WebP conversion (too slow). WebP might do an unexpected come-back in the future 😉
- The list of users has been removed from the API entrypoint (`/`) as the list is ever growning, but it can still be accessed through the `/users` endpoint.

Version 2.6.0

⚠️ Important Note: This version add several quite long migrations. After the migration are run, you should run as a database administrator:

```sql
VACUUM FULL pictures, sequences;
```

or use a tool like [pg_repack](https://github.com/reorg/pg_repack) to remove dead tuples and save lot's of space.

Note that the vacuum will hold an exclusive lock whereas pg_repack will not.

⚠️ Important Note ⚠️ : This new versions uses postgres [`session_replication_role`](https://www.postgresql.org/docs/current/runtime-config-client.html) for non blocking migrations. This means that the users used to connect to the database must either have superuser privilege, or if postgres version is >= 15, you can grant the permission to the user with:

```psql
GRANT SET ON PARAMETER session_replication_role TO you_user;
```

### Added

- API routes returning items embed original datetime with timezone information in a new property `datetimetz`.
- New routes offering [MapLibre Style JSON files](https://maplibre.org/maplibre-style-spec/) for each vector tile endpoint (`/api/map/style.json`, `/api/users/me/map/style.json` and `/api/users/:userId/map/style.json`). This will allow more flexibility in offered vector tiles zoom levels and features against clients. These routes are advertised in API landing page with `xyz-style` and `user-xyz-style` links.
- A new configuration `DB_CHECK_SCHEMA` to tell GeoVisio not to check the database schema on startup, use only if you know you'll not use GeoVisio before updating its schema.
- Pictures metadata now embed pitch & roll information (`pers:pitch` and `pers:roll` fields).
- A new configuration `PICTURE_PROCESS_REFRESH_CRON` tell the background workers when to refresh the database stats (they execute the `flask db refresh` command).

### Changed

- All sequences geometries have been updated to be split if pictures were too far apart (linked to change done in [this MR](https://gitlab.com/panoramax/server/api/-/merge_requests/244)).
- Vector tiles from zoom 0 to 5 now offers a grid of available pictures (instead of simplified sequences). They are computed on-demand, so you may want to run `flask db refresh` once a day to keep them up-to-date.
- Doc and links to match the Gitlab organization rename from GeoVisio to Panoramax.

### Fixed

- Migration to change LineString sequence geometry into MultiLinestring was not compatible with older PostGIS version, making Docker image not able to migrate populated sequences tables from 2.4 to 2.5.
- Route `/api/collections/:cid/geovisio_status` was returning non-empty list of items when no pictures was yet associated to the sequence.
- Improve `/api/collections/:cid/geovisio_status` performance for big sequences.
- Improve performance of big sequences deletion in database.

### Removed

- All binary exif fields are removed from the database. This should save quite a lot of database storage.

Release 2.5.0

### Added

- Picture search route (`/api/search`) now allows to look for a place that should be visible in pictures. This allows to find pictures for illustrating a POI page. This is done using `place_position`, `place_distance` and `place_fov_tolerance` parameters.
- Route `PATCH /api/collections/:id` has new parameters for editing in a single shot all pictures in a sequence:
  - `relative_heading`: to change all picture headings based on sequence movement track
  - `sortby`: to change the property used to sort pictures in the sequence (GPS date, file date or file name, either ascending or descending order).
- Route `PATCH /api/collections/:col_id/items/:id` has a new parameter for editing a picture's heading.
- Add 2 new tables `pictures_changes` and `sequences_changes` to track updates on `pictures` and `sequences`. Those tables are not yet exposed via an http API.

### Changed

- Updated Geopic Tag Reader to version 1.0.5
- Variable `DB_URL` is not set by default on the Dockerfile anymore, a real value must be given.
- Do not persist binary exif fields. The most notable one is `MakerNote` that took ~15% of the database storage.
- Pictures's are now displayed from zoom level 15 in the tiles to lower the tile's size.
- Collection geometries are now stored as multilinestrings, and split if pictures are too far apart.

### Fixed

- Pytest path in CI for Docker image release.
- When searching collections with a bounding box (in `/api/collections` or `/api/users/:id/collection`), a real intersection is done in the search (before, only the bounding box of the collection was considered).
- Permission problems in Dockerfile when writing to `/data/geovisio`

Release 2.4.0

# Release 2.3.1

### Added
- Basic `Cache-Control` headers, only setting `public` or `private` for the moment.
- `original_file:size` and `original_file:name` in an item's response's `properties`
- User collection list (`/api/users/:id/collection`) now also supports search with `bbox`, as proposed in [STAC _Collection Search_ extension](https://github.com/stac-api-extensions/collection-search).

### Changed
- Docker container work directory is now `/opt/geovisio`.
- Docker compose file with blurring is lighter, to for easier maintenance.
- Update [geo-picture-tag-reader](https://gitlab.com/geovisio/geo-picture-tag-reader) to [1.0.3 version](https://gitlab.com/geovisio/geo-picture-tag-reader/-/tags/1.0.3).

### Removed
- Tests in Docker are removed, tests can be run locally and are automatically run through repository CI.
- `docker-compose-auth.yml` file, as it is now redundant with `docker-compose-full.yml` (as blurring is separated in a lighter compose file).

Release 2.3.0

+++ Added
- Performance and crash metrics can be sent now to a Sentry server.
- (Almost) full list of picture EXIF tags are shown in `properties.exif` field in API routes returning STAC items. Some keys are skipped because of their low added-value and wide size (maker notes, long binary fields).
- On route `POST /api/collections/:cid/items`, new `override_Exif.*` and `override_Xmp.*` parameters are available to manually define EXIF or XMP tags to override in picture metadata. You can pass any valid [Exiv2](https://exiv2.org/metadata.html) EXIF/XMP tag in query parameters.
- A documentation about [STAC API and GeoVisio API differences](./docs/80_STAC_Compatibility.md).
- Sequences can be filtered by their last update time with `GET /api/collections?filter=...` parameter (uses _Common Query Language_ expression). Deleted sequences can also be show using the `filter` `status` (`filter=status='deleted'`).
- Picture processing will now be retried (10 times) in case the blurring API fails.
- Sequence title can be updated through `PATCH /api/collections/:id` route.

+++ Changed
- Picture EXIF tags are now stored in database following the [Exiv2](https://exiv2.org/metadata.html) naming scheme. A database migration that could take some time (up to half an hour) is offered to update existing metadata in running instances.
- Upgrade Swagger to 5.9
- Fix a deadlock in the database when a picture is deleted while a worker is preparing it.
- Sequences last updated date corresponds to either last edit of sequence itself or any pictures it embeds.
- Surrounding pictures are listed in details of a single picture (`GET /api/collections/:collectionId/items/:itemId`), as `rel=related` links.
- User detailed catalog (`/api/users/:userId/collection`) now offers paginated and filterable results for its `child` links. Query parameter are `filter, limit, sortby`.
- Tag reader library update to 1.0.2 to fix various fractions value issues.
- `API_MAIN_PAGE` and `API_VIEWER_PAGE` environment variables can now also take a full URL to use instead of default API front-end pages. Useful if you're using a third-party front-end to keep RSS links consistent.

+++ Fixed
- Tag reader dependency was incorrectly evaluated in pyproject.

+++ Deprecated
- Parameters `created_before` and `created_after` in `GET /api/collections` route, in favor of `filter` parameter.

Release 2.2.0

+++ Changed
- [GeoPic Tag Reader](https://gitlab.com/geovisio/geo-picture-tag-reader) updated to 0.4.1 to embed stronger checks on picture coordinates.

+++ Fixed
- If a picture was having invalid coordinates in its EXIF tags, geometry in database was landing outside of WGS84 bounding box, and `GET /api` returned an invalid spatial extent. API now limits returned bounding box to maximum authorized value for WGS84 projection.

+++ Added
- a new route: `/api/users/:id/collection` that returns a collection of all the users's collections (can also be accessed with `/api/users/me/collection` with authentication). It's similar to `/api/users/:id/catalog` but with more metadata since a STAC collection is an enhanced STAC catalog.

Release 2.1.1

+++ Added
- On picture upload, some metadata can be passed through HTTP request parameters instead of being read from picture file EXIF tags. Available metadata overrides are: GPS coordinates, capture time and picture type. This allows API clients to handle a wider set of input files (for example GeoPackage, CSV, Shapefile...) without needing to insert all information into picture file.
- To make API compatible with a broader range of clients, the `GET /api/collections/{collectionId}/items` route has new metadata in its `properties` field (`geovisio:producer`, `geovisio:image`, `geovisio:thumbnail`). These properties are duplicated regarding STAC standard (which puts them directly at _Feature_ level) to allow compatibility with clients which only reads metadata from `properties` field (like uMap or QGIS).
- A favicon is shown in default API pages.
- A RSS feed is now offered to list recently uploaded collections, it can be accessed through `GET /api/collections?format=rss` (or with `Accept: application/rss+xml` HTTP request header).
- Collections list (`/api/collections`) now also supports search with `bbox` and `datetime` parameters, as proposed in [STAC _Collection Search_ extension](https://github.com/stac-api-extensions/collection-search).

+++ Changed
- The Docker compose file `docker-compose-full.yml` now embeds [GeoVisio Website](https://gitlab.com/geovisio/website), available on `localhost:3000`.

+++ Fixed
- Database migration `20230720_01_EyQ0e-sequences-summary` was having a failing SQL request, causing invalid computed sequence metadata being present in database.
- Search parameters `collections` and `ids` for `/api/search` route where not correctly handled when passed through `POST` JSON body.

Release 2.1.0

+++ Added
- A way to customize the picture's license. If none is set, the pictures's license is considered to be proprietary.
- A new route `PATCH /api/collections/:cid` is offered to change visibility of a sequence
- A way to call the vector tiles as an authenticated user. It is mainly used to be able to see objects only visible for this user.
- A `hidden` property in the vector tiles, to mark a sequence or picture as only visible for the owner of this sequence or picture. If the property is not set, the object is visible by all
- A new route `DELETE /api/collections/:cid` is offered to delete a collection. The deletion is done asynchronously.
- A new route `DELETE /api/collections/:cid/items/:id` is offered to delete a picture
- [OpenAPI](https://swagger.io/specification/) conformance tests are now automatically run through `tests/test_api_conformance.sh` script
- [Support of pagination](https://github.com/radiantearth/stac-api-spec/tree/main/ogcapi-features#item-pagination) for `GET /api/collections/:cid/items` API route (`first, last, prev, next` relation types)
  * also support a `?withPicture=:picture_id` query paramater to ask for a page with a specific picture in it
- Add a `flask sequences reorder` subcommand to reorder all or some sequences using the picture's datetime.
- [Support of pagination](https://github.com/radiantearth/stac-api-spec/blob/master/ogcapi-features/README.md#collection-pagination) for `GET /api/collections` API route (`first, last, prev, next` relation types), with a default limit to 100 sequences retrieved per call
- Add 2 optional parameters to the `GET /api/collections` api: `created_after`/`created_before` used to filter the collection by their creation date.

+++ Changed
- Docker compose files now use `latest` API image instead of `develop`
- In default pages (`/` and `/viewer.html`), web viewer version is now synced to current API version instead of develop
- Algorithm used for generating smaller versions of pictures changed from _NEAREST_ to _HAMMING_ for better results
- API documentation and specifications moved to `/api/docs/swagger` and `/api/docs/specs.json`, and with improved readability of their content
- Now heading are recomputed if set to 0 because some camera use this value by default
- Hidden pictures will now always be served through the API to be able to check permissions.

+++ Fixed
- Raw picture bytes are sent to blurring API instead of Pillow-based version, avoiding various issues (too large files, missing EXIF)
- Docker-compose files are now compatible with MacOS (replaced `network_mode: host` to use a more classic approach)
- Some CORS HTTP headers were missing in API responses to allow client send credentials

Release 2.0.2

Release 2.0.1

Release 2.0.0

Release 1.5.0

Release 1.4.1

Release 1.4.0

Release 1.3.1