Commit a3da62f6 authored by Open Science Conservation Fund's avatar Open Science Conservation Fund
Browse files

updated docs; fix for readthedocs build

parent bf5a0b07
Loading
Loading
Loading
Loading

docs/.readthedocs.yml

0 → 100644
+33 −0
Original line number Diff line number Diff line
# Read the Docs configuration file for TRAPPER project
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

version: 2

# Set the OS, Python version and other tools you might need
build:
  os: ubuntu-22.04
  tools:
    python: "3.11"
  jobs:
    post_checkout:
      # Install system dependencies if needed
      - echo "Build started for TRAPPER documentation"

# Build documentation in the docs/ directory with Sphinx
sphinx:
  configuration: docs/conf.py
  builder: html
  fail_on_warning: false

# Optionally declare the Python requirements required to build your docs
# python:
#   install:
#     - requirements: docs/requirements.txt
#     - method: pip
#       path: .
#       extra_requirements:
#         - docs

# Optional but recommended, declare the Python requirements required to build your docs
formats:
  - pdf
+320 −17
Original line number Diff line number Diff line
@@ -4,10 +4,10 @@
Administration
==============

The Admin site is available in the top menu and it is only visible for users with admin permissions.
.. warning:: 🚧 Work in progress

.. image:: images/administration/top_bar.png
    :alt: top menu with Admin site button
  This section is partly outdated and currently under active development and may change without notice. Content,
  examples, and screenshots may be incomplete or updated soon.

Admin actions for User model
++++++++++++++++++++++++++++
@@ -68,31 +68,334 @@ In the form, choose classes to import taxa from and submit.
.. note::
    Species import is run asynchronously, so the imported species may not be visible right away after running the action.

Add and configure AIProvider
++++++++++++++++++++++++++++
Quick TrapperAI Provider Setup
+++++++++++++++++++++++++++++

To be able to use AI methods for classification, you need to add an *AIProvider* (for example Megadetector)
To be able to use AI methods for classification, you need to add a *TrapperAI Provider* 
to your *Research project*.

*AIProvider* needs to be manually added and configured via admin panel first.
*TrapperAI Provider* needs to be manually added and configured via admin panel first.

Go to admin panel -> *Media Classification* -> *AIProvider* -> *+ Add ai provider*.
Go to admin panel  *Media Classification*  *AI Provider*  *+ Add ai provider*.

.. image:: images/administration/ai_provider_type.png
    :alt: ai provider form part one

In the first part if the form, choose type of *AIProvider* to add.
Regardless of the chosen type, several fields are required to fill:
In the first part of the form, choose **TrapperAI Provider** type.
Several fields are required to configure:

- name for your *AIProvider* instance
- minimum confidence: default value is 0.9, meaning that predictions from AI with confidence lower than that will not be saved
- trapper instance url: url of your Trapper server
- API url: url of AI API
- **Name**: Descriptive name for your *TrapperAI Provider* instance
- **API URL**: URL of TrapperAI API endpoint
- **API Auth Login/Password**: Authentication credentials
- **Minimum Confidence**: Default value is 0.9, meaning predictions below this confidence will not be saved

Last required part of *AIProvider* configuration is adding mappings.
The most important part of *TrapperAI Provider* configuration is adding mappings.

.. image:: images/administration/ai_provider_mappings.png
    :alt: ai provider form part two

The mappings are used to translate AI classification results to Trapper classification attributes - add the values
returned by your chosen AI API and map them to attribute values.
The mappings translate AI classification results to TRAPPER classification attributes - add the values
returned by your TrapperAI API and map them to appropriate attribute values.

.. seealso::
   For comprehensive TrapperAI Provider configuration with detailed examples, see the following section.

TrapperAI Provider Configuration
++++++++++++++++++++++++++++++

TRAPPER provides sophisticated artificial intelligence integration through **TrapperAI Providers** 
that leverage machine learning models for automated camera trap image analysis. This section covers 
the complete configuration and administration of AI providers and their integration with Classification Projects.

What is TrapperAI Provider?
===========================

**TrapperAI Provider** is the primary AI service integration in TRAPPER that provides automated image analysis 
capabilities through local or remote AI models. It supports:

- **Object Detection**: Identifying and locating animals, humans, and vehicles in images using MegaDetector models
- **Species Classification**: Classifying detected animals to species level using specialized classifiers like DeepFaune
- **Confidence-based Processing**: Filtering results based on configurable confidence thresholds
- **Flexible Mapping**: Translating AI model outputs to TRAPPER classification attributes

Creating TrapperAI Provider in Admin Interface
==============================================

**Basic Configuration**

1. Navigate to admin interface (*Admin* > *Media Classification* > *AI Providers*)
2. Click *Add AI Provider* and select **TrapperAI Provider**
3. Configure basic settings:

   - **Name**: Descriptive name (e.g., "MegaDetector v5 Object Detection", "DeepFaune v1.3 Species Classifier")
   - **Version**: Model version identifier (e.g., "5a", "1.3")
   - **Description**: Detailed description of the model capabilities
   - **API URL**: Endpoint URL for the TrapperAI service
   - **API Auth Login/Password**: Authentication credentials for the AI service
   - **AI Model ID**: Unique identifier of the model in TrapperAI system

**Advanced Settings**

Configure specialized options:

- **Video Support**: Enable for models that can process video files
- **Object Based**: Enable for object detection models (usually true)
- **Crop Image**: Enable for species classifiers that require cropped images
- **Minimum Confidence**: Global threshold for discarding low-confidence predictions (0.0-1.0)
- **Skip Empty**: Discard resources without detected observations
- **Trapper Instance URL**: Base URL of your TRAPPER server for callbacks

**AI Model Mappings**

Configure how AI model outputs map to TRAPPER attributes:

**Object Detection Mapping Example (MegaDetector v5)**

.. code-block:: javascript

   // Observation Type Labels Mapping
   [
     {
       "value": 1,
       "observation_type": "animal"
     },
     {
       "value": 2, 
       "observation_type": "human"
     },
     {
       "value": 3,
       "observation_type": "vehicle"
     }
   ]

**Species Classification Mapping Example (DeepFaune v1.3)**

.. code-block:: javascript

   // Species Labels Mapping (excerpt)
   [
     {
       "value": 0,
       "species": "Meles meles"          // European badger
     },
     {
       "value": 3,
       "species": "Cervus elaphus"       // Red deer
     },
     {
       "value": 7,
       "species": "Capreolus capreolus"  // Roe deer
     },
     {
       "value": 15,
       "species": "Lynx lynx"            // Eurasian lynx
     },
     {
       "value": 18,
       "species": "Lepus europaeus"      // European hare
     },
     {
       "value": 21,
       "species": "Mustela erminea"      // Stoat
     },
     {
       "value": 24,
       "species": "Sus scrofa"           // Wild boar
     },
     {
       "value": 29,
       "species": "Vulpes vulpes"        // Red fox
     }
   ]

.. _administration-classification-project-ai-configuration:

Classification Project AI Configuration
======================================

**AI Model Assignment**

Configure AI models for automatic processing in Classification Projects:

1. Navigate to *Classification Projects* in admin interface
2. Edit your target classification project  
3. In the **AI Configuration** section, configure:

   - **Object Detection AI Model**: Select TrapperAI Provider configured for object detection (e.g., MegaDetector)
   - **Species AI Model**: Select TrapperAI Provider configured for species classification (e.g., DeepFaune)

**AI Processing Settings**

Configure how AI results are processed and integrated:

- **Required AI**: Require AI classification before human annotation can begin
- **Copy AI Classifications**: Automatically copy AI results as initial user classifications
- **Species Matching IOU Threshold**: Intersection over Union threshold for matching species predictions to detections (default: 0.5)

**Confidence Thresholds and Quality Control**

Set up warning thresholds for quality control:

- **Observation Type Confidence Warning Threshold**: Show warnings when object detection confidence is below this value
- **Species Confidence Warning Threshold**: Show warnings when species classification confidence is below this value

**Privacy Protection Settings**

Configure automatic blurring for privacy protection:

- **Blur Humans**: Automatically blur detected human faces and figures
- **Blur Vehicles**: Automatically blur detected vehicles and license plates
- **Blur Backup**: Keep backup copies of original images before applying blur
- **Blur Humans and Vehicles Immediately**: Apply blurring immediately after classification (irreversible)

**Video Processing Configuration**

For projects supporting video analysis:

- **Video Support Enabled**: Enable video file processing
- **Target FPS**: Set target frame rate for video analysis (optional, uses source FPS if not set)

**Additional Project Settings**

Control annotation behavior and data visibility:

- **Exclude Humans**: Hide human observations from citizen science interface
- **Exclude Blank**: Hide blank/empty observations from citizen science interface
- **Hide Classification Attributes for Non-Animals**: Simplify interface by hiding species attributes for non-animal observations

Complete Configuration Example
=============================

**Example 1: Object Detection Setup (MegaDetector v5)**

.. code-block:: text

   TrapperAI Provider Configuration:
   ├── Name: "MegaDetector v5 Object Detection"
   ├── Version: "5a"
   ├── Description: "YOLOv5-based model for detecting animals, people, and vehicles"
   ├── API URL: "https://your-trapper-ai.org/api/v1/detect"
   ├── Minimum Confidence: 0.7
   ├── Object Based: ✓ True
   ├── Crop Image: ✗ False
   └── Mappings:
       ├── Value 1 → "animal"
       ├── Value 2 → "human"  
       └── Value 3 → "vehicle"

   Classification Project Settings:
   ├── Object Detection AI Model: "MegaDetector v5 Object Detection"
   ├── Required AI: ✓ True
   ├── Copy AI Classifications: ✓ True
   ├── Observation Type Confidence Warning: 0.6
   ├── Blur Humans: ✓ True
   └── Blur Vehicles: ✓ True

**Example 2: Species Classification Setup (DeepFaune v1.3)**

.. code-block:: text

   TrapperAI Provider Configuration:
   ├── Name: "DeepFaune v1.3 European Mammals"
   ├── Version: "1.3"
   ├── Description: "Classifier for 34 European mammal species"
   ├── API URL: "https://your-trapper-ai.org/api/v1/classify"
   ├── Minimum Confidence: 0.8
   ├── Object Based: ✓ True
   ├── Crop Image: ✓ True
   └── Mappings:
       ├── Value 0 → "Meles meles"
       ├── Value 3 → "Cervus elaphus"
       ├── Value 7 → "Capreolus capreolus"
       ├── Value 15 → "Lynx lynx"
       └── [... 30 more species mappings]

   Classification Project Settings:
   ├── Species AI Model: "DeepFaune v1.3 European Mammals"
   ├── Species Matching IOU Threshold: 0.5
   ├── Species Confidence Warning: 0.7
   └── Hide Classification Attributes for Non-Animals: ✓ True

Monitoring AI Provider Performance
==================================

**Performance Metrics**

Track AI provider performance through admin interface:

- **Processing Statistics**: Number of images processed, success rates, processing times
- **Accuracy Metrics**: Comparison of AI predictions with human review results
- **Confidence Analysis**: Distribution of confidence scores across classifications
- **Error Reports**: Detailed logs of processing failures and issues

**Quality Assurance**

Implement quality control measures:

- **Regular Model Updates**: Keep AI models current with latest training data
- **Accuracy Monitoring**: Track how often AI classifications are accepted by reviewers
- **Bias Detection**: Monitor for systematic biases in AI classifications
- **Performance Alerts**: Receive notifications when performance metrics drop

**Usage Analytics**

Monitor AI usage and resource consumption:

- **API Call Tracking**: Monitor API usage and costs
- **Processing Queue**: Track processing backlogs and queue times
- **Resource Utilization**: Monitor computational resource usage
- **User Feedback**: Collect feedback from users on AI classification quality

Troubleshooting AI Integration
==============================

**Common Configuration Issues**

- **API Connection Errors**: Verify API URLs and authentication credentials
- **Mapping Problems**: Ensure AI output values are properly mapped to TRAPPER attributes
- **Performance Issues**: Check confidence thresholds and processing queue status
- **Authentication Failures**: Verify API keys and access permissions

**Error Resolution**

Steps for resolving AI processing errors:

1. Check AI Provider configuration in admin interface
2. Verify API service availability and authentication
3. Review processing logs for specific error messages
4. Test with individual images to isolate issues
5. Contact AI service provider for service-specific problems

**Performance Optimization**

Optimize AI processing performance:

- **Batch Processing**: Process multiple images together when possible
- **Confidence Tuning**: Adjust thresholds based on accuracy analysis
- **Queue Management**: Monitor and manage processing queues
- **Resource Scaling**: Scale AI services based on processing demand

Best Practices for AI Configuration
===================================

**Initial Setup**

- **Start Simple**: Begin with object detection before adding species classification
- **Test Thoroughly**: Validate AI performance on representative sample data
- **Set Conservative Thresholds**: Start with higher confidence thresholds and adjust based on results
- **Monitor Continuously**: Regularly review AI accuracy and adjust settings as needed

**Production Deployment**

- **Gradual Rollout**: Deploy AI processing gradually across projects
- **Human Oversight**: Maintain human review processes for critical classifications
- **Regular Updates**: Keep AI models and configurations current
- **Documentation**: Document all AI provider configurations and changes

**Quality Management**

- **Validation Protocols**: Establish clear protocols for validating AI classifications
- **Training Integration**: Use AI results to train new annotators
- **Consistency Checks**: Ensure AI classifications align with project standards
- **Continuous Improvement**: Use feedback to improve AI model performance
+11 −13
Original line number Diff line number Diff line
# -*- coding: utf-8 -*-
#
# Configuration file for the Sphinx documentation builder.
#
# This file does only contain a selection of the most common options. For a
@@ -19,14 +17,14 @@

# -- Project information -----------------------------------------------------

project = u"trapper-project"
copyright = u"2020, Open Science Conservation Fund (OSCF)"
author = u"Open Science Conservation Fund (OSCF)"
project = "trapper-project"
copyright = "2025, Open Science Conservation Fund (OSCF)"
author = "Open Science Conservation Fund (OSCF)"

# The short X.Y version
version = u""
version = "1.7.2"
# The full version, including alpha/beta/rc tags
release = u"1.1.0"
release = "1.7.2"


# -- General configuration ---------------------------------------------------
@@ -64,7 +62,7 @@ language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = [u"_build", "Thumbs.db", ".DS_Store"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = None
@@ -75,7 +73,7 @@ pygments_style = None
# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
#
# html_theme = 'alabaster'
html_theme = "sphinx_rtd_theme"

# Theme options are theme-specific and customize the look and feel of a theme
# further.  For a list of options available for each theme, see the
@@ -129,8 +127,8 @@ latex_documents = [
    (
        master_doc,
        "trapper-project.tex",
        u"trapper-project Documentation",
        u"Open Science Conservation Fund (OSCF)",
        "trapper-project Documentation",
        "Open Science Conservation Fund (OSCF)",
        "manual",
    ),
]
@@ -141,7 +139,7 @@ latex_documents = [
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
    (master_doc, "trapper-project", u"trapper-project Documentation", [author], 1)
    (master_doc, "trapper-project", "trapper-project Documentation", [author], 1)
]


@@ -154,7 +152,7 @@ texinfo_documents = [
    (
        master_doc,
        "trapper-project",
        u"trapper-project Documentation",
        "trapper-project Documentation",
        author,
        "trapper-project",
        "One line description of project.",
+120 −9
Original line number Diff line number Diff line
@@ -4,6 +4,11 @@
Configuration
=============

.. warning:: 🚧 Work in progress

  This section is partly outdated and currently under active development and may change without notice. Content,
  examples, and screenshots may be incomplete or updated soon.

You can override many default settings in TRAPPER by adjusting the environmental variables listed
in the file `.env`. Further in this section we provide some examples of how to customize selected
aspects of TRAPPER.
@@ -261,17 +266,123 @@ If you are a developer of TRAPPER you can turn on a very cool tool for debugging
   DEBUG_TOOLBAR_USERS=admin1,admin2


Advanced system-level variables
+++++++++++++++++++++++++++++++
AI Pipeline Configuration
+++++++++++++++++++++++++

TRAPPER features a sophisticated **automatic AI pipeline** that processes uploaded camera trap 
images through multiple stages of machine learning analysis. The AI pipeline is configured 
primarily through **Classification Project** settings in the Django admin interface, rather 
than environment variables.

Pipeline Architecture
=====================

The AI pipeline consists of four sequential stages:

1. **Object Detection**: Identify and locate animals, humans, and vehicles in images
2. **Species Classification**: Classify detected animals to species level  
3. **Sequence Building**: Group related images into meaningful temporal sequences
4. **Privacy Blurring**: Automatically blur human faces and vehicle license plates

Configuration is managed through the **Classification Project** model in the admin interface.

.. seealso::
   For detailed AI Provider setup instructions, see :ref:`administration` → AI Provider and 
   Classification Project Configuration.

Classification Project AI Settings
==================================

AI pipeline behavior is configured through Classification Project settings in the Django admin interface:

**Object Detection and Species Classification**

Configure AI models for your Classification Project:

- **Object Detection AI Model**: Select TrapperAIProvider with MegaDetector (v5 or v6 variants)
- **Species AI Model**: Select TrapperAIProvider with species classifier (e.g., DeepFaune v1.3)
- **Copy AI Classifications**: Copy AI results to user classifications for review
- **Required AI**: Require AI classification before human annotation

**Confidence and Quality Control**

Set confidence thresholds and quality control parameters:

- **Species Matching IOU Threshold**: Overlap threshold for matching species predictions (default: 0.5)
- **Observation Type Confidence Warning Threshold**: Warn when object detection confidence is low
- **Species Confidence Warning Threshold**: Warn when species classification confidence is low

**Privacy and Blurring Options**

Configure automatic privacy protection:

- **Blur Humans**: Automatically blur detected human faces
- **Blur Vehicles**: Automatically blur detected vehicles and license plates  
- **Blur Backup**: Keep backup copies of original images before blurring
- **Blur Humans and Vehicles Immediately**: Apply blurring immediately after classification

**Video Processing**

These variables are set automatically when starting TRAPPER with the script ``start.sh``. You can override them, but please keep
defaults when you are not really sure what you can change here.
Configure video-specific settings:

1) Set UID & GID for a user that will be used to run TRAPPER `Gunicorn <https://gunicorn.org/>`_ server within a docker container
   (by default these values are taken from a user starting docker containers using the ``start.sh`` script). The example custom
   configuration:
- **Video Support Enabled**: Enable video processing for the project
- **Target FPS**: Set target frame rate for video processing (optional)

Deployment Upload Options
=========================

**Large Dataset Uploads (trapper-tools)**

For large-scale deployments with thousands of images:

.. code-block:: bash

      TRAPPER_USER_UID=1000
      TRAPPER_USER_GID=1000
   # Install trapper-tools
   pip install trapper-tools
   
   # Configure connection
   trapper-tools configure --url https://your-trapper-instance.org
   
   # Process and upload with AI pipeline
   trapper-tools pipeline --data-path /path/to/data --convert --trigger

**Single Deployment Uploads (trapper-frontend)**

For smaller deployments or user-friendly uploads:

- Use the web-based upload interface in trapper-frontend
- Drag-and-drop image uploads with automatic AI processing
- Real-time progress tracking and results preview
- Suitable for citizen science projects and smaller datasets
- Real-time progress tracking and results preview
- Suitable for citizen science projects and smaller datasets

General Environment Settings
============================

While AI pipeline configuration is primarily managed through the admin interface, 
some general system settings can be configured via environment variables:

**Basic System Settings**

.. code-block:: bash

   # Django settings for AI processing
   DJANGO_SETTINGS_MODULE=trapper.settings.production
   
   # Celery configuration for background AI tasks
   CELERY_BROKER_URL=redis://localhost:6379/0
   CELERY_RESULT_BACKEND=redis://localhost:6379/0
   
   # File storage for AI processing
   MEDIA_ROOT=/storage/media
   STATIC_ROOT=/storage/static

.. note::
   AI pipeline configuration requires careful tuning based on your specific use case, 
   available computational resources, and quality requirements. Start with conservative 
   settings and adjust based on performance monitoring results.

.. warning::
   Enabling AI processing will increase computational and storage requirements. Ensure 
   adequate resources are available and monitor system performance regularly.

docs/cs_frontend.rst

0 → 100644
+348 −0

File added.

Preview size limit exceeded, changes collapsed.

Loading