Detection API / mechanism for easy run of detection/classification on available pictures

In the goal of having sign detection on the pictures, and to make it possible for all kind of detections to be done and shared, I think it's nice if we lay out the plan.

Now that we have the ability to add key/value semantics on the pictures and on part of the pictures, maybe we can have the following plan:

Store the blurring detections in the API

In order to minimize the resources needed, the blurring API is also trained to detect road signs and return the detections as a JPEG comment and in the x-sgblur heading.

Those detections could be stored in the database and translated as key/value semantic (associated to the default account I guess? or a custom created account?)

An issue will need to be created to discuss the semantic used (a small part of #242 (closed) ).

Doing so won't need to leak ids in the blurring API (we don't want to do this due to privacy questions), and will isolate the blurring/detection process at the cost of a small strange thing (the picture-workers would add themselves the result of another model).

Implement an external service to classify the detections

For the moment, the existing detection model is run manually on part of images stored by the blurring/detection API, we'll need to industrialize things more here.

I think could add an external service, that could be easily extended by whoever wants to automatize detections and share the results.

The service could poll for new images with some filtering, like pictures from a given account, a specific regions, and in our first case pictures with specific key/value semantics (the one added on part of the pictures, telling us there are road_sign to classify).

This service would use the same crawling mechanism as the meta-catalog, and poll regularly the instance asking for everything that changed since its last crawl (but it would do so via the /search endpoint, so we'll need to extend it I guess).

The service would store the stuff to classify in a queue (to be able to be restarted easily and to have multiple classifiers if we manage to have several GPUs for this), and after each classification, it would store the results in a local database for the one who ran the model and would also publish the classification results as new key/value semantics (on the same annotation for the classification model, but some model could create another annotation if for example a better shape of the detected object is computed).

Note that for the detection and classification models, we should also add semantics about the model and its version, to be able to update results if needed.

Add a way to search pictures with specific tags

We'll need to have a way to find all the pictures with a specific key/values. Filtering on the defined key/values for road sign might lead to classify more than intended since every one could add tags like those, but I find it a nice feature since it would mean that everyone could manually add road sign annotation on a picture and those would be classified.

linked to #240 (closed)

Add pagination on /search

To be able to get all results since there could easily be more than the max limit (10 000 pictures), we'll need to handle pagination on the /search endpoint

Conclusion

Questions

• I would have preferred if the external service was crawling the meta catalog, to be able to do cross instance detection/classification, but then the added annotation have to be do on each panoramax instance with an account on the said instance, so maybe it would be to complex to handle this for the moment ? Any thoughts on this ?

Is this plan ok for you @PanierAvide @cquest ?

If so, I'll create all the underlying issues.