NLP pipeline integration to the framework

This task encompasses the integration of our NLP tools into the Framework as internal and external services.

Our tools

  • jtf-lib
    • atf checker
    • clean transit
    • sign names
  • atf2CoNLL converter
  • Morphology pre-annotation tool
    • CDLI-Conll Checker (probably in the morphology pre-annotator)
  • CDLI-CoNLL to CoNLL-U converter
  • CoNLL-U to Brat standalone converter
  • Brat standalone to CDLI-CoNLL converter
  • Conll2rdf
  • Conllmerge
  • metadata converter (CSV to TTL-RDF)
  • A Dependency parser

Set up the tools inside the framework

  • tools must be made available inside the framework network
  • Route each tool using Nginx rules, eg. /tools/atf2conll → node-tools:3032 (dev/conf/nginx.conf)
  • Document those URLs; what data can go in and what comes out in https://gitlab.com/cdli/framework/-/blob/phoenix/develop/dev/docs/API.md (also add commodity API and jtf-lib API (see here !373 (merged)) and !419 (comment 731297994))
Edited Sep 28, 2022 by Lars Willighagen
Assignee Loading
Time tracking Loading