CoNLL-U and CoNLL-RDF export

Fixes Partially addresses #126 (closed) Partially addresses #242 (closed)

Description Cleans up some of the view traits used for exporting, and implements views for CoNLL-U and CoNLL-RDF.

Type of PR This PR is a feature.

Technicalities

• adds CoNLL-U and CoNLL-RDF converters to the Node.JS service
  • Note: not optimal, as Node.JS has to communicate through the CLI. However, the servers could easily be replaced by Python and Java servers respectively, in this container or another, without having to change the PHP end of things.
• adds a CLI for CoNLL-U that supports stdin/stdout
• adds caching for CoNLL-U and CoNLL-RDF conversions (!)

Tests

• Run curl -LH 'Accept: text/x-conll-u' http://127.0.0.1:2354/inscriptions/2341997
• Run curl -LH 'Accept: text/x-conll+turtle' http://127.0.0.1:2354/inscriptions/2341997

Checklist:

• My pull request has a descriptive title (not a vague title like "Update index.md").
• My pull request targets the phoenix/develop branch of the repository.
• My commit messages follow best practices.
• My code follows the established code style of the repository.
• I added tests for the changes I made (if applicable).
• I added or updated documentation (if applicable).
• I tried running the project locally and verified that there are no visible errors.

marked the checklist item My pull request has a descriptive title (not a vague title like "Update index.md"). as completed

marked the checklist item My pull request targets the phoenix/develop branch of the repository. as completed

marked the checklist item My commit messages follow best practices. as completed

marked the checklist item My code follows the established code style of the repository. as completed

marked the checklist item I tried running the project locally and verified that there are no visible errors. as completed

added 1 commit

• dadf4849 - Update API (dev) documentation

Compare with previous version

marked the checklist item I added or updated documentation (if applicable). as completed

What would be the url to get it from the artifact id?

Same as with the rest of inscription APIs, i.e.:

• curl -LH 'Accept: text/x-conll-u' http://127.0.0.1:2354/artifacts/100188
• curl -LH 'Accept: text/x-conll+turtle' http://127.0.0.1:2354/artifacts/100188

However, is_latest needs to be set. Also, forgot to mention: CoNLL-RDF breaks on the CDLI-CoNLL of P100149 because the word(?) identifiers contain single quotes, which doesn't work with RDF URIs.

cURL Error (6) Could not resolve host: node-tools

Yes, sorry. The node-tools docker service gets disabled every now and then to fix the CI. You should be able to run it now but the CI is broken.

added 1 commit

• 70c61415 - Enable the node-tools service again

Compare with previous version

So the conllu works (It's beautiful ;D) but for the ttl I get a curl error empty reply

On P100149? I think CoNLL-RDF breaks on the CDLI-CoNLL of P100149 because the word(?) identifiers contain single quotes, which doesn't work with RDF URIs.

Oh yes ok sorry. How to we fix that though, can we escape the quote or something? I'll merge this now.

Maybe. I can open an issue at the CoNLL-RDF repository, if the CoNLL file is okay to share?

Yes ! The CDLI-CoNLL files are actually public domain :) https://github.com/cdli-gh/mtaac_gold_corpus (They are stored in the to_dict folder)

Ah, good. Seems like I'm not the first one though: https://github.com/acoli-repo/conll-rdf/issues/11#issuecomment-505987456. Apparently the ID column should contain numbers only (except perhaps hyphens?). I can rename the ID column though (CDLI_ID?), so it treats it as a regular column and generates new IDs automatically.

:s1_51  a              nif:Word ;
        nif:nextWord   :s1_52 ;
        conll:CDLI_ID  "r.7'.2" ;
        conll:FORM     "sila3" ;
        conll:SEGM     "sila[unit]" ;
        conll:XPOSTAG  "N" .

instead of the invalid

:s1_r.7'.2 a           nif:Word ;
        nif:nextWord   :s1_r.7'.3 ;
        conll:ID       "r.7'.2" ;
        conll:FORM     "sila3" ;
        conll:SEGM     "sila[unit]" ;
        conll:XPOSTAG  "N" .

but is it a restriction in the rdf format that was implemented in the software or a restriction based on usage in the software itself? In other words, why is conll-rdf expecting a number for the id? @max-ionov

Just had a memory, i think strict conll-u only take numbers. If you check the cdli-conll to conllu converter you might be able to verify that. If it's the case, then yes we should add a numerical id as the main id.

Should we add a numerical id (i.e. in the CDLI-CoNLL) or just let CoNLL-RDF generate numeric IDs like I mentioned above?

Only in the Conll-RDF

added 1 commit

• f8f17fea - Support :format as a URL parameter

Compare with previous version

Note: I disabled the service again to fix the CI. You can enable it locally (see !177 (70c61415)).

mentioned in issue #216 (closed)

added 1 commit

• 2c0dff67 - Add P000000.conll download file names

Compare with previous version

approved this merge request

mentioned in commit cfd99f7f

merged