Generic dictionary encoder
We have several encoders now, but they are all dictionary-based, and some functionality could (and should) be shared among them. That way, only the dictionaries need to be changed.
- Refactor code common to all encoders to a generic "dictionary encoder"
Refactor current encoders to
importthe dictionary encoder and export a specific instance / decorator of it.
- Improve regexps by concatenating dictionary terms into it, instead of assuming latin alphabet. The regexp optimizers will do the rest.