Changelog ========= release/v0.12 (2020-07-06) -------------------------- Highlights ~~~~~~~~~~ - rename PyVallex to Vallex Tools - upload the pypi package so that it is now installable directly from PyPi instead of just from a whl file - a new way for installing on windows: we provide a simple powershell script which installs python and then installs vallex-tools using pip - pyinstaller based executables are no longer built during CI, eventually the code to generate them will probably be removed - updater functionality, which is tied to pyinstaller, is deprecated and will be replaced by `pip upgrade` - simplified production deployment Bugfixes ~~~~~~~~ - Display lexemes on loading (fixes Issue `#86 `_) [Jonathan L. Verner] - Do not fail if attribute source does not contain a newline (see Issue `#88 `_) [Jonathan L. Verner] Core ~~~~ - Substantial work on valdiff [Anša Vernerová] The valdiff attribute may be generated by running: ./vallex-cli --no-sort --output-format txt -o ../aktualni_data/data-txt/ scripts transform vallex/scripts/transforms/add_valdiff.py and then removing the changes created in the v-vallex file - Decide how to compute valdiff in case of multiple source verbs - Prefer no change in obligatoriness in case the two source verbs do not agree - Handle Forms of common and added functors. - Handle Common functors. new: verb1 verb2 noun valdiff no yes yes = already was: verb1 verb2 noun valdiff yes yes yes = yes no yes = - Handle Additional removed slots. new: verb1 verb2 noun valdiff no yes no - already was: verb1 verb2 noun valdiff yes no no - - Ignore removed typical complementations. - what to do for derivedV=vallex_no - Exceptions for 3>2,pos are now hardcoded in the add_valdiff transform (but not in the calculation of Specval.diff itself!): 3>2,pos should NOT be treated as typical in the case of velení-1, velení-3, vyučování-vyučení-1 and výuka-1 - Handle DPHR. Currently we assume a typical change if a deleted form and an added for coincide on the first 4 characters - Better treatment of CPHR, 2>2 and 3>2,pos. - Add_specval -> add_valdiff. [Anša Vernerová] - Rename cspecval -> valdiff. [Anša Vernerová] Server ~~~~~~ - Use server timestamps to detect changes (Fixes `#81 `_) [Jonathan L. Verner] - Fix missing help texts (fixes `#74 `_) [Jonathan L. Verner] - Serve icon fonts locally for offline use. [Jonathan L. Verner] - Extract more data for events sent to sentry.io. [Jonathan L. Verner] - Don't pretend we can handle editing lu ids. [Jonathan L. Verner] - Implement a rpc method to capture logs from the frontend. [Jonathan L. Verner] - Add error loging for failed HTTP requests. [Jonathan L. Verner] - update the example deploy/production/server.ini. [Jonathan L. Verner] -- show how multiple lexicons can be specified -- provide a correct setting for dist-dir -- include a logging section - Add a script to reload the service when code changes. [Jonathan L. Verner] - Allow client logging even in server mode. [Jonathan L. Verner] Packaging ~~~~~~~~~ - Disable pyinstaller related CI actions. [Jonathan L. Verner] - Make bsdiff4 & pynacl dependency optional. [Jonathan L. Verner] - Explicitly specify qtwebengine version dependency. [Jonathan L. Verner] - Rename pypi package to vallex-tools. [Jonathan L. Verner] - Add vallex-web, vallex-gui scripts to the package. [Jonathan L. Verner] - Specify gunicorn as an extra dependency (for the server) [Jonathan L. Verner] - Update version before building python package & set it back afterwards. [Jonathan L. Verner] User Interface ~~~~~~~~~~~~~~ - Icons are now usable when offline - Show a dot after typical changesd in the Valdiff attribute [Anša Vernerová] Command Line Interface ~~~~~~~~~~~~~~~~~~~~~~ - Do not fail if nacl library is not present. [Jonathan L. Verner] - Only log import errors as logging.ERROR when they are required. [Jonathan L. Verner] - Add sync/reset db commands, remove maintenance cmd. [Jonathan L. Verner] sync_db - Saves user edits present in the web db back into the lexicon files and loads any changes from the lexicon files into the db and any new lexicons specified in the config. reset_db - Recreates the database and loads the lexicons specified in the config into the newly created db. The second command is run when, e.g., the database contains some lexicons, which are not specified in the config and one wants to remove them. Documentation ~~~~~~~~~~~~~ - Update install instructions in README.rst. [Jonathan L. Verner] Update the instructions now that a pypi package is available. Also, provide a script to install vallex-tools on windows. - Rename pyvallex to vallex-tools. [Jonathan L. Verner] Start renaming pyvallex to vallex-tools. This commit does it for most of the documentation. Links, however, still point to `verner.gitlab.io/pyvallex`. They will be updated once we rename the repo & move the static site to the new address. What remains to be done: - rename project & repo on gitlab - move static doc site to new url - rename config files (pyvallex.ini, ...) - update the production scripts & configs - rename files under `doc/_static` - determine under what url the production server will run and update the `VUE_APP_BASE_URL` in `vallex/server/frontend/.env.server` accordingly - Document the preferred git branching structure. [Anša Vernerová] - Add basic documentation the scripts in $VALLEX_HOME/bin. [Jonathan L. Verner] - Add a troubleshooting section to production doc. [Jonathan L. Verner] - Add a section to production docs on logs. [Jonathan L. Verner] - Make the reload/update scripts print some documentation. [Jonathan L. Verner] - Document using compute-slovko.py. [Anša Vernerová] Fixes `#66 `_ - Document more methods in config.py. [Jonathan L. Verner] - Link the LREC paper from the left menu on verner.gitlab.io [Anša Vernerová] - More informative dev-setup output. [Anša Vernerová] - Comments in the update-production script. [Anša Vernerová] DevOps ~~~~~~ - Add tests for editing + changes web api. [Jonathan L. Verner] - Fix devsetup (workaround poetry bug `#197 `_) [Jonathan L. Verner] - Disable pyinstaller jobs - Add setup target to Makefile (Fixes `#84 `_) [Jonathan L. Verner] - Upgrade js packages used to compile frontend files. [Jonathan L. Verner] - Simplify how production code is run & deployed. [Jonathan L. Verner] In the new setting, the server is run directly from the git repo, which should make it easier to update (just do a git pull in the repo, recompile js & update dependencies) and also modify in place. Also, move the server config out of version control (so that it isn't overwritten every time we update the repo). For more info see doc/production.rst. Other ~~~~~ release/v0.11 (2020-04-17) -------------------------- Highlights ~~~~~~~~~~ - Move from pipenv to poetry for managing dependencies - Create a PyPi package - Add bash command-completion to the cli script - Add log file configuration + log uncaught exceptions including nice backtraces (using the better_exceptions library) - Add a script to setup the development environment Bugfixes ~~~~~~~~ - Fix yarn build when vallex/__version__ does not exist (See Issue `#59 `_) [Jonathan L. Verner] Packaging (pkg) ~~~~~~~~~~~~~~~ - Add a changelog. [Jonathan L. Verner] - Build the win-pyinstaller image in CI. [Jonathan L. Verner] - Update PyInstaller image to use v3.6. [Jonathan L. Verner] This fixes a bug with bundling QtWebEngine and required OpenGL drivers. Also we need to install vallex dependencies before runnning pyinstaller. We extract these from the Pipfile.lock and create a temporary requirements.txt to be used by pip. - Make a local copy of the docker image used to build the win exe. [Jonathan L. Verner] - Add a command to -- remove deleted releases from releases.json. -- show the list of remotely available releases. -- sign/verify files to the release subcommand. -- check for updates [Jonathan L. Verner] - Make updater an optional dependency. [Jonathan L. Verner] User Interface (ui) ~~~~~~~~~~~~~~~~~~~ - Fix saving lexicons to include changes from the current session [Jonathan L. Verner] - Implement a document browser dialog & use it to replace the doc links. [Jonathan L. Verner] - Set the window icon / favicon. [Jonathan L. Verner] - Fix search not working on windows. [Jonathan L. Verner] The query user enters is added to a query constructed from the pathnames of the lexicons to search through. On windows, these pathnames typically contain '\' characters, which have special meaning in a regular expression. We must therefore escape them (and others). - Fix reload lexicons not working. [Jonathan L. Verner] For some reason (probably a socket in wait state) the backend server, when restarted, doesn't accept connections. This fix is really a hacky workaround which instructs the server to listen on a different port. - Fix a bug in backup file creation. [Jonathan L. Verner] We handle the case when a previous '.backup' file exists and come up with a new 'backup' name. The changed code just keeps appending increasing numbers to generate a file name and uses the first one which is free. - Add an icon for the program. [Jonathan L. Verner] - Update vuetify to 1.5.10, Qt to 5.14 [Jonathan L. Verner] - Fix group select all button in display settings. [Jonathan L. Verner] - Move & rename 'Set Vallex Repo' from Settings to File menu. [Jonathan L. Verner] - Clicking on about icon should not open the sidebar. [Jonathan L. Verner] - Add command to save query results to pdf. [Jonathan L. Verner] - When updating, do not automatically restart, wait for confirmation. [Jonathan L. Verner] Server (srv) ~~~~~~~~~~~~ - Format wsgi server messages nicely. [Jonathan L. Verner] - Set the default locale in vallex.env (for sorting) [Jonathan L. Verner] Otherwise sorting would crash. Command Line Interface (cli) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Make config return absolute paths for cfg files. [Jonathan L. Verner] - Add a basic terminal formatter for "markdown" text. [Jonathan L. Verner] - Add help text to the version and completion commands. [Jonathan L. Verner] - When listing subcommands, print only a short description. [Jonathan L. Verner] - Fix error-reporting when running a script from the cli fails [Jonathan L. Verner] - Add a powershell script to launch vallex-cli. [Jonathan L. Verner] - If a user specifies a port on cmdline, respect it and don't try to find a free one. [Jonathan L. Verner] Library (lib) ~~~~~~~~~~~~~ - Test uniqueness of lu ids on lexical units (so that it shows up in gui) [Jonathan L. Verner] - Fix unique lu id's script. [Jonathan L. Verner] - Add a script to check lu-id uniqueness (in a lexeme) [Jonathan L. Verner] - IsVerb is not a function. [Jonathan L. Verner] It should be a boolean, but the way AttribAccessor is implemented, the checking is convoluted. Perhaps we should drop isVerb/isNoun et al. entirely? - Compute dynamic properties for edited lu before running validation. [Jonathan L. Verner] - Explicitly specify utf-8 encoding when opening/reading/writing files. [Jonathan L. Verner] Windows, unfortunately, still lives in the last century and can have the default encoding set to something as weird as cp1250. We nevertheless expect all our input and output to be in utf-8. - Further CI debugging. [Jonathan L. Verner] - Repair comments for roundtrip. [Anša Vernerová] - Parse derivedN, Numbered attributes + attributes with refs. [Anša Vernerová] - Rename ParamFunctor to ValencySlot and Scope to FunctorCombination. [Anša Vernerová] - Rename noun, verb, computed reflexive attrs to isNoun, isVerb, isReflexverb. [Anša Vernerová] - Don't crash if frame has bad format. [Jonathan L. Verner] - When saving, do not put additional newlines after lexemes. [Jonathan L. Verner] Documentation (doc) ~~~~~~~~~~~~~~~~~~~ - Differentiate between comments & commented-out settings in example config. [Jonathan L. Verner] - Add log-configuration to the example configuration file. [Jonathan L. Verner] - Start writing miscellaneous howto guides to the code. [Jonathan L. Verner] - Improve the documentation for the vallex.term module. [Jonathan L. Verner] - Make sphinx not choke on cli imports. [Jonathan L. Verner] - Add a warning that running from source does not work as described. [Jonathan L. Verner] Cloning the repo and installing python dependencies is not enough to get a working installation since it will miss compiled frontend files. This needs to be fixed somehow. Either we produce release tarballs which will include the frontend files or we commit the frontend files to version control (?) or ...? - Add more info about setting up node in windows (still doesn't work) [Jonathan L. Verner] - Start documenting the windows setup steps. [Jonathan L. Verner] - Document the 'eval' way of activating bash completion. [Jonathan L. Verner] - Mention the Makefile in the development documentation. [Jonathan L. Verner] - Document that NodeJS v != 11.* will probably have problems. [Jonathan L. Verner] - Document that we recommend installing Vue.js devtools. [Jonathan L. Verner] - Update makefile with documentation & fix exe builds. [Jonathan L. Verner] - Put short documentation on top of dev-setup.sh. [Jonathan L. Verner] - Add some documentation to quality/test-devsetup-docker.sh. [Jonathan L. Verner] - Document building. [Anša Vernerová] - Rewrite dev-setup & update dev documentation. [Jonathan L. Verner] The `dev-setup.sh` script is now more user friendly. The price is a more complicated bash script which is probably unreadable for unseasoned bash hackers. It does basic error checking, provides a --help option, detects which programs are installed and skips their installation, in case there are multiple options, it asks the user how to proceed and has nice coloured output. The dev documentation has been updated with a short description of the tools we need, the warning to read the script has been removed and the manual installation instructions have been restructured to fit this description. Additionally the script `quality/test-devsetup-docker.sh` can be used to test the `dev-setup.sh` script in a clean docker image, like so: ./quality/test-devsetup-docker.sh ubuntu:16.04 This should fetch the docker image `ubuntu:16.04` (if not present locally), run the image and mount the repo as into the image under `/src/`. Any changes made to `/src/` in the image are lost when the image is terminated. - Document gitlab CI jobs. [Anša Vernerová] - Development documentation: troubleshooting gitlab CI. [Ansa211] - Frontend development documentation. [Ansa211] Add instructions for debugging and compiling frontend javascript. See https://gitlab.com/Verner/pyvallex/issues/59#note_278235439 - Virtual env for pre-commit hooks. [Anša Vernerová] - Add a link to pdf & LREC conference to the bibliography. [Jonathan L. Verner] - Integrate the LREC paper into the architecture section of doc. [Jonathan L. Verner] - Add the Euralex abstract. [Anša Vernerová] - More comments in dev-setup.sh. [Ansa211] pyenv is used for running multiple python versions on the same system -> it is not needed if the default version is high enough Devops (ci/devops) ~~~~~~~~~~~~~~~~~~~~~~ - Build a pypi package during the build stage. [Jonathan L. Verner] - Enable automatic security scanning. [Jonathan L. Verner] - Fix frontend tests & benchmarks. [Jonathan L. Verner] -- missing py-cpuinfo dependency -- selenium opened port 8080 ignoring the port where server was started -- server was needs to be killed before its stdout/stderr can be read (otherwise this leads to a deadlock) - Colorize test-devsetup-docker.sh. [Jonathan L. Verner] - Do not run autodoc on the tests package. [Jonathan L. Verner] - Fix webdev target in Makefile. [Jonathan L. Verner] The target now runs the backend & frontend servers in the correct order listening on the correct ports and, both servers are killed on Ctrl-C. - Fix Makefile target showdoc to run the server & browser in parallel. [Jonathan L. Verner] release/v0.10 (2019-04-02) -------------------------- Highlights ~~~~~~~~~~ - A new Qt-based gui is now available. - An automatic update system for compiled executables is available Bugfixes ~~~~~~~~ - Fix sub/pscripts in frame (finally fully Fixes #7) [Jonathan L. Verner] The original fix (2ca5959e) for #7 only took care of the cspecval attribute. - Point data revision in about dialog to trac (Fixes #30) [Jonathan L. Verner] - Add option to show full lexemes (Fixes #34) [Jonathan L. Verner] - Make histogram drilldown match full source (Fixes #37) [Jonathan L. Verner] release/v0.9 (2019-03-11) ------------------------- Hilights ~~~~~~~~ - Start implementing frontend display of search stats. [Jonathan L. Verner] - Start implementing a help system. [Jonathan L. Verner] - Add an about dialog. [Jonathan L. Verner] - Implement support for map-reduce scripts. [Jonathan L. Verner] These scripts need to define a map_... and reduce_... function. The map function is run on all lexical units (optionally filtered by a post-query) and should use emit to emit (key, value) pairs, where key is a tuple and value is any value. Finally, the reduce method runs the reduce_... function and produces the results. There is a helper method to create a table out of the results by converting the first part of each key to a column and each unique extension of the key then corresponds to a single row. - Fix editor cursor placement bug. [Jonathan L. Verner] Bugfixes ~~~~~~~~ - Add a test for reciprocal verb attributes (Fixes #33). [Jonathan L. Verner] - Don't show '/' separator in cspecval when not necessary (Fixes #38). [Jonathan L. Verner] - Move cspecval below frame + bugfix oblig in subscript (Fixes #7) [Jonathan L. Verner] - Finish alternative histograms (fixes #18) [Jonathan L. Verner] Allow the user to choose between a histogram counting each occurance, or a histogram counting each occurance only once per lu. - Send histogram query as urlencoded JSON (fixes #11). [Jonathan L. Verner] - Show search stats in addition to histogram (fixes #19). [Jonathan L. Verner] - Allow negated queries (fixes #17). [Jonathan L. Verner] Negated queries use '!=' instead of '=' to join the match_key with the pattern. Documentation ~~~~~~~~~~~~~ - Improve documentation slightly. [Jonathan L. Verner] -- add notes on python installation in devel -- add new modules (mapreduce, constants, maint) to api docs -- add additional permission commands to deploy instructions -- fix a typo in bottle_utils docs - Add documentation for mapreduce scripts. [Jonathan L. Verner] - Document Api.js. [Jonathan L. Verner] - Improve the search-form documentation. [Jonathan Verner] - Javascript/html linting & documentation. [Jonathan L. Verner] - Add examples. [Jonathan L. Verner] -- a more complicated mapreduce example -- an example python script not using the vallex.script infrastructure but using pyvallex as a library, which loads lexicons and does some computations with them Continuous Integration ~~~~~~~~~~~~~~~~~~~~~~ - Disable selenium tests on gitlab due chromium crashing there, no matter what we do. [Jonathan L. Verner] - Implement infra for frontend testing. [Jonathan L. Verner] -- add pytest-selenium to dependencies -- add script to download firefox webdriver -- add tox env to run selenium tests -- add a few basic tests in tests/browser/... Currently: -- test that the help dialog for search form opens -- test that the permalink for search form opens and copies url to clipboard when the correct button is clicked -- test that entering search query shows autocomplete stuff and appends '=' on enter -- update gitlab-ci to run the browser tests - Update tests for match_lu. [Jonathan Verner] Other ~~~~~ - Fix error & matchmap buttons not showing up in source view. [Jonathan L. Verner] - Another cspecval search bugfix: [Jonathan L. Verner] -- search keys of the form `ACT.eq.form.eq` should work properly. -- should fix (mostly) #43 - Include the current query in the bugreport created by report bug btn. [Jonathan L. Verner] - Add productive computed attribute. [Jonathan L. Verner] - Implement search by ACTANT functors. [Jonathan L. Verner] - Handle TODO values for frame attribute. [Jonathan L. Verner] - Preserve forms order in typical changes (cspecval). [Jonathan L. Verner] - Preserve forms ordering in cspecval attribute. [Jonathan L. Verner] - Make the vertical space between specval & cspecval bigger. [Jonathan L. Verner] - Make src of attribs more useful when computed. [Jonathan L. Verner] - Handle typical changes in specval. [Jonathan L. Verner] -- introduce a new attribute of SpecvalElement `forms_typ` which is a list of pairs (verb_form, noun_forms_set) -- compute it for diffs -- show it in the frontend interface - Rename the `functor-form` key to `functor:form`. [Jonathan L. Verner] - Fix histogram counts when there is no match. [Jonathan L. Verner] - Add & rename match keys for frame. [Jonathan L. Verner] As per the request in Issue #26 this commit: -- renames frame.forms to frame.form -- adds frame.forms list of all form-tuples (one per functor) -- adds frame.functor-form list of functor;form pairs (one per each form of each functor) - Allow showing hilighted lu source. [Jonathan L. Verner] - Add reload action to error alert when initializing. [Jonathan L. Verner] - Show a progressbar when computing histogram. [Jonathan L. Verner] - Show total stats even when no search is performed. [Jonathan L. Verner] - Prepare for alternative histogram counting methods (see #18). [Jonathan L. Verner] - additionally show percentages instead of total count - show total count in radiobuttons for selecting counting method release/v0.8 (2019-02-22) ------------------------- Highlights ~~~~~~~~~~ Bugfixes ~~~~~~~~ - Implement dlg to show what strings a given query will match against (fixes #15) [Jonathan L. Verner] - Recompute the histogram on query change (fixes #5) [Jonathan L. Verner] - Add documentation to search keys (fixes #16). [Jonathan L. Verner] Documentation ~~~~~~~~~~~~~ - Link to the compiled executables from the docs. [Jonathan L. Verner] Continuous Integration ~~~~~~~~~~~~~~~~~~~~~~ - Add a skeleton file for cli tests (from the original app_state tests) [Jonathan Verner] - Add tests for the maint module + update test_webapp for the new db handling. [Jonathan Verner] Other ~~~~~ - Show matched lu count in lexeme header. [Jonathan L. Verner] - Redo sorting of the search keys. [Jonathan L. Verner] First come keys without any dot in them ordered according to popularity and then alphabetically. After that keys with dots in the same ordering. - Update searching a bit. [Jonathan L. Verner] -- for cspecval, some keys only makes sense for common functors -- add oblig.changed to cspecval -- refactor comment searching by adding the match_key_values method to comments -- make frame search the list of slot sources as opposed to frame.src searching the whole source - Add help for dynamically computed attributes. [Jonathan L. Verner] - Rename computedspecval to cspecval to make it faster to type. [Jonathan L. Verner] - Use order of functors from noun frame. [Anša Vernerová] - Rename frame slot to form. [Jonathan L. Verner] - Run the dev webserver by default. [Jonathan L. Verner] Since, on Windows, the program will typically be run without arguments and the intended action is to run the web-interface. - Ignore lexicons starting with underscore ('_'). [Jonathan L. Verner] release/v0.7 (2019-02-18) ------------------------- Highlights ~~~~~~~~~~ - Implement partial search-match hilighting. [Jonathan L. Verner] - Implement permalinks for search/histogram queries. [Jonathan L. Verner] - Implement sorting search keys by popularity. [Jonathan L. Verner] Bugfixes ~~~~~~~~ - Display matched lu numbers (fixes #1) [Jonathan L. Verner] For each lexeme also display number of matched lexical units in addition to the total number of its lexical units. - Allow equal sign in queries. [Jonathan L. Verner] - Fix histogram drilldown when value selector is specified. [Jonathan L. Verner] Frontend ~~~~~~~~ - Stop displaying search in progress bar on net error. [Jonathan L. Verner] - Add application loading progress page. [Jonathan L. Verner] - Fix hilighting when invalid query specified. [Jonathan L. Verner] Documentation ~~~~~~~~~~~~~ - Update documentation. [Jonathan L. Verner] -- include vallex.server documentation in devel -- fix restapi doc (moved from vallex.server.app:app._ to wsgi:app) -- rename deployment instructions (server.rst to deploy.rst) - Fix bugs in development documentation. [Jonathan L. Verner] - Prepare server deployment documentation & compile assets. [Jonathan L. Verner] -- add sample configs in deploy subdir: - a nginx snippet - a systemd service definition - config for the server -- make server aware whether it should allow rpc (in single user setting --- client mode) or not (in multiuser setting --- server mode); the mode is determined in the web section of config by setting the mode key to either ``client`` or ``server`` -- start writing deployment documentation in docs/server.rst (currently just a log of what I did on logic.ff.cuni.cz) -- - Frontend documentation work: Attrib.vue + Store.js + Stats.vue. [Jonathan L. Verner] release/v0.6 (2019-02-10) ------------------------- New Features ~~~~~~~~~~~~ - Add progressbars to indicate lexicon loading in cli & web app. [Jonathan L. Verner] - Implement computed specval (backend + frontend). [Jonathan L. Verner] The specval attribute is a diff between the frame of a verb and the frame of its derived noun. Documentation ~~~~~~~~~~~~~ - Documentation work. [Jonathan L. Verner] -- add the scripts module to api contents -- add specval + specvalelement to attrib class list -- add documentation to some specval methods Continuous Integration ~~~~~~~~~~~~~~~~~~~~~~ - Show number of skipped tests in console. [Jonathan L. Verner] - Add a datavalidation test for derived[VN] links. [Jonathan L. Verner] Other ~~~~~ - Fix frontend did not show links for non-lvc attributes (e.g. derivedV). [Jonathan L. Verner] - Do not discard duplicate attributes. [Jonathan L. Verner] When a duplicate attribute is found a new unique name is generated for it and the attribute is renamed with the old name saved in its 'duplicate' property. -- when searching, the condition is or'ed accross all duplicate attributes -- when outputting, the original name (saved in the duplicate property) is used - Do not discard 'EMPTY' in empty frames. [Jonathan L. Verner] - Make grep not discard lexeme comments. [Jonathan L. Verner] - Allow filtering for other commands than grep. [Jonathan L. Verner] - Implement noun, verb and refl as dynamic properties. [Jonathan L. Verner] release/v0.5.1 (2019-02-05) --------------------------- - Rename js package & page title from frontend to NomVallex frontend. [Jonathan L. Verner] - Search form: color query keys. [Jonathan L. Verner] - Plus unify colors with editor hilighting. - Plus sort attributes alphabetically - Make sidebar more consistent. [Jonathan L. Verner] -- settings & stats dialogs look similar (yellow title, close button) -- when open, settings & stats have titles and are left-aligned -- the sidebar closing arrow is aligned with the submenu arrows release/v0.5 (2019-02-05) ------------------------- Highlights ~~~~~~~~~~ - A new frontend using Vue.js. [Jonathan L. Verner] - basic editing functionality - shows data-validation results - histogram - attr hiding - Implement configuration file for cli. [Jonathan L. Verner] - Add infrastructure for data tests. [Jonathan L. Verner] - Add rudimentary ability to run transforms on lexicon collections. [Jonathan L. Verner] A transform consists of a file defining (at least) one of the methods def process_collectio(collection) def process_lexicon(collection, lexicon) def process_lexeme(collection, lexeme) def process_lu(collection, lu) When the cli command 'transform' is invoked, these methods are applied to the loaded collection and then the collection is sent to the output. Optionally the transform file may define a global attribute TRASFORMED_ATTRS, a list of attribute names. If this attribute is defined, when outputting the collection, all attributes which are not in the list are output via their src property (i.e. as if they were unparsed). This should limit the diff to the input file. Frontend ~~~~~~~~ - Rework layout of lexeme & adapt to new attrib structure. [Jonathan L. Verner] - Fix lvc-links dialog on click. [Jonathan L. Verner] - Add progressbars when querying server. [Jonathan L. Verner] - Allow restricting search to a data-source. [Jonathan L. Verner] Documentation ~~~~~~~~~~~~~ - Further documentation work. [Jonathan L. Verner] Mainly: -- token documentation (txt_tokenizer) -- sql tables documentation (sql_store) -- filter_attrs (grep) -- bottle_utils module documentation Plus some other more minor stuff. - Finish documenting data_structures.attribs. [Jonathan L. Verner] - Finish documenting the data_structures.lexical_unit module. [Jonathan L. Verner] - Document collections, lexical units & utils modules. [Jonathan L. Verner] - Further API documentation work. [Jonathan L. Verner] -- start documenting data_structures.collections -- (mostly) finish documenting server.sql_store -- Start documenting the Attrib class. [Jonathan L. Verner] -- Start documenting the rest api. [Jonathan L. Verner] -- finish documenting txt_parser & txt_tokenizer -- write an intro in vallex/__init__.py explaining code layout -- move api/index.rst to development.rst (its not just api docs) -- provide some instructions for devel env setup - how to install pipsi, pipenv and create the virtualenv - how to install node, yarn and the javascript dependencies -- provide some instructions on running unit tests, and mypy static tests -- simple info on pre-commit hooks -- finish documenting vallex.grep -- document vallex.json_utils, vallex.location -- start documenting the whole vallex package -- restructure the package documentation (doc/api/vallex.rst) a bit (add toc, reorganize module contents...) -- add a basic README.rst + set up spinx documentation infrastructure under doc - Add a short readme describing templates. [Jonathan L. Verner] Continuous Integration ~~~~~~~~~~~~~~~~~~~~~~ - Move code tests to top-level tests directory. [Jonathan L. Verner] - Allow loading data-validation tests from configured dirs. [Jonathan L. Verner] - Add a test to test txt roundtrip. [Jonathan L. Verner] - Implement three data tests. [Jonathan L. Verner] acc_bez_deagent (test-acc-bez-AuxRT.pl) chybejici_pat (test-chybejici-PAT.pl) chybejici_varianta (test-chybejici-varianta.pl) - Add pytest pre-commit hook. [Jonathan L. Verner] - Add test to check lexeme locations. [Jonathan L. Verner] Parser ~~~~~~ - Fix parsing of aspect. [Jonathan L. Verner] - Warn on invalid attribute names + extend functor list (txt2xml_b.pl) [Jonathan L. Verner] - For unknown attribs, still parse comments. [Jonathan L. Verner] - Feat: parse the frame attr properly. [Jonathan L. Verner] - Fix parsing lvc attrib (take '|' into account) [Jonathan L. Verner] - Add recipr rendering to txt template. [Jonathan L. Verner] - Implement attr: note. [Jonathan L. Verner] - Do not throw away comments. [Jonathan L. Verner] - Handle all example types. [Jonathan L. Verner] - Handle lus whose id ends with a comment. [Jonathan L. Verner] - Handle idiom attr in frame. [Jonathan L. Verner] - Fix parsing lus with missing mandatory attrs (infinite loop). [Jonathan L. Verner] Other ~~~~~ - Add check for missing ACT. [Jonathan L. Verner] - Add a check for missing lvc references. [Jonathan L. Verner] - Implement example count vs lvc count check. [Jonathan L. Verner] -- see _tests-lvc_TODO.txt, Kontrola sloves bod 7. - Handle lvc attrs in txt output. [Jonathan L. Verner] - Do not discard comments. [Jonathan L. Verner] - Grep returns locale-aware sorted data. [Jonathan L. Verner] - Parse all lvc-type attributes. [Jonathan L. Verner] - Cleanup. [Jonathan L. Verner] - Implement basic grep functionality. [Jonathan L. Verner] - Implement output & check. [Jonathan L. Verner]