Skip to content

[#977] Refuse to startup if database format is higher than build format

Ganesh Mahesh requested to merge zylog1O1/YDBOcto:ydbocto977 into master

Issue (copied from #977 (closed) description)

@shabiel came across an issue when looking into #211 (closed) (!1244 (merged)).

The usecase is as follows:

  • A VIEW was created in #211 (closed) branch (which has a higher value of FMT_BINARY_DEFINITION than the master branch)
  • Branch was switched to master
  • Octo was started
  • An error saying CREATE VIEW cannot be parsed was seen.

This happened as a binary upgrade happened at the start of Octo. The above case is specific to VIEWS but this is a general issue where any new feature usage which stores binary information can result in.

In cases like these where the binary format changed from a higher value to a lower value an upgrade should be avoided and an error should be generating saying the DB has binary data which is not compatible with the current binary format definition.

Analysis

An upgrade was allowed in this case as all the conditions check whether the binary definition format value in the DB and the current value of FMT_BINARY_DEFINITION are not equal. This doesn't take into the account of the possibility where the binary definition value in DB is greater. In such a case we need to generate an error and avoid further processing as the artifacts that were created with the higher binary definition format cannot be upgraded to the current binary definition format.

Since octo-seed auto load also needs to go through binary upgrade (in case if a table depends on a function in the seed), octo-seed auto load should also be done after verifying that the binary upgrade can be done. Plan upgrade invocation should also be moved as its processing is wasteful if anyways the upgrade is not going to be allowed.

Fix

  • is_auto_upgrade_valid() is called before octo-seed load or binary definition upgrade
  • The function checks that the binary definition format value in DB is not greater than FMT_BINARY_DEFINITION value. If it is then an error exit is done. The error seen at this point is pasted below. 
    [ERROR]: ERR_AUTO_UPGRADE_DB_HIGHER_FMT: Cannot auto upgrade as DB has data in higher format. Please delete data or manually
upgrade.
  • ERR_AUTO_UPGRADE_DB_HIGHER_FMT is a newly added error message.

Recovering after the ERROR

  • Deleting and re-creating %ydbocto.dat enables auto upgrade again in the current build as the data with higher format version is deleted.

Doc

  • doc/errors.rst is also updated to include ERR_AUTO_UPGRADE_DB_HIGHER_FMT

Test

  • TC078 subtest is added to test_createtable.bats.in to validate this fix.

Verified ae8b0822

Edited by Ganesh Mahesh

Merge request reports