when blockchain replay is attempted "blocks" file increases in size
Steps:
- Installed peerplays docker container (connection to mainnet)
- Allowed the witness node to be run for a day so that it is synced and catches up with rest of the nodes
- Tried replay the blockchain again, which started properly but it is seen that the "blocks" file increases from the size before replay was started
Issue Seen:
- Say that the block size at T1 is 1G
- Replay takes 1 hr in which new blocks size of 10Mb is created
Then if we run repay 5 times the block size would be ~6.15Gb whereas actually it should have been ~1.05GB
Ideally when the replay is started, the existing blocks file should be used to create a new blocks file and when the replay is finished the original blocks file should be removed
Logic for replay: -- Docker container is started using the following command sudo docker run -p0.0.0.0:2001:2001 -v /dev/shm:/shm -v /home/qa/latest/peerplays-docker/data:/peerplays -d --name peerplays -t datasecuritynode/peerplays:latest witness_node --data-dir=/peerplays/witness_node_data_dir -- to replay the following steps are done
- Stops the running peerplays docker container, if any and removes the container
- Started it with the following command sudo docker run -p0.0.0.0:2001:2001 -v /dev/shm:/shm -v /home/qa/latest/config-file-from-build/peerplays-docker/data:/peerplays -d --name peerplays -t peerplays witness_node --replay-blockchain --data-dir=/peerplays/witness_node_data_dir
-- Before replay
qa@PBSA-Dev:~/latest/config-file-from-build/peerplays-docker$ while true; do ls -lrt data/witness_node_data_dir/blockchain/database/block_num_to_block/*;date;sleep 2; done
-rw-r--r-- 1 root root 1146269760 Nov 27 08:05 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
-rw-r--r-- 1 root root 5839105369 Nov 27 08:05 data/witness_node_data_dir/blockchain/database/block_num_to_block/blocks
Fri Nov 27 08:05:22 EST 2020
-rw-r--r-- 1 root root 1146269760 Nov 27 08:05 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
-rw-r--r-- 1 root root 5839105369 Nov 27 08:05 data/witness_node_data_dir/blockchain/database/block_num_to_block/
Fri Nov 27 08:05:24 EST 2020-- Started the replay
3595306ms th_a accounts_list_plugin.cpp:129 list_accounts ] accounts list plugin: list_accounts()
3595306ms th_a bookie_plugin.cpp:469 plugin_initialize ] bookie plugin: plugin_startup() begin
3595306ms th_a db_management.cpp:290 force_slow_replays ] enabling slow replays
3595306ms th_a bookie_plugin.cpp:502 plugin_initialize ] bookie plugin: plugin_startup() end
3595307ms th_a witness.cpp:92 plugin_initialize ] witness plugin: plugin_initialize() begin
3595307ms th_a witness.cpp:104 plugin_initialize ] Public Key: PPY6MRyAjQq8ud7hVNYcfnVPJqcVpscN5So8BhtHuGYqET5GDW5CV
3595307ms th_a witness.cpp:122 plugin_initialize ] witness plugin: plugin_initialize() end
3595307ms th_a db_management.cpp:180 wipe ] Wiping database
3595307ms th_a object_database.cpp:93 wipe ] Wiping object database...
3595397ms th_a object_database.cpp:95 wipe ] Done wiping object databse.
3595397ms th_a object_database.cpp:106 open ] Opening object database from /peerplays/witness_node_data_dir/blockchain ...
3595397ms th_a object_database.cpp:111 open ] Done opening object database.
3595398ms th_a application.cpp:349 operator() ] Initializing database...
3597155ms th_a db_debug.cpp:85 debug_dump ] total_balances[asset_id_type()].value: 239781396552 core_asset_data.current_supply.value: 546051993031
3597157ms th_a db_management.cpp:107 reindex ] reindexing blockchain
3597157ms th_a db_management.cpp:113 reindex ] Replaying blocks, starting at 1...
3597160ms th_a db_maint.cpp:1321 process_dividend_ass ] In process_dividend_assets time 2017-06-06T16:00:03
3597475ms th_a db_maint.cpp:1321 process_dividend_ass ] In process_dividend_assets time 2017-06-06T17:00:00
3597688ms th_a db_maint.cpp:1321 process_dividend_ass ] In process_dividend_assets time 2017-06-06T18:00:00-- During replay
qa@PBSA-Dev:~/latest/config-file-from-build/peerplays-docker$ while true; do ls -lrt data/witness_node_data_dir/blockchain/database/block_num_to_block/*;date;sleep 2; done
-rw-r--r-- 1 root root 6248687095 Nov 27 12:06 data/witness_node_data_dir/blockchain/database/block_num_to_block/blocks
-rw-r--r-- 1 root root 1146269952 Nov 27 12:06 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
Fri Nov 27 12:06:57 EST 2020
-rw-r--r-- 1 root root 1146269952 Nov 27 12:06 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
-rw-r--r-- 1 root root 6250662887 Nov 27 12:06 data/witness_node_data_dir/blockchain/database/block_num_to_block/blocks
Fri Nov 27 12:06:59 EST 2020
-rw-r--r-- 1 root root 1146269952 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
-rw-r--r-- 1 root root 6252597960 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/blocks
Fri Nov 27 12:07:01 EST 2020
-rw-r--r-- 1 root root 1146269952 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
-rw-r--r-- 1 root root 6254548444 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/blocks
Fri Nov 27 12:07:03 EST 2020
-rw-r--r-- 1 root root 1146269952 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
-rw-r--r-- 1 root root 6256454395 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/blocks
Fri Nov 27 12:07:05 EST 2020
-rw-r--r-- 1 root root 1146269952 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
-rw-r--r-- 1 root root 6258449823 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/blocks
Fri Nov 27 12:07:07 EST 2020
-rw-r--r-- 1 root root 1146269952 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
-rw-r--r-- 1 root root 6260414011 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/blocks
Fri Nov 27 12:07:09 EST 2020
-rw-r--r-- 1 root root 1146269952 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
-rw-r--r-- 1 root root 6262065692 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/blocks
Fri Nov 27 12:07:11 EST 2020
-rw-r--r-- 1 root root 1146269952 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/index
-rw-r--r-- 1 root root 6263677500 Nov 27 12:07 data/witness_node_data_dir/blockchain/database/block_num_to_block/blocks
Fri Nov 27 12:07:13 EST 2020