Skip to content

refactor(asuran-chunker): Replace SmallRng with ChaCha20 and use a precomputed table for buzhash

snsmac requested to merge (removed):chunker-distribution into master

SmallRng is replaced with ChaCha20 because the rand documentation recommends against it for reproducible results.

To get an even distribution in the buzhash table a precomputed table is used. To prevent fingerprinting this table is xored with an random value. The table is generated with the following program:

use intbits::Bits;
use rand::seq::SliceRandom;
use rand::SeedableRng;
use rand_chacha::ChaCha8Rng;

fn main() {
    let mut rng = ChaCha8Rng::seed_from_u64(1337);
    // Number of entries to generate
    let count = 256;

    // Fill a vector with count elements with half zeroes and half ones
    let mut init = vec![false; count];
    for i in 0..(count / 2) {
        init[i] = true;
    }

    // Generate look up table
    let mut buztbl = vec![0u64; count];
    for i in 0..64 {
        init.shuffle(&mut rng);
        for j in 0..count {
            buztbl[j].set_bit(i, init[j]);
        }
    }
    //Check if we have duplicates
    buztbl.sort_unstable();
    buztbl.dedup();
    assert_eq!(buztbl.len(), count);
    buztbl.shuffle(&mut rng);

    //print
    println!("#[rustfmt::skip]");
    println!("const TABLE: [u64; 256] = [");
    for (c, i) in buztbl.iter().enumerate() {
        let mut a = format!("{:#018x}, ", i);
        a.insert(6, '_');
        a.insert(11, '_');
        a.insert(16, '_');
        if c % 4 == 0 {
            print!("    ");
        }
        print!("{}", a);
        if c % 4 == 3 {
            print!("\n");
        }
    }
    println!("];");
}

BREAKING CHANGE: Buzhash may create different chunks compared to the previous version

Closes #50 (closed)

Edited by snsmac

Merge request reports

Loading