Integration of sequoia into sett (and potential replacement of buggy gpg)

changed milestone to %Sprint 2022/02/18

changed the description

We could either use sequoia python bindings or use sequoia native rust API directly in sett-rs.

Summary

It seems that sequoia provides all the necessary functionality for sett. I was able to crate a short PoC for the encryption workflow in Rust using sequoia. As with the transfer workflow it's possible to expose this encrypt workflow to Python and replace the original workflow implementation. I expect that the decrypt workflow and key management can be also implemented with sequoia. One thing I didn't explore was key signature verification. However, since we are going to move to a different key verification process, key signature verification is not needed any more. In summary, we should be able completely drop gnupg in favor of sequoia.

I was not able to find multi-threading support for encryption, but it should be possible to add multi-threading to the compression step using gzp, a library inspired by pigz. Which should significantly improve the overall encryption workflow speed.

How to test the PoC

Prepare before running the PoC:

Have the sq (from sequoia) binary installed.
- Clone the sequoia repository
- cargo build --release -p sequoia-sq
- The sq binary can be found at target/release/sq
Clone sett-rs and add the code below to sett-rs/examples/encrypt.rs (note: you could also create an empty project, but using sett-rs is probably faster since the file structure is already present).

Add dependencies to sett_rs/Cargo.toml

flate2 = "1.0"
sequoia-openpgp = "1.7"
tar = "0.4"
zip = "0.5"

In the current working directory (sett-rs repo root) create a data directory with some files (and/or directories) inside.

Run the command below to test the PoC.

# Create a new PGP key
sq key generate --userid "Chuck <chuck@example.org>" --export chuck.key.pgp
# Extract the public key
sq key extract-cert --output chuck.cert.pgp chuck.key.pgp
# Run the PoC encryption workflow: tar and compress files inside of the `data`
# directory, encrypt the archive using the generated public key, and store
# the encrypted archive in a zip file
cargo run --release --example encrypt -- chuck.cert.pgp
# Confirm that compression and encryption worked
unzip dpkg.zip
sq inspect data.tar.gz.pgp
sq decrypt --recipient-key chuck.key.pgp -o data.tar.gz data.tar.gz.pgp
tar --list -f data.tar.gz

The PoC (place the code in the examples directory).

use std::fs::File;
use std::path::Path;

extern crate sequoia_openpgp as openpgp;
use flate2::write::GzEncoder;
use flate2::Compression;
use openpgp::parse::Parse;
use openpgp::policy::{Policy, StandardPolicy};
use openpgp::serialize::stream::{Encryptor, LiteralWriter, Message};
use zip::{write::FileOptions, CompressionMethod, ZipWriter};

fn compress_and_encrypt(
    input: &str,
    output: &str,
    policy: &dyn Policy,
    recipient: &openpgp::Cert,
) -> openpgp::Result<()> {
    let recipients = recipient
        .keys()
        .with_policy(policy, None)
        .supported()
        .alive()
        .revoked(false)
        .for_transport_encryption();

    let package_file = File::create(output)?;
    let mut zip = ZipWriter::new(package_file);
    let options = FileOptions::default().compression_method(CompressionMethod::Stored);
    zip.start_file("data.tar.gz.pgp", options)?;

    // Start streaming an OpenPGP message.
    let message = Message::new(&mut zip);
    let message = Encryptor::for_recipients(message, recipients).build()?;
    let mut writer = LiteralWriter::new(message).build()?;

    // Encrypt the data.
    {
        let enc = GzEncoder::new(&mut writer, Compression::default());
        let mut tar = tar::Builder::new(enc);
        tar.append_dir_all(input, input)?;
    }

    // Finalize the OpenPGP message to make sure that all data is written.
    writer.finalize()?;
    // Finish the last file and write all other zip-structures
    zip.finish()?;
    Ok(())
}

fn main() -> Result<(), String> {
    let certificate = std::env::args().nth(1).ok_or("No key passed".to_string())?;
    let key = openpgp::Cert::from_file(Path::new(&certificate)).unwrap();
    let policy = &StandardPolicy::new();
    compress_and_encrypt("data", "dpkg.zip", policy, &key).unwrap();
    Ok(())
}

We could implement key checking before going to Rust. But probably better to wait until we migrated keyserver.

closed

mentioned in issue sett-rs#12 (closed)

added to epic &7 (closed)

Integration of sequoia into sett (and potential replacement of buggy gpg)

Child items ...

Activity