Commit 035ec33c authored by Max Kaye's avatar Max Kaye
Browse files

Prepare for production, update fees, add reddit PSA

parent a472c048
Pipeline #6824881 passed with stage
in 6 minutes and 2 seconds
......@@ -11,7 +11,7 @@ The stress test is scheduled for 1488812400 (which is Mon, 06 Mar 2017 16:00:00
* You will need a new version of `docker-compose` and `docker`.
* You should also have 500GB of space to use (though you shouldn't _need_ more than 400 GB).
* This system expects Ubuntu 16.04 ONLY - no other OSs are supported yet.
* At _least_ 24 hrs (preferably 48hrs) setup time before the stress test as Bitcoin needs to sync with the network.
* At _least_ 24 hrs setup time before the stress test as Bitcoin needs to sync with the network. It might be easier if you can copy your `.bitcoin` folder from elsewhere.
* At _least_ 1.5 MB/s down consistently
Tested with AWS and Digital Ocean nodes (though we are only using AWS atm as we found them more reliable performance wise).
......@@ -30,8 +30,13 @@ Virtual servers will run hot all day, though some high performance desktops or l
Requirements: have a remote server with Ubuntu 16.04 on it and the ability to SSH in (best via a key stored in `.ssh/id_rsa.pub`, etc).
* Clone the repository (https://gitlab.com/exo-one/svst-docker) to your machine.
* From `svst-docker/` run `./mainnet/0_setup_remote.sh <USER> <REMOTE>` from `./svst-docker` making sure to put in the
* If you don't want to bootstrap the blockchain download:
* From `svst-docker/` run `./mainnet/0_setup_remote.sh <USER> <REMOTE>` from `./svst-docker` making sure to put in the
right username and hostname/IP. (Use bash on windows maybe if you're a windows user)
* If you do want to bootstrap
* First run `./mainnet/l1_setup_repos.sh USER REMOTE`
* Then copy `.bitcoin` to `svst-docker/bitcoind-data/.bitcoin` as mentioned in the bootstrap section below
* Then run `./mainnet/x_init.sh USER REMOTE` to start the containers
* This will SSH into a machine, install the requirements, build the containers, and start the stress test.
Tested on Ubuntu 16.10 and macOS 10.12.3
......@@ -108,7 +113,6 @@ stored in these folders, essentially.
* `ipfs` uses `./ipfs-data` to store _all_ downloaded pallets and headers (except invalid ones)
* keep in mind, all IPFS chunks are stored here and the other services only download them via the API temporarily to
verify / process them.
* TODO: the haskell verifier might not actually do that. Not sure if we need to create a cludge or not
* `postgres` uses `./postgres-data` to store the DB files
### Bootstrapping off existing Bitcoind data
......@@ -117,6 +121,26 @@ If you have an existing bitcoin full node you can use it's blockchain data inste
You will need to make sure `bitcoind` is shut down before copying any DB files, and you will want to make sure you don't
copy your wallet.dat if you use one.
**WARNING** - You will need to delete your previous `bitcoin.conf`. The stress test will set up a new one with a known
RPC password automatically. Please also ensure you don't copy your wallet.dat over.
Copy `~/.bitcoin` to `svst-docker/bitcoind-data/.bitcoin` and it should be picked up
by the bitcoin docker container. You should do this before you bring the containers up.
Final directory structure should look like:
```
svst-docker/
├── bitcoind-data/
│   ├── bitcoind-data # just an empty file
│   └── .bitcoin/
│      ├── bitcoin.conf
│      ├── blocks/
│      ├── chainstate/
└── other files...
(more files)
```
### Building Individual Images
Upon occasion you will need to rebuild a single image after changes have been
......
......@@ -10,9 +10,9 @@ IPFS_PATH=/data/ipfs
VERIFY_PUBKEY=faaf31cc2125b01f14c52dec88b2fca37eecae6bfeec8cfb8ba7fdd5fd773eb2
NUM_PALLETS=3000
PALLET_PERIOD=28.0
BOX_P_PALLET=2025
NUM_PALLETS=2750
PALLET_PERIOD=31.0
BOX_P_PALLET=2210
NUM_PRODUCERS = 4
# these imply 64 * 1650 * 2500 * 6 votes = 1.584 billion votes
# over 34 * 2500 seconds = 23.61 hours
......
#!/usr/bin/env bash
if [ $# -ne 2 ]; then
echo "useage: ./mainnet/l0_initialize_remote.sh USER HOST"
echo "useage: ./mainnet/0_initialize_remote.sh USER HOST"
echo "Requirement: Need to be able to login via ssh to USER@HOST"
echo "This script will then automatically configure the box to work for XO.1's secure.vote stress test."
exit 1
......
Title: PSA: We're running a stress test of our voting system, and there might be some congestion.
# What's Happening
At 1488812400 UTC (06 Mar 2017 16:00:00 UTC, or 2am on the 7th in Australia)
[XO.1](https://xo1.io) and [Flux](https://voteflux.org) will be demonstrating the
throughput of the [SecureVote](https://secure.vote) voting system. Over 1 million
votes per minute will be anchored to the Bitcoin blockchain, for a
total of approximately 1.5 billion votes over the 24 hours.
This will take place through approximately 11,000 transactions over 24 hours. This is about 4% of Bitcoin's capacity
so may cause increased confirmation times and/or fees.
Source code and documentation can be found at: https://gitlab.com/exo-one/svst-docker and we have a [white-knapkin](https://gitlab.com/exo-one/svst-docker/blob/master/svst-docs/secure.vote.white.knapkin.md)
if you'd like some more technical data.
# What is SecureVote
SecureVote is a product by XO.1 - the world's first high capacity, secure, general purpose, end to end verifiable voting system.
There is no central point of failure, all votes are fully anonymized, and no fancy new crypto is used (IE: not ZKPs or homomorphic encryption).
# What is XO.1
XO.1 is a small startup (currently in the early stages of our next fundraising round) founded by myself (Max Kaye) and Nathan Spataro.
XO.1 was created to provide a commercially viable secure voting system, something which does not exist anywhere on the planet today.
A key requirement for us is high throughput, something that requires a second-layer solution.
# What is Flux
[Flux](https://voteflux.org) is a political movement also founded by myself and Nathan Spataro.
The Flux Movement aims to instantiate a novel form of democracy called Issue Based Direct Democracy (IBDD).
Unlike all other forms of democracy, IBDD is deeply concerned with epistemology and providing the best decision
making possible.
IBDD is designed to bias _objectivley true knowledge_ and is possibly the first democratic [truth machine](http://www.philosophersbeard.org/2010/11/democracy-is-not-truth-machine.html).
We hold that the main problem with democracy today is that it does not bias the best knowledge possible. The
symptoms of this are corruption, bloat, sluggishness, low approval ratings, etc.
If that sort of thing interests you, I have a pre-release paper I can send on request (but not yet publish). Also,
the philosophical groundwork is laid out in David Deutsch's breathtakingly profound book _[The Beginning of Infinity](https://en.wikipedia.org/wiki/The_Beginning_of_Infinity)_ (2011).
Also we're expanding with chapters globally so if you'd like to get involved please get in touch.
## Why don't we use Ethereum
I've been involved in the Ethereum world since [December 2013](https://web.archive.org/web/20140208053651/https://bitcointalk.org/index.php?topic=428589.0)
and have been [publically credited](https://youtu.be/o6D8Up411dI?t=29m16s) a [few times](http://imgur.com/piL9eYc)
with contributing to scaling discussions around that sort of technology.
My view boils down to this: Ethereum can do _anything_, but it can never do _everything_.
This stress test will produce about 150 GB of data. This isn't something the main Ethereum chain can handle, and even
many years into the future it would be selfish to dump this volume of data into a shared resource.
Additionally, relying on others to eventually come up with a scaling solution isn't a great strategy IMO.
Since we needed a 2nd layer solution anyway it only makes sense to build on the most secure blockchain around: Bitcoin.
Additionally: _secure_ private blockchains are a pipe dream. Private blockchains themselves are both easy and pointless (when it comes to public security).
## Footnotes:
1. Secure here means a lot of things: no central point of failure, able to be run without the _active_ participation of
a central authority, secret ballot, fully cryptographically verifiable by all parties, using battle tested crypto only, etc.
......@@ -35,7 +35,9 @@ Block: a Bitcoin block
Quanta was the name of a type of blockchain I designed in 2014, see https://github.com/XertroV/quanta-test
The initial selling point was instant transactions on a PoW blockchain (yes really)
We've adapted the idea to work on top of the Bitcoin blockchain (so a blockchain on a blockchain, if you will).
The initial selling point for Quanta was instant transactions on a PoW blockchain (yes really)
The basic idea:
......@@ -46,44 +48,66 @@ The basic idea:
* However there would be txs with large deltawork that would help secure the chain
* Block reward was based on the deltawork
We're mostly just using multiple backlinks and deterministic ordering via a proof-of-stake esq method (where the stake is actually votes)
In SecureVote we're mostly just using multiple backlinks and deterministic ordering via a proof-of-stake esq method (where the stake is actually votes)
We're using this idea to build a scalability layer over Bitcoin - which will eventually be able to be used for arbitrary networks of smart contracts.
We actually use multiple DAGs, one for the whitelist of eligeable voters, one for each election, etc. That way they're 'disposable' - IE not needed for future elections.
My innovation was developing an algorithm to deterministically order nodes, essentially collapsing the dag into a chain.
Side note: Since all relevant quanta dags will be anchored to the Bitcoin blockchain they can safely talk to each other, and rely on each other for concensus, even though the only common history they share is via Bitcoin, not via either of their consensus protocols. This is relevant to my communication-across-chains hobby (see http://xk.io/2013/12/22/factum-exchange/ for example)
Side note: Since all relevant quanta dags will be anchored to the Bitcoin blockchain they can safely talk to each other, and rely on each other for consensus, even though the only common history they share is via Bitcoin, not via either of their consensus protocols. This is relevant to my communication-across-chains hobby (see http://xk.io/2013/12/22/factum-exchange/ for example)
Because we can run many parallel chains, we can run a new chain for every new whitelist (i.e. set of eligable voters), and a new chain for every new democracy, or election, etc. These election chains would then refer back to the whitelist chain.
Because we can run many parallel chains, we can run a new chain for every new whitelist (i.e. set of eligible voters), and a new chain for every new democracy, or election, etc. These election chains would then refer back to the whitelist chain.
This also means chains are somewhat more 'disposable' -- running a full verifier for some election doesn't require validating any other elections, and at the end of an election we can just delete the data.
This also means chains are somewhat more 'disposable' -- running a full verifier for some election doesn't require
validating any other elections, and at the end of an election we can just delete the data.
We call this sort of chain (or DAG) an 'open propriatary blockchain'. That is to say there is no privacy, but there's probably an authority deciding the rules. The idea here is that these chains are like individual smart contracts.
We call this sort of chain (or DAG) an 'open propriatary blockchain'. That is to say there is no privacy, but there's
probably an authority deciding the rules. The idea here is that these chains are like individual smart contracts.
Because quanta DAGs are anchored to Bitcoin, they are able to determine state based on other quanta DAGs, so you can establish links between them (either one or two ways). Additionally, because such a link adopts the foreign chain as part of the consensus protocol it forms a much stricter network than the 'anyone to anyone' model of Ethereum et al.
Because quanta DAGs are anchored to Bitcoin, they are able to determine state based on other quanta DAGs, so you can
establish links between them (either one or two ways). Additionally, because such a link adopts the foreign chain as
part of the consensus protocol it forms a much stricter network than the 'anyone to anyone' model of Ethereum et al.
Quanta dags are highly scalable, and are limited only by the complexity of the operations, the size of the data, and the length of time the chain exists for (they are intended to expire or remain sustainable). 100 GB in 1 day for a national election is completely plausible, for example, as the chain is killed (essentially) after the election is done. The 100GB can then be archived, and don't need to be used for any other purpose than just verifying the election in the future.
Quanta dags are highly scalable, and are limited only by the complexity of the operations, the size of the data, and
the length of time the chain exists for (they are intended to expire or remain sustainable). 100 GB in 1 day for a
national election is completely plausible, for example, as the chain is killed (essentially) after the election is done.
The 100GB can then be archived, and don't need to be used for any other purpose than just verifying the election in the future.
It is my conjecture that the majority of Bitcoin transactions will soon become anchored chains, which are just as flexible as side chains, but have the entire security of Bitcoin behind them. Many pallets, perhaps from different chains, can be combined via a merkle tree or in series, also, to futher reduce transaction burden. That will only happen once tx fee pressure forces them too, though (and 30 US cents is so cheap).
It is my conjecture that the majority of Bitcoin transactions will soon become anchored chains, which are just as
flexible as side chains, but have the entire security of Bitcoin behind them. Many pallets, perhaps from different
chains, can be combined via a merkle tree or in series, also, to futher reduce transaction burden. That will only
happen once tx fee pressure forces them too, though (and 30 US cents is so cheap).
If we can put 100,000 votes in a bitcoin tx for 30 cents, we'll happily put 1,000 votes 100 times for 30 dollars too, since that's still a negligiable cost.
If we can put 100,000 votes in a bitcoin tx for 30 cents, we'll happily put 1,000 votes 100 times for 30 dollars
too, since that's still a negligiable cost.
## Backbone - Blockchain
We use nulldata transactions (aka OP_RETURN txs) to store a 6 byte tag and the 34 byte multihash of a pallet header. Pallet headers (like pallets) are hosted via IPFS primarily, though any other protocol can be used also (like HTTP or swarm) provided the multihash is verified.
We use nulldata transactions (aka OP_RETURN txs) to store a 6 byte tag and the 34 byte multihash of a pallet header.
Pallet headers (like pallets) are hosted via IPFS primarily, though any other protocol can be used also
(like HTTP or swarm) provided the multihash is verified.
Pallet headers reference the multihash of a pallet, which is downloaded and verified by verifying all signatures.
Pallet headers also link together to form a quanta DAG (providing deterministic ordering) and ensures that even though data is primarily accessed through a DHT it will remain available. The DAG gives us a consensus mechanism that pretty much guarentees pallets will remain available.
Pallet headers also link together to form a quanta DAG (providing deterministic ordering) and ensures that even
though data is primarily accessed through a DHT it will remain available. The DAG gives us a consensus mechanism
that pretty much guarentees pallets will remain available.
### y u no use ethereum?!?!?!?!?!?
* NSA have lots of GPUs (only relevant till serenity ofc)
* PoS - still don't think it's more secure than PoW in the context of ethereum (our voting networks will be PoS effectivley, but we can introudce a rule like 'no links more than 6 blocks behind' to the pallet headers and we essentially get the best of both worlds)
* We can always switch if ethereum magically becomes more secure
* since we're talking 100s of gigs of data, Ethereum can't handle it directly (and why would we want random ethereum users to _have_ to verify our votes anyway)
* PoS - still don't think it's more secure than PoW in the context of ethereum (our voting networks will be
PoS effectively, but we can introduce a rule like 'no links more than 6 blocks behind' to the pallet headers
and we essentially get the best of both worlds)
* We can always switch if Ethereum magically becomes more secure
* since we're talking 100s of gigs of data, Ethereum can't handle it directly
(and why would we want random ethereum users to _have_ to verify our votes anyway)
and finally
Bitcoin works for us just as it is, we don't need bigger blocks or any of that. Ethereum still has a lot of work to do and we don't know what it'll look like going into the future, not nearly as much as we know what Bitcoin will look like, at least.
Bitcoin works for us just as it is, we don't need bigger blocks or any of that. Ethereum still has a lot of work to
do and we don't know what it'll look like going into the future, not nearly as much as we know what Bitcoin will look like, at least.
## Smart Phones & Apps
......@@ -103,16 +127,23 @@ Pretty much all of these problems have been solved already, so deets not include
This method is covered by AU2017900257 (Patent Pending).
Hereon out we use the term oblivious shuffle to mean a process whereby the association between a public key (used for authentication) and data the actor would like included in a set (such as a vote or bitcoin output) is removed. That is to say: their contribution is anonymised.
Hereon out we use the term oblivious shuffle to mean a process whereby the association between a public key
(used for authentication) and data the actor would like included in a set (such as a vote or bitcoin output) is
removed. That is to say: their contribution is anonymised.
Essentially we do two oblivious shuffles (using either the oblivious shuffle algorithm outlined in the CoinShuffle paper, or using a MixNet) - the first for an ephemeral voting pubkey, the second for the actual vote. The initial identity is a voter's Main Public Key which is on a whitelist maintained by the electoral authority.
Essentially we do two oblivious shuffles (using either the oblivious shuffle algorithm outlined in the CoinShuffle
paper, or using a MixNet) - the first for an ephemeral voting pubkey, the second for the actual vote. The initial
identity is a voter's Main Public Key which is on a whitelist maintained by the electoral authority.
By shuffling ephemeral voting keys first, then shuffling votes, we have the property that if an active attacker attempts to de-anonymise someone they can either obtain:
By shuffling ephemeral voting keys first, then shuffling votes, we have the property that if an active attacker
attempts to de-anonymise someone they can either obtain:
* A mapping of main public keys to ephemeral voting public keys (round 1)
* or, A mapping of ephermeal voting public keys to votes (round 2)
An active attack is not possible without alerting the voters involved in the shuffle, so if this occurs during round 1 the voters just generate new ephemeral voting keys, or if it happens in round 2 it does not de-anonymise the voter because they're already using an anonymised voting key.
An active attack is not possible without alerting the voters involved in the shuffle, so if this occurs during
round 1 the voters just generate new ephemeral voting keys, or if it happens in round 2 it does not de-anonymise
the voter because they're already using an anonymised voting key.
If something goes wrong a 'blame game' is played, which is described in more detail in the CoinShuffle paper.
......
......@@ -3,7 +3,7 @@
if [[ $VERIFY_SECRET_KEY ]]; then
echo "Producer starting... with not_before: $NOT_BEFORE"
./producer_wrapper.py --secret-key $VERIFY_SECRET_KEY --btc-rpc-user $RPCUSER \
--btc-rpc-password $RPCPASSWORD --fee-rate 0.00201 --magic-bytes $MAGICBYTES \
--btc-rpc-password $RPCPASSWORD --fee-rate 0.00225 --magic-bytes $MAGICBYTES \
--box-per-pallet $BOX_P_PALLET --pallet-n $NUM_PALLETS --pallet-period $PALLET_PERIOD \
--not-before $NOT_BEFORE
else
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment