README.md 18.2 KB
Newer Older
Michel Schudel's avatar
Michel Schudel committed
1 2
# CraftsCoin workshop

3
In this workshop we will build a blockhain ourselves. The smart contract that we are going to make is called CraftsCoin, a crypto coin that we can use within Craftsmen to pay for dinners, declarations and the like.
Michel Schudel's avatar
Michel Schudel committed
4

5
Step by step we are going to build up the blockchain. We are going to implement the following functionality:
Michel Schudel's avatar
Michel Schudel committed
6

7 8 9 10 11
- The SmartContract, in our case a simple transaction
- The blockchain where our contracts are stored
- Mining; or build a new block
- Validation, check whether the blockchain is still in order
- Connecting to the peer-to-peer network (your colleagues!) So that we can exchange coins together.
Michel Schudel's avatar
Michel Schudel committed
12

13
The following aspects will be out of scope for this workshop: 
14

15 16
- Collisions, what happens if multiple blocks are mined at the same time.
- Asymmetric encryption due to wallets / addresses. A wallet is identified by your first name.
17

Michel Schudel's avatar
Michel Schudel committed
18 19
## Requirements

20
- Java 8 (*no Java 9, this clashes with the current version of SpringBoot*)
Michel Schudel's avatar
Michel Schudel committed
21
- Intellij / Eclipse IDE
22
- NodeJS (*version 8.9.4 or higher*).
Michel Schudel's avatar
Michel Schudel committed
23

24
## Initial build
Michel Schudel's avatar
Michel Schudel committed
25

26
After you have checked out the project from GitHub, you first have to put your mining walletId. Therefore, modify the property `miningWalletId` in` application.properties` to your own first name.
Michel Schudel's avatar
Michel Schudel committed
27

28
After this you have to build the project once:
Michel Schudel's avatar
Michel Schudel committed
29

30
`gradlew build`
Michel Schudel's avatar
Michel Schudel committed
31

32
This takes about 2 minutes. You can now run the project:
Michel Schudel's avatar
Michel Schudel committed
33

34
`gradlew bootRun`
Michel Schudel's avatar
Michel Schudel committed
35

36
This starts a SpringBoot application on port 8080 including GUI. Go to http://localhost:8080/index.html to see the UI. Here you see the following tabs:
Michel Schudel's avatar
Michel Schudel committed
37

38 39 40 41
1. *Pending transactions*. Here you see all transactions that are not yet included in the blockchain, and you can start a new transaction here.
2. *Blockchain*. Here you see the current blockchain, and you can start a mining action here.
3. *Peers*. Here is an overview which peers are all connected to this node.
4. *Wallet*. Here you can view a wallet. Remember that a wallet is not something "private", everyone can derive a wallet from someone else by just collecting his or her transactions.
Michel Schudel's avatar
Michel Schudel committed
42

43
For this the following endpoints are used:
Michel Schudel's avatar
Michel Schudel committed
44

45 46 47 48 49 50
* `GET /api/pending transactions`
* `POST /api/newtransaction`
* `GET /api/blockchain`
* `GET /api/peers`
* `POST /api/mine`
* `GET /api/wallet/{walletId}`
51

52
Of course you can also approach these API's through Postman. The Frontend is there only for convenience.
Michel Schudel's avatar
Michel Schudel committed
53

54
There is also a swagger UI that you can use: http://localhost:8080/swagger-ui.html
Michel Schudel's avatar
Michel Schudel committed
55

56
`Ctrl + C` stops the process.
Michel Schudel's avatar
Michel Schudel committed
57

58
#### Fast turnaround
Michel Schudel's avatar
Michel Schudel committed
59

60
During the development of your blockchain backend it is easier to just let the frontend run continuously and just restart your backend when needed.
Michel Schudel's avatar
Michel Schudel committed
61

62
Start your *frontend* as follows:
63

64
`frontend/ng serve-pc proxy.conf.json` or `frontend/npm start`
Michel Schudel's avatar
Michel Schudel committed
65

66
You can (restart) your * backend * by means of:
Michel Schudel's avatar
Michel Schudel committed
67

68
`gradlew assemble bootRun`
69

70 71
Or even better, from your IDE. The project uses the SpringBoot devTools.
After a code change, you can hit `Ctrl + F9` (in IntelliJ) to activate the code changes.
Michel Schudel's avatar
Michel Schudel committed
72

73
## Building the block
Michel Schudel's avatar
Michel Schudel committed
74

75
The project contains a `Block` class that represents a block in the blockchain. Fill this with the following fields. Please adhere strictly to the field names:
Michel Schudel's avatar
Michel Schudel committed
76

77
- `index` (long). This is the so-called BlockHeight and indicates the distance from the genesis block.
Michel Schudel's avatar
Michel Schudel committed
78
- `timestamp` (long)
79 80 81
- `transactions`: list of` Transaction` objects.
- `previousHash` (string). The hash of the previous block. This gives the blockchain immutability.
- `proof` (long). The proof-of-work of a mining action, more about this later.
Michel Schudel's avatar
Michel Schudel committed
82

83
Make getters and setters so you can fill create and read the block.
84

85
Also generate a `toString()` method with all fields so that the Block object is legible in any logging.
86

87
## Initializing the blockchain
Michel Schudel's avatar
Michel Schudel committed
88

89
In the `Blockchain` class, add a private field `chain` of type `List<Block>`. Initialize it with a new `ArrayList`.
Michel Schudel's avatar
Michel Schudel committed
90

91 92 93
When we start the application, we do not have any blocks yet. 
That is why it is important to make a first block when starting the application,
the so-called *genesis* block.
Michel Schudel's avatar
Michel Schudel committed
94

95 96
Implement the static factory method `create` on the `Blockchain` class that returns a `Blockchain`.
This method should create a new blockchain with a first block. 
Michel Schudel's avatar
Michel Schudel committed
97
Implement this function by constructing a new `Blockchain` object with a first block (add it to the `chain` list by implementing and calling the `addBlock` method):
Michel Schudel's avatar
Michel Schudel committed
98 99

- `index`: 0
100 101 102 103
- `timestamp`: (now)
- `transactions`: empty list.
- `previousHash`: 0. There is no previous block.
- `proof`: arbitrary number, eg `100`. This block is not mined, so the proof-of-work is made-up here.
104

105
Now that we have the code for creating an initial blockchain, we should actually create it.
Michel Schudel's avatar
Michel Schudel committed
106

107 108 109
- Create a new private field `blockchain` in the class `BlockChainService`. 
- Implement the `initializeBlockchain` method so that the `blockchain` field gets assigned the result of calling the static factory method `create`
- Let the `retrieveBlockChain` function return the `blockchain` field so we can get it in a browser..
Michel Schudel's avatar
Michel Schudel committed
110

111
Test via the UI or PostMan (GET to http://localhost:8080/api/blockchain) if you see an initial blockchain.
Michel Schudel's avatar
Michel Schudel committed
112

Michel Schudel's avatar
Michel Schudel committed
113
## Transactions
Michel Schudel's avatar
Michel Schudel committed
114

115
Before we can mine our first block, we need to be able to create transactions that will end up in the block. For this workshop, we will implement a simple coin transaction.
Michel Schudel's avatar
Michel Schudel committed
116

117
Implement the `Transaction` class as follows, strictly adhering to the field names:
Michel Schudel's avatar
Michel Schudel committed
118

119 120 121 122
- fields `id` (UUID),` from` (String), `to` (String),` amount` (BigDecimal)
- In the constructor, the `id` field must be initialized with a new UUID (`UUID.randomUUID();`)
- Create or generate `equals` and `hashCode` functions, *only* based on `id`. In other words, 2 transaction objects are equal to each other if the id's match.
- Create or generate a `toString()` function so the object is legible.
123

124
## The transaction pool
Michel Schudel's avatar
Michel Schudel committed
125

126 127
We need a place where we can temporarily store our transactions so that we can
add these transactions to the blockchain later. This is similar to the MemPool concept in BitCoin.
Michel Schudel's avatar
Michel Schudel committed
128

129
Implement the `TransactionPool` class:
Michel Schudel's avatar
Michel Schudel committed
130

131
- Add a private field `Set<Transaction>` `currentTransactions`
Michel Schudel's avatar
Michel Schudel committed
132 133
- Implement method `addTransaction` that adds a new `Transaction` to the pool
- Implement method `getAllTransactions` that returns an Unmodifiable set of transactions.
134

135
In the `BlockChainService` there is a` createTransaction` function that receives new transaction data from the frontend and returns the blockheight of the block where the transaction will be entered.
136

137 138 139
- Implement the body of this function so that the transaction is added to the transaction pool and the blockheight (the index, long type) of the next block is returned.
- Have the `getPendingTransactions` method on the` BlockchainService` return all transactions from the pool by calling `getAllTransactions` on the` TransactionPool` class.
- Test through the frontend, or Postman, that the REST services `/api/createtransaction` and `/api/pendingtransactions` now work properly.
Michel Schudel's avatar
Michel Schudel committed
140

141
## Mining a new block
Michel Schudel's avatar
Michel Schudel committed
142

143 144
Now that we have the transaction pool working, it's time to mine a new block!
Mining of a block should require some effort. In our model it also yields 10 free CraftCoins!
Michel Schudel's avatar
Michel Schudel committed
145 146 147

### Proof of work

148
First we have to show that we have done the mining work with a new proof of work. This proof must be difficult to create, but easy to validate. Other nodes in the network must be able to check whether a block is valid.
Michel Schudel's avatar
Michel Schudel committed
149

150
Implement the `proofOfWork` function on the` BlockChain` class:
Michel Schudel's avatar
Michel Schudel committed
151

152 153 154
1. Take the proof of work of the most recent block in the blockchain
2. Create a new proof of work: 0.
3. Concatenate the old and the new proof. So for block 0 and 1, you would have '00' to start with.
Michel Schudel's avatar
Michel Schudel committed
155
4. Create a hash of the string using `DigestUtils.sha256Hex(string)`
156 157
5. Does the hash start with `0000`? Then the new proof of work is valid. Return it.
6. Does the hash not start with `0000`? Increment the new proof of work by 1 and return to step 3.
Michel Schudel's avatar
Michel Schudel committed
158

159
BitCoin uses the HashCash algorithm, which is very similar to the algorithm above. The end condition is only much stricter (lots more zeros), so it takes a long time before a new proof of work is found.
Michel Schudel's avatar
Michel Schudel committed
160

161
### Creation of the new block
162

163
Now that we've created a function to find the new proof of work, it's time to create a new block.
164

165 166 167 168 169 170 171 172 173
Implement the `mine()` method on the `BlockChainService` class as follows:
- Collect all transactions in the transaction pool by calling getAllTransactions().
- Create a new transaction as a reward for the miner:
    - `from`: "", there is no sender.
    - `to`: use the `miningWalletId` field (yourself) here.
    - `amount`: 10
- Call the `mineNewBlock` method on the `Blockchain` class. We'll implement that method in a second.
- Empty the transaction pool by calling and implementing the `clearTransactions()` function on the `TransactionPool` class. After all, all transactions are now in the new block.
- Return the Block that has been returned by the `mineNewBlock` method.
174

175
Now, implement the `mineNewBlock (..)` method on the `Blockchain` class, as follows:
176

177 178 179 180
- Determine the new proof-of-work with the `proofOfWork` method that you have just implemented.
- Create a new Block object:
    - `index` (long) -> index of the most recent block in the blockchain + 1.
    - `timestamp` (long) -> now
181
    -  set of transactions (a merged set containing the transactions from the pool and the reward)
Michel Schudel's avatar
Michel Schudel committed
182
    - `previousHash` (string). The hash of the most recent block. We will calculate the hash over the json representation of that block. There is a utility method in the `blockchain` class:`createHashOf(Block block)` that you can use for this.
183 184
    - `proof` (long). The new proof-of-work.
- Add the new block to the chain.
185

186 187


188
To test this new functionality:
189

190 191 192 193
- Add a transaction via the UI or a POST to http://localhost:8080 /api/newtransaction
- Call the mine function via the UI or a POST request to http://localhost:8080/api/mine
- Obtain the blockchain via the UI or via: http://localhost:8080/api/blockchain. You now see that a new block has been added.
- Also retrieve the pending transactions via http://localhost:8080/pendingtransactions. This list should now be empty.
194

195
You now have a fully functional Blockchain node that has a blockchain, a transaction pool, and mining functionality!
196

Michel Schudel's avatar
Michel Schudel committed
197 198


199
## Persistence
Michel Schudel's avatar
Michel Schudel committed
200

201 202 203
Right now, your blockchain exists in memory only. Therefore,
it is useful to write the blockchain to disk. 
There is a `GenericRepository` class that has a `load` and `save` method you can use to store arbitrary data.
Michel Schudel's avatar
Michel Schudel committed
204
- Modify the `initializeBlockchain` method on the`BlockChainService` bean so that
205 206 207 208 209
 you first try to load the blockchain from disk with `genericRepository.load(Blockchain.class)`. 
 If this does not work (loading returns an empty optional),
  then initialize the blockchain with a genesis block.
   Save the blockchain immediately with `genericRepository.save(...)`.
- Also save the blockchain after adding a new block to the blockchain, (in the `mine()` function).
Michel Schudel's avatar
Michel Schudel committed
210

211
Test again with the UI or Postman if the blockchain is actually saved. A json file (`hostport-blockchain.json`) is created in the root of the project.
Michel Schudel's avatar
Michel Schudel committed
212 213


214
## A distributed blockchain
215 216 217 218 219
A Blockchain on just 1 node is not very trustworthy, of course.
 That is why we are now going to connect multiple nodes to each other.
 Initially we will do this with multiple nodes on your own laptop,
  but of course it would be nice if we could also connect to nodes on machines of your fellow
  workshop participants!
Michel Schudel's avatar
Michel Schudel committed
220

221 222 223 224 225
- To make your node accessible to others: first check whether
  the ip address found by the application (found in the system output `node name of this node:xxxx`)
   is indeed the dns address was handed out by the router and can be accessed by others. 
   (to determine the correct dns address, try `ipconfig` in Windows).
- If this is not the case, uncomment andset the property `node.ipAddress` to the correct value in the` application.properties`.
Michel Schudel's avatar
Michel Schudel committed
226

227 228


229
### Connection to the network
230

231
The `Network` class already contains all the functionality to communicate with the peer-to-peer network.
232

233 234
- First start your node on port 8080.
- Start a second node on port 9000, using `gradlew bootRun -Pport=9000`. If your node has no peers yet, it first tries to connect to a local node on port 8080. (This can also be modified by the properties `bootstrap.peer.host` and `bootstrap.peer.port` in the `application.properties`).
235

236
You now see that both nodes record each other as a peer in a json file in the root: `xxxxx-peers.json`. You can also see this in the peers tab in the UI.
Michel Schudel's avatar
Michel Schudel committed
237

238
### Reaching consensus
239

240
It is important that all nodes in the network agree on the current state of the blockchain.
241

242 243 244 245 246
- When starting up a node, all blockchains must be retrieved from all known peers;
- The node's own blockchain should be replaced by the blockchain of the peer if:
    - The blockchain of the peer is valid;
    - The blockchain of the peer more blocks then has its own blockchain;
    - If both are equally long: if the last block of the peer's blockchain of the peer has an older timestamp than the last block of its own blockchain.
247

248
For this we need to build the following:
249

250
#### Validating a blockchain
251

252
Implement the `boolean isValid()` method on the `Blockchain` class that tests whether the blockchain is valid. A blockchain is valid if:
253

254
- The `previousHash` field of each block is equal to the hash of the previous block;
255 256 257 258
- The hash of the string (`proof` of block n-1) + (string of` proof` of block n) starts
 with "0000". So if the proof of block n-1 100 and the proof of block n is 250,
  the hash of "100250" must start with "0000". 
  This is the actual verification of the proof-of-work done by the node that mined the block.
259

260 261
It goes without saying that this method should always yield 'true' on
 the node's own blockchain. You can test this by doing a GET at http://localhost:8080/api/valid.
262

263
#### Determine which blockchain is better
264

265
Implement the boolean `isInferiorTo (..)` method on the `BlockChain` class as follows:
266

267
- Is the other blockchain valid (using `isValid ()`)
268 269 270
- Is the other blockchain longer? Replace this node's blockchain with that one.
- If both blockchains are the same length, check the timestamps as described above. If the peer's blockchain's last block has an older timestamp,
  replace this node's blockchain with that one.
271

272
#### Reaching consensus
273

274
Now it's time to put validation and blockchain superiority together.
275

276
Implement the method `void reachConsensus()` method on the `BlockChainService` class:
277

278
- Retrieve blockchains from all peers through `network.retrieveBlockchainsFromPeers ()`
279 280 281 282 283 284
- Determine which blockchain is the best blockchain acccording to the criteria above and replace
- Save the blockchain to disk using the generic repository. 
- Remove all transactions that are present in the new blockchain from the transaction pool,
  by implementing and calling the `clearTransactions(List<Transaction> transactions)` method 
  on the `TransactionPool`. (** note: ** In the case of a
   collision, we do risk that transactions are lost, but this is out of scope for this workshop.)
285

286
Remove the blockchain file from the node which ran on port 9000. Now start the node on port 8080 (using `gradlew boot Run`). Enter a transaction and mine a new block. Then start a node on port 9000 (using `gradlew bootRun -Pport = 9000`). View the blockchain status on both nodes. It should now be the same on both nodes. After all, the node at 9000 had a blockchain that was shorter than that at node 8080, so that blockchain was taken over at startup time.
287

288
** Question **: * is it really necessary to collect all blockchains in their entirety? How would you optimize the achievement of consensus? *
Michel Schudel's avatar
Michel Schudel committed
289

290
### Distribution of transactions
291 292
If a new transaction is entered on a node, it must be propagated to the rest of the network.
 After all, everyone must have the opportunity to mine a block with new transactions.
293

294
Implement the `newTransactionReceived (...)` function on the `BlockChainService`:
295

296
- If the incoming transaction is not yet in the transaction pool (compare `transactionId`), then add the transaction to the transaction pool, and make a call to` network.notifyPeersOfNewTransaction (..) `.
297
- This makes the transaction known to the other transaction pools in the network.
298
- Also add the call to `network.notifyPeersOfNewTransaction (..)` to the `createNewTransaction()` method. After all, transactions made on this node must also be made known to other peers.
299

300
To test this you can make a new transaction on node 8080 or 9000. Both transactions must then be known shortly in both (or all) nodes.
301

302
### Distribution of blocks
303
The same applies to newly mined blocks: these must also be further propagated in the network.
304

305
- Add a call to `network.notifyPeersOfNewBlock (..)` at the end of the `mine ()` method on the `BlockChainService` class. This sends new mined blocks to the network.
306 307
- The `BlockChainService` class also contains a method` newBlockReceived (Block block, String sourcePeer) `. This method is called when a new block arrives on the interface. 
Implement this function:
Michel Schudel's avatar
Michel Schudel committed
308 309
- check whether the new block is valid with respect to the node's own blockchain. Implement and use the method `isNewBlockValid` on the `Blockchain` class. This is the same check as the validation of the blockchain itself: is the proof of work of the block correct, and does the previousHash match the hash of the last block?
- if the block is valid, add block to the blockchain (with a method `addBlock` on the `Blockchain` class);
310 311 312
- call to `network.notifyPeersOfNewBlock (..)` to propagate the block further in the network;
- store blockchain using the `genericRepository.save(newlyReceivedBlock)`.
- Remove all transactions that were in the last block from the transaction pool. After all, they have already been mined.
313

314
To test this you can start a mining action on one of the nodes. The other node must then get the new mined block and add it to its own blockchain.
315

316 317 318 319
### Connecting to remote nodes
At this point we have a fully working, distributed blockchain!
Now, try to connect with the node of a fellow participant in the workshop by entering their
ip address in the list of peers. *Note: clear the *-blockchain.json files before you attempt to do this, so you can test a happy case with your peers without any collisions.*
320

Michel Schudel's avatar
Michel Schudel committed
321
## Bonus: wallet
322 323
A wallet is nothing more than a sum of all transactions that you as a user 
have ever been involved with. Still got some energy left? 
Michel Schudel's avatar
Michel Schudel committed
324

325
Implement the `getWallet` method in the `WalletService` class, that it collects all confirmed and unconfirmed transactions for a certain name, and returns it as a Wallet object.