The Emerald Benchmark is based on the well understood use case of international payments. The SEPA ICT requirements provide an easily consumable set of real requirements for international payments. The key requirement is that payments must be sent, received and acknowledged back to the sender in less than 10 seconds.
Ulster Bank, a wholly owned subsidary of Royal Bank of Scotland, piloted a distributed clearing and settlement mechanism (CSM) to meet the requirements for the major banks of Ireland. It became known as Emerald. As part of this pilot they asked GFT to independently test the CSM. The results were made public domain in order to reduce the fear, uncertainty and disinformation in late 2016 around performance and scalability of distributed ledger technologies (DLT).
Emerald code was made open source to provide a reference implementation of a CSM using DLT.
The Emerald Benchmark aims to define a common performance test for distributed ledger technologies seeking to prove their latency, performance and scalability. This common performance test allows comparison against a number of existing international payment benchmarks.
No single point of failure : Test must continue if any single component (including infrastructure) of the network is removed
Nodes geographically distributed : Nodes must be evenly located across different regional data centres
Minimum latency is maintained : Payments must be received and acknowledged in a minimum round trip time
realistic payload : Payment payload must use mock ISO20022 payment pacs.008 and acknowledge messages pacs.002 or equivalent to simulate real payments
bi-directional payments : All Payment nodes must send and receive payments randomly to other nodes in the network
seconds to receive a transaction's acknowledgement (ack)
transaction acks received per second
number of nodes creating transactions
Rate x Scale
In order to ease comparison we would recommend the following visuals:
Graph showing Average Rate vs Scale (where minimum latency is maintained)
Graph showing Scale, Latency and Rate (average, min and max) vs time
The test is based on a set of geographically distributed nodes sending payments to one another. Each payment must be sent and an acknowledgement received back to the sender in a minimum round trip time (latency). The test should only increase the rate at which payments are sent once the minimum latency can be maintained.
parameters - MinLatency, MinRate, MaxRate Rate = MinRate Send transaction messages every Rate seconds to another random node on the network On receiving transaction ack: latency = transaction ack received timestamp - transaction sent timestamp record latency and rate if latency > MinLatency then decrease Rate but not lower than MinRate else increase Rate but not higher than MaxRate
A project should show the badge if it can evidence completing one of the above scenarios. eg a green should be shown if the test showed evidence of a Scale and 'Rate' of >1000 while maintaining a minimum Latency of < 10 seconds. Conversely the red badge should be shown if the test showed evidence of a Scale and Rate of <1000 or the minimum Latency failed to maintain <10 seconds.