Commit d847a227 authored by Sharon Urban's avatar Sharon Urban

factored out main loop and API, persisting calculated audit paths, updated readme

parent 81c73b3b
......@@ -24,13 +24,13 @@ The `config.yaml` file contains the following configurable items:
- `nodeApi`: API address of the Zen Node used by the oracle service
- `api`: Incoming HTTP requests endpoint
- `tickers`: A list of tickers to fetch from Intrinio and commit in the blockchain
- `interval`: In hours, the interval between data fetching requests
- `acs`: Active Contract Set settings:
- `activate`: When activating the contract, this is the amount of blocks to have it activated for
- `threshold`: If the contract is found to be active for less blocks than specified here, 'extend' will be triggered
- `extend`: The amount of blocks to extend contract activation for
- `contract`: Oracle code file name
- `chain`: Specifies the chain (main, test, local)
- `derivationPath`: Specifies the HD derivation path used to perform authenticated contract execution
# Wallet Setup
......@@ -49,14 +49,14 @@ Lock some funds (Zen) to it. This will be used to activate and extend the oracle
Refer to [examples](https://github.com/zenprotocol/contracts) repository for Oracle contract source code. Make the source file available in the filesystem.
2. Authentication
As the orcale contract incorporate authenticated execution, a public key needs to be extracted from the node's wallet and embedded into the body (code) of the contract.
As the oracle contract incorporates authenticated execution, a public key needs to be extracted from the node's wallet and embedded into the body (code) of the contract.
Use the following command to extract a public key:
```
zen-cli publickey "m/44'/258'/0'/3/0" <ZEN-WALLET-PASSWORD>
```
> The `m/44'/258'/0'/3/0` [path](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki) is used by the oracle to perform an authenticated contract execution transaction.
> The `m/44'/258'/0'/3/0` [path](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki) is used by default by the oracle to perform an authenticated contract execution transaction.
Then, in the source file, in line `let oraclePubKey = ""`, embed the result.
......@@ -68,58 +68,24 @@ Lock some funds (Zen) to it. This will be used to activate and extend the oracle
```
3. Packing
Using [Zebra, Zen SDK tool](https://github.com/zenprotocol/ZFS-SDK), pack the updated contract:
Using [Zebra, Zen SDK tool](https://github.com/zenprotocol/ZFS-SDK), pack the updated contract, for example:
```
zebra --pack Oracle.fst
```
Update `config.yaml` to refer to the generaed, packed contract:
Update `config.yaml` to refer to the generated, packed contract, for example:
```
contract: Zf57a7bf099b4a1d13a22c0ff5a3706297d7ca74f9aca6cc78222175030bbf997.fst
```
# Running
Use systemd to manage the deamon. See `scripts/zen-oracle.service` for environment variable configuration.
Alternatively, set the needed environment variable and launch `zen-oracle.exe`:
Set the following environment variable and launch `zen-oracle.exe`
```
export intrinio_user_name=<INTRINIO-USERNAME>
export intrinio_password=<INTRINIO-PASSWORD>
export zen_account_password=<ZEN-WALLET-PASSWORD>
./zen-oracle.exe
```
# Wiping
Pass the 'wipe' command line switch to wipe all commitment records from the Oracle service's database
# HTTP API
The oracle service responds to HTTP requests to the following routes:
> `<DATE>` format: `yyyy-MM-dd`
- `/proof?date=<DATE>&ticker=<TICKER>`
outputs a YAML represented proof-of-inclusion of oracle data which is used to validate against the data committed to in the blockchain, for example:
```
Hash: bdbac8bf395a74d45e34c04d3661ebd607dcd21a9f0eec52264b4f734ee8e288
Index: 0
AuditPath:
- 6e7ce38340711d436aae480719c7c30b5bc488476bb452591b0c8dedb30149bf
- dacf022e7b964b27eb40b45c3c0c69a6db439a09f7ea8c1f819b65f7ae7d5a7d
- 411784e894b3d6adb3091a71d119d95fcc3c35fcb3c7239ec45e778bc86359b8
- 2ab9b3f0f2014218623f84279fb26027a9d60504e67ffb7ba4b6fe991cf73d45
- 776bcba2254cccaff61bc22f58f8da1a7e015dd07ec8c0d0e577d11cc14a293b
```
- `/data?date=<DATE>&ticker=<TICKER>`: outputs the price of TICKER for DATE
- `/data?date=<DATE>`: lists ticker-price values for DATE in Json format
- `/data?ticker=<TICKER>`: lists date-price values for TICKER in Json format
# TODO
Implement scheduling
\ No newline at end of file
```
\ No newline at end of file
# zen-oracle systemd unit
[Unit]
Description=Zen Oracle
After=network.target
[Service]
Environment=intrinio_user_name=<intrinio_user_name>
Environment=intrinio_password=<intrinio_password>
Environment=zen_account_password=<zen_account_password>
Type=simple
WorkingDirectory=/home/user_name/
ExecStart=/usr/bin/mono /home/user_name/zen-oracle.exe
Restart=on-failure
[Install]
WantedBy=multi-user.target
......@@ -3,8 +3,8 @@ module DataAccess
open MongoDB.Bson
open MongoDB.Driver
open MongoDB.FSharp
open System
open Infrastructure.Timestamp
open Consensus.Hash
[<Literal>]
let ConnectionString = "mongodb://localhost"
......@@ -15,20 +15,24 @@ let DbName = "oracle"
[<Literal>]
let CollectionName = "data"
type Item = {
ticker: string
price: decimal
leaf: Hash
path: Hash[]
}
type Data = {
id : BsonObjectId
id: BsonObjectId
timestamp: Timestamp
data : (string * decimal) array
data: Item array
}
let client = MongoClient(ConnectionString)
let db = client.GetDatabase(DbName)
let collection = db.GetCollection<Data>(CollectionName)
let wipe() =
collection.DeleteMany(fun _ -> true)
let insert data timestamp =
let insert timestamp data =
{
id = MongoDB.Bson.BsonObjectId(MongoDB.Bson.ObjectId.GenerateNewId())
timestamp = timestamp
......
module Main
open System
open Result
open Consensus
open Serialization
open FsBech32
......@@ -9,47 +7,48 @@ open Json
open Util
open Contract
open Config
open DataAccess
open Serialization
open Timestamp
[<EntryPoint>]
let main args =
Server.init config.yaml.api
if Array.exists ((=) "wipe") args then
DataAccess.wipe()
|> ignore
while true do
result {
do! Contract.ensureActive()
let closingTime = currentClosingTime()
let! data = Intrinio.fetch()
let data =
data.Data
|> Array.map (fun root -> root.Identifier, root.Value)
do! data
|> Array.map (Merkle.hashLeaf closingTime)
|> Array.toList
|> MerkleTree.computeRoot
|> Hash.bytes
|> Zen.Types.Data.data.Hash
|> Data.serialize
|> Base16.encode
|> contractExecuteRequestJson contractId "Add" false "m/44'/258'/0'/3/0" Array.empty
|> execute "wallet/contract/execute"
DataAccess.insert data closingTime
info <| sprintf "data successfully persisted. %d record(s) found" (DataAccess.count())
}
|> mapError error
|> ignore
info "waiting..."
let main _ =
result {
let closingTime = currentClosingTime()
Threading.Thread.Sleep (1000 * 60 * 60 * config.yaml.interval)
0
\ No newline at end of file
do! Contract.ensureActive()
let! data = Intrinio.fetch()
let hashes =
data
|> Array.map (Merkle.hashLeaf closingTime)
|> Array.toList
do! hashes
|> MerkleTree.computeRoot
|> Hash.bytes
|> Zen.Types.Data.data.Hash
|> Data.serialize
|> Base16.encode
|> contractExecuteRequestJson contractId "Add" false config.yaml.derivationPath Array.empty
|> execute "wallet/contract/execute"
data
|> Array.mapi (fun index (ticker, price) ->
{
ticker = ticker
price = price
leaf = Merkle.hashLeaf closingTime (ticker, price)
path = MerkleTree.createAuditPath hashes index |> List.toArray
}
)
|> DataAccess.insert closingTime
}
|> function
| Error e ->
eprintfn "%s" e
1
| Ok _ ->
printfn "data successfully persisted. %d record(s) preset" (DataAccess.count())
0
module Server
open FsNetMQ
open FSharp.Control.Reactive
open Infrastructure.Http
open Consensus
open FSharp.Data
module Actor = FsNetMQ.Actor
let init address =
Actor.create (fun shim ->
use poller = Poller.create ()
use observer = Poller.registerEndMessage poller shim
use httpAgent = Server.create poller address
use observer =
Server.observable httpAgent |>
Observable.subscribeWithError (fun (request,reply) ->
let reply =
function
| None ->
TextContent "Not found"
|> reply StatusCode.NotFound
| Some content ->
reply StatusCode.OK content
//let (|Param|) query key = Map.tryFind key
match request with
| Get ("/proof", query) ->
match Map.tryFind "date" query, Map.tryFind "ticker" query with
| Some date, Some ticker ->
Timestamp.parse date
|> Option.bind DataAccess.find
|> Option.bind (fun data ->
data.data
|> Array.map fst
|> Array.tryFindIndex ((=) ticker)
|> Option.map (fun index ->
let hashes =
data.data
|> Array.map (Merkle.hashLeaf data.timestamp)
|> Array.toList
[
"Time",
data.timestamp
|> Timestamp.normalize
|> sprintf "%iUL"
"Ticker", ticker
"Price",
index
|> Array.get data.data
|> snd
|> Price.normalize
|> sprintf "%dUL"
"Hash",
Merkle.hashLeaf data.timestamp (index |> Array.get data.data)
|> Hash.toString
"Index",
sprintf "%iul" index
"AuditPath",
MerkleTree.createAuditPath hashes index
|> List.map Hash.toString
|> String.concat "\n - "
|> sprintf "\n - %s"
]
|> List.map (fun (key, value) -> sprintf "%s: %s" key value)
|> String.concat "\n"
|> TextContent
)
)
| _ -> None
| Get ("/data", query) ->
match Map.tryFind "date" query, Map.tryFind "ticker" query with
| Some date, Some ticker ->
Timestamp.parse date
|> Option.bind DataAccess.find
|> Option.map (fun data -> data.data)
|> Option.bind (Array.tryFind (fst >> (=) ticker))
|> Option.map (snd
>> JsonValue.Number
>> JsonContent)
| Some date, None ->
Timestamp.parse date
|> Option.bind DataAccess.find
|> Option.map (fun data ->
data.data
|> Array.map (fun (ticker, value) ->
JsonValue.Record [|
"ticker", JsonValue.String ticker
"value", JsonValue.Number value
|])
|> Seq.toArray
|> JsonValue.Array
|> JsonContent
)
| None, Some ticker ->
DataAccess.take 10
|> Seq.toList
|> Infrastructure.Option.traverseM (fun item ->
Array.tryFind (fst >> (=) ticker) item.data
|> Option.map (fun (_, value) -> item.timestamp, value)
)
|> Option.map (Seq.map (fun (timestamp, value) ->
[|
"date", timestamp
|> Timestamp.toString
|> JsonValue.String
"value", JsonValue.Number value
|]
|> JsonValue.Record))
|> Option.map (
Seq.toArray
>> JsonValue.Array
>> JsonContent)
| _ -> None
| _ -> None
|> reply
) (fun error ->
printfn "error %A" error
raise error
)
Actor.signal shim
Poller.run poller
)
|> ignore
\ No newline at end of file
......@@ -47,3 +47,8 @@ let chain =
| "main" -> Consensus.Chain.Main
| "test" -> Consensus.Chain.Test
| _ -> Consensus.Chain.Local
let (|UInt64|_|) str =
match UInt64.TryParse(str) with
| (true, i) -> Some i
| _ -> None
\ No newline at end of file
......@@ -20,10 +20,10 @@ tickers:
- QCOM
- AABA
- TSLA
interval: 1
acs:
activate: 5
threshold: 5
extend: 5
contract: Oracle.fst
derivationPath: "m/44'/258'/0'/3/0"
chain: test
\ No newline at end of file
......@@ -49,7 +49,6 @@
<Compile Include="Util.fs" />
<Compile Include="DataAccess.fs" />
<Compile Include="Merkle.fs" />
<Compile Include="Server.fs" />
<Compile Include="Json.fs" />
<Compile Include="Intrinio.fs" />
<Compile Include="Contract.fs" />
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment