[BUG] Outbound hash has not been correctly mapped to Observed TxIn
There is an issue reported in #thornode-dev channel https://discord.com/channels/838986635756044328/961569089501954058
When user swap DAI -> BTC , the inbound transaction will be observed, send to thornode https://thornode.ninerealms.com/thorchain/tx/970AFA4D6E15017995B89F08F14EAE581B166A076660CB2C1D8B2D137360CED9/signers
{
"tx_id": "970AFA4D6E15017995B89F08F14EAE581B166A076660CB2C1D8B2D137360CED9",
"tx": {
"tx": {
"id": "970AFA4D6E15017995B89F08F14EAE581B166A076660CB2C1D8B2D137360CED9",
"chain": "ETH",
"from_address": "0xdc334fb6d445943f80615f4fccabd1c18a355e88",
"to_address": "0x0114eA17e85767940fbe3f5da6384b41BB0E0B82",
"coins": [
{
"asset": "ETH.DAI-0X6B175474E89094C44DA98B954EEDEAC495271D0F",
"amount": "200000000000"
}
],
"gas": [
{
"asset": "ETH.ETH",
"amount": "372433"
}
],
"memo": "=:BTC.BTC:bc1qrrqlp2056yyfpcev45yuak43jetf4gqy4zw72y::thor1a427q3v96psuj4fnughdw8glt5r7j38lj7rkp8:100"
},
"out_hashes": [
"0000000000000000000000000000000000000000000000000000000000000000"
],
"block_height": 14533959,
"signers": [
"thor17e4uzrfqjxfewldwpqszkcelkj9l2c3qkj3k60",
"thor1c2z2lp2fdjjsmmxymlnjuadw7wae7n2jrgwcu9",
"thor1yedw5cuz4l93wyj58wdn7s7z8qppfvusl3c66p",
"thor1jaegcehmsx952h9zak3gvetewtscu5vzzt8hpq",
"thor1ufyppfz6q32yxtymfx23zdr2d6gek0m72zlmm2",
"thor1rc936rg4ffp6jn9ddcmwca75crgarsdnzjuy3a",
"thor1hx3gayvwx0j92nf0ev6c837km4yg4kdk0w6fjl",
"thor1v6664w08cnvy9cfltw4pzzfttvwljgawmaw7yc",
"thor1ee5ec0mnvhqvu84lgtxpgcc4yrdhalylfty246",
"thor1newujzwu4aek3x0u536dx0yt0ayt3ffa7x247x",
"thor144rj67r7tvzuxa39xjrln9mefvhkra290mrp5t",
"thor1rf5dc0gq4vsaza6q66xw5297jdpx2xly7raks5",
"thor1vevr4l9khpqm5zktk4uqaexv07gewhc6elt3l5",
"thor1xd4j3gk9frpxh8r22runntnqy34lwzrdkazldh",
"thor10697ffyya4fddsvpwj6crfwe06pxkxkmdl8kev",
"thor1zn6jxyjgasd5z5j3yvxvyxseprvd4mvss74v27",
"thor1er2c4ml3txvlj5hpzpghtzqp7gfxy7ax6cx9pn",
"thor1kkcfgh8wwqfhfc2hmwf7nrkdexswa0kyhz724j",
"thor1gs8vvxclqz7rplk9l4jjzs6ztmr9y5zq94xsce",
"thor1hlchjpmteu7u7f5ukfq9glmzg7r4crmsyjn3dg",
"thor10vmz8d0qwvq5hw9susmf7nefka9usazzcvkeaj",
"thor12hd2keknqm4ddq8e8h2wn4rwu4rz6fmag2k47a",
"thor1qp8288u08r2da9sj9pkzv3fkh0ugfutkl9gqdj",
"thor1pmlsgfychtcp7qky9psc6ywlkzp5wwtdea3act",
"thor1c0355gama8pgmdjtnstrshpg2r2lrrg8u4ez2z",
"thor16l2myl9tlsz3w9kaawt2hjajr5verr9lctvpj5",
"thor1nrdz990wj9phcfsc0w5txwstjgm6vp9ydg5kke",
"thor12xm549pv0hdyk72nj9j3e6vs8tpcr489jansar",
"thor1zxz5mueksmdq2esgskh8zey2qqqdv3d95zjcdt",
"thor1jk93saw08vwmqdj684f4w5y6v4dkgdfy2jkrdp",
"thor1kqypcumzc55eyf4exq8lzlc8gw8cyp27rs78fe",
"thor18zykspgra2avmwkstucrx5uyvqd4y9stu0ld74",
"thor1ns5h2ed3rksswnc5359hscjv2dr95sxlemxvlt",
"thor1aw876sn7sdyllrzpnp0vekmcqwxnegl2mvy299",
"thor1zwryqce8f68murrc8428cytf38hlayncr4smjt",
"thor12fw3syyy4ff78llh3fvhrvdy7xnqlegvru7seg",
"thor1kpsyuptekwsf4mmfkmlc4tmpf2td7yfeg5mmh3",
"thor1p9vpn7wn0lnhdq6a3nr4frpstxpn09mn2rns88",
"thor1vdgkmgfud0uy69su9th90hhwlu6gjkx4yxpdrk",
"thor1enssn8jaztv6q8quuqnlpw0cge3fnfhqhphp5p",
"thor1ss9ex4axvwh8e2gzy3jhdx8jsqgs3gzduq747m",
"thor1ejwn32j6ws8femj3fzpzmvtkhlfnjz90rymq7h",
"thor1z2xacl3uzy92mjmm0rh0deglmvkgpxp5vnmdp8",
"thor1j8s7j02756hkd9psg4hkjud79p8a9gl7a20eht",
"thor1ey2753rz6kklxt2tx735k09ultkpfz4pkmuhrv",
"thor15enf2fg7560srddcgd2arhdhxawmgnm3j359n4",
"thor1pwz55q79dwwhx5wsa3tucvlpgwj274l3sr0mvu",
"thor170xscqs5d469chdt83fxatjntc79zucrygsfxj",
"thor1rxqvnj5whqdwc5aq9f8urenjfme638z6ardppc",
"thor1n88l0tjqhltx94q7wmxtqyxsa83lnla8csvxmq",
"thor1kma8d2uxr6edapn48ekhuyzc9t49uws6ryd2cx",
"thor1d8xdly7hkauaz9xpdu0s4h6v6as83pefwgrz69",
"thor1szdphaapkxxf0dah4dj02jyrjkn8rc8plcre0d",
"thor159steswryj4tqt6rnttwy8tghns36hw4s6578e",
"thor1hsga5e2ul8jsy4t6tnxuqakulhmxs32ln7zy2g",
"thor17x539jfnlxm3hjcpd7cn29t3nyn3uv4myf5n5p",
"thor153qnmh7ev7z6xradw49l4m7gx6ge88hdy60rg8",
"thor15qt4h269d78jyxe4jfjpcnzlg4s0wuamhefqru",
"thor1e06hmxf5h43htdn7e3rtsff5xyk84w7e4fyd40",
"thor1k29hf83nx7jgp764zucevuhz3sg75wfc0entkr",
"thor1dum24k9ky5afrp9lprtap5st657gwz6uhuayrq",
"thor1xdwkyz3t7fu7jg5a9nqshe2gueldunj74any0m",
"thor1wkfdzllk8ykzjmd9d7qhdqdknfxsupe9cjxcpn",
"thor1xczxhtnu4vmtmkazny5vkdplfm7gtdfwcerdss",
"thor1ffpnzky5r44unseradnreh9nrcq0l9fp7v9wvx",
"thor15tjtgxq7mz3ljwk0rzw6pvj43tz3xsv9f2wfzp",
"thor12zne4xx0gnhscx2kztct57807mrtfxxsp2rrpl",
"thor17ergq7rnhkph6ev62szywfq7vmzwnm7vtx2pv6"
],
"observed_pub_key": "thorpub1addwnpepq0yxfuwlsam90rla0qxj5l0macx4egr49549m70c3xy9gmfleecwj9k62un",
"finalise_height": 14533959
},
"txs": [
{
"tx": {
"id": "970AFA4D6E15017995B89F08F14EAE581B166A076660CB2C1D8B2D137360CED9",
"chain": "ETH",
"from_address": "0xdc334fb6d445943f80615f4fccabd1c18a355e88",
"to_address": "0x0114eA17e85767940fbe3f5da6384b41BB0E0B82",
"coins": [
{
"asset": "ETH.DAI-0X6B175474E89094C44DA98B954EEDEAC495271D0F",
"amount": "200000000000"
}
],
"gas": [
{
"asset": "ETH.ETH",
"amount": "372433"
}
],
"memo": "=:BTC.BTC:bc1qrrqlp2056yyfpcev45yuak43jetf4gqy4zw72y::thor1a427q3v96psuj4fnughdw8glt5r7j38lj7rkp8:100"
},
"out_hashes": [
"0000000000000000000000000000000000000000000000000000000000000000"
],
"block_height": 14533959,
"signers": [
"thor17e4uzrfqjxfewldwpqszkcelkj9l2c3qkj3k60",
"thor1c2z2lp2fdjjsmmxymlnjuadw7wae7n2jrgwcu9",
"thor1yedw5cuz4l93wyj58wdn7s7z8qppfvusl3c66p",
"thor1jaegcehmsx952h9zak3gvetewtscu5vzzt8hpq",
"thor1ufyppfz6q32yxtymfx23zdr2d6gek0m72zlmm2",
"thor1rc936rg4ffp6jn9ddcmwca75crgarsdnzjuy3a",
"thor1hx3gayvwx0j92nf0ev6c837km4yg4kdk0w6fjl",
"thor1v6664w08cnvy9cfltw4pzzfttvwljgawmaw7yc",
"thor1ee5ec0mnvhqvu84lgtxpgcc4yrdhalylfty246",
"thor1newujzwu4aek3x0u536dx0yt0ayt3ffa7x247x",
"thor144rj67r7tvzuxa39xjrln9mefvhkra290mrp5t",
"thor1rf5dc0gq4vsaza6q66xw5297jdpx2xly7raks5",
"thor1vevr4l9khpqm5zktk4uqaexv07gewhc6elt3l5",
"thor1xd4j3gk9frpxh8r22runntnqy34lwzrdkazldh",
"thor10697ffyya4fddsvpwj6crfwe06pxkxkmdl8kev",
"thor1zn6jxyjgasd5z5j3yvxvyxseprvd4mvss74v27",
"thor1er2c4ml3txvlj5hpzpghtzqp7gfxy7ax6cx9pn",
"thor1kkcfgh8wwqfhfc2hmwf7nrkdexswa0kyhz724j",
"thor1gs8vvxclqz7rplk9l4jjzs6ztmr9y5zq94xsce",
"thor1hlchjpmteu7u7f5ukfq9glmzg7r4crmsyjn3dg",
"thor10vmz8d0qwvq5hw9susmf7nefka9usazzcvkeaj",
"thor12hd2keknqm4ddq8e8h2wn4rwu4rz6fmag2k47a",
"thor1qp8288u08r2da9sj9pkzv3fkh0ugfutkl9gqdj",
"thor1pmlsgfychtcp7qky9psc6ywlkzp5wwtdea3act",
"thor1c0355gama8pgmdjtnstrshpg2r2lrrg8u4ez2z",
"thor16l2myl9tlsz3w9kaawt2hjajr5verr9lctvpj5",
"thor1nrdz990wj9phcfsc0w5txwstjgm6vp9ydg5kke",
"thor12xm549pv0hdyk72nj9j3e6vs8tpcr489jansar",
"thor1zxz5mueksmdq2esgskh8zey2qqqdv3d95zjcdt",
"thor1jk93saw08vwmqdj684f4w5y6v4dkgdfy2jkrdp",
"thor1kqypcumzc55eyf4exq8lzlc8gw8cyp27rs78fe",
"thor18zykspgra2avmwkstucrx5uyvqd4y9stu0ld74",
"thor1ns5h2ed3rksswnc5359hscjv2dr95sxlemxvlt",
"thor1aw876sn7sdyllrzpnp0vekmcqwxnegl2mvy299",
"thor1zwryqce8f68murrc8428cytf38hlayncr4smjt",
"thor12fw3syyy4ff78llh3fvhrvdy7xnqlegvru7seg",
"thor1kpsyuptekwsf4mmfkmlc4tmpf2td7yfeg5mmh3",
"thor1p9vpn7wn0lnhdq6a3nr4frpstxpn09mn2rns88",
"thor1vdgkmgfud0uy69su9th90hhwlu6gjkx4yxpdrk",
"thor1enssn8jaztv6q8quuqnlpw0cge3fnfhqhphp5p",
"thor1ss9ex4axvwh8e2gzy3jhdx8jsqgs3gzduq747m",
"thor1ejwn32j6ws8femj3fzpzmvtkhlfnjz90rymq7h",
"thor1z2xacl3uzy92mjmm0rh0deglmvkgpxp5vnmdp8",
"thor1j8s7j02756hkd9psg4hkjud79p8a9gl7a20eht",
"thor1ey2753rz6kklxt2tx735k09ultkpfz4pkmuhrv",
"thor15enf2fg7560srddcgd2arhdhxawmgnm3j359n4",
"thor1pwz55q79dwwhx5wsa3tucvlpgwj274l3sr0mvu",
"thor170xscqs5d469chdt83fxatjntc79zucrygsfxj",
"thor1rxqvnj5whqdwc5aq9f8urenjfme638z6ardppc",
"thor1n88l0tjqhltx94q7wmxtqyxsa83lnla8csvxmq",
"thor1kma8d2uxr6edapn48ekhuyzc9t49uws6ryd2cx",
"thor1d8xdly7hkauaz9xpdu0s4h6v6as83pefwgrz69",
"thor1szdphaapkxxf0dah4dj02jyrjkn8rc8plcre0d",
"thor159steswryj4tqt6rnttwy8tghns36hw4s6578e",
"thor1hsga5e2ul8jsy4t6tnxuqakulhmxs32ln7zy2g",
"thor17x539jfnlxm3hjcpd7cn29t3nyn3uv4myf5n5p",
"thor153qnmh7ev7z6xradw49l4m7gx6ge88hdy60rg8",
"thor15qt4h269d78jyxe4jfjpcnzlg4s0wuamhefqru",
"thor1e06hmxf5h43htdn7e3rtsff5xyk84w7e4fyd40",
"thor1k29hf83nx7jgp764zucevuhz3sg75wfc0entkr",
"thor1dum24k9ky5afrp9lprtap5st657gwz6uhuayrq",
"thor1xdwkyz3t7fu7jg5a9nqshe2gueldunj74any0m",
"thor1wkfdzllk8ykzjmd9d7qhdqdknfxsupe9cjxcpn",
"thor1xczxhtnu4vmtmkazny5vkdplfm7gtdfwcerdss",
"thor1ffpnzky5r44unseradnreh9nrcq0l9fp7v9wvx",
"thor15tjtgxq7mz3ljwk0rzw6pvj43tz3xsv9f2wfzp",
"thor12zne4xx0gnhscx2kztct57807mrtfxxsp2rrpl",
"thor17ergq7rnhkph6ev62szywfq7vmzwnm7vtx2pv6",
"thor1krcz33mejvc5f6grj2c5w5x3kuj7mnjhgqltj8",
"thor1z3dmy779shx8x9903ldnyqnt3a3g6vjqx68hkt",
"thor1c8sn8s8szz2sl2q3gk4aaxcvpd2s0rap6q6nja",
"thor1rl9jndu3avht2l4zusw7m8km7uarvth5uvyffp",
"thor1jcn6ez7w4cdcs9mxdzm4lu9054z3dukehmm8sk",
"thor10smhtnkaju9ng0cag5p986czp6vmqvnmnl90wh",
"thor1jaeem3rqcnw6vweg8q2m5vyht4cyc3vl5h7g5a",
"thor1khtj7tmmuxm0mm7sjgem9vndzr66xz4h6k9rwp",
"thor19ph9mawujd73gvep82pk09pmmsdl35983mk73x",
"thor1errw9wx5pv8rhevexfxa950jx6tux0qywrlwlp",
"thor1mpsmhp3twv97xcw3j5kmhf2r7u9rrxwe74dszm",
"thor1faa0c6sqryr0am6ls9u8y6zs22ju2y7yw8ju9g",
"thor1pcylx2quurhr44fg35jgvlrypvag70aszgd2t3",
"thor13j8meu00n8u52dxx0ukwq2sc08z5qpcr85n0yl",
"thor1m7tg8nncuyjm0mggmxqgsp8l8lpldrfvytmyce",
"thor1ypjwdplx07vf42qdfkex39dp8zxqnaects270v",
"thor1h0xd53y8yvfsu5t8p6t4ky52h9dzfz3mvsem3z",
"thor13s6nzwa926js8x9jjx3sggyr8xw9500g0fj5dl",
"thor1p4jq8gaq9n24enguz526ahh5tllu336ckgtjw2",
"thor1slnkw42th9tryux32l9r9qg4vx2nnajzvufs5v",
"thor1k6yw0r9lrl7r7ne0dszfrw4aeukzyc9z4theqk",
"thor1scwn3nw4lc9s5nzp39evym6mxn8705gh4hcjvp",
"thor1a39p456jcetqhy8dls4pnt8l7aqu5f8hx9r65t",
"thor1t63ew3e87m3sa0aeqygje92ag62ydlezw3cqth",
"thor1w0qlnmtk2u9sxntts8zu0hmx4mtj9y42jmqrhl",
"thor17w5hd43edzmhgznl3vumahawzm5vhhj3x6x3jv",
"thor1wzgs276v929el645huxf9uql797j66lvjexj02",
"thor1sw6nfu59rvsfzmzxdgka0puulvy6r89hk9zj8m",
"thor1gxkrwu2khhg2dr26w22tdeqvfyr3rs9vg490np",
"thor12vq8zqu84s2mxkgf52fwmrc4fv9a62mvfx2gzm",
"thor1jnf3dwfjanvtdd2g0dhkgzy0yfs6v7vzelzdxy",
"thor1zqr623vc4xrfnc7h6wxe2xsg7knvx0h236fnqy",
"thor1el68x6hagewxamzefha28v3du0fn32m48f4qhf"
],
"observed_pub_key": "thorpub1addwnpepq0yxfuwlsam90rla0qxj5l0macx4egr49549m70c3xy9gmfleecwj9k62un",
"finalise_height": 14533959
}
],
"actions": [
{
"chain": "BTC",
"to_address": "bc1qrrqlp2056yyfpcev45yuak43jetf4gqy4zw72y",
"vault_pub_key": "thorpub1addwnpepqwakhxwqzgwd97df39wjvsjylnt2hgmjmdtcz83tq6mlnrlf06mycdzk7ch",
"coin": {
"asset": "BTC.BTC",
"amount": "4440296"
},
"memo": "OUT:970AFA4D6E15017995B89F08F14EAE581B166A076660CB2C1D8B2D137360CED9",
"max_gas": [
{
"asset": "BTC.BTC",
"amount": "10500",
"decimals": 8
}
],
"gas_rate": 10,
"in_hash": "970AFA4D6E15017995B89F08F14EAE581B166A076660CB2C1D8B2D137360CED9"
},
{
"chain": "THOR",
"to_address": "thor1a427q3v96psuj4fnughdw8glt5r7j38lj7rkp8",
"coin": {
"asset": "THOR.RUNE",
"amount": "207543687"
},
"memo": "OUT:970AFA4D6E15017995B89F08F14EAE581B166A076660CB2C1D8B2D137360CED9",
"max_gas": [
{
"asset": "THOR.RUNE",
"amount": "0",
"decimals": 8
}
],
"gas_rate": 2000000,
"in_hash": "970AFA4D6E15017995B89F08F14EAE581B166A076660CB2C1D8B2D137360CED9"
}
],
"out_txs": [
{
"id": "0000000000000000000000000000000000000000000000000000000000000000",
"chain": "THOR",
"from_address": "thor1g98cy3n9mmjrpn0sxmn63lztelera37n8n67c0",
"to_address": "thor1a427q3v96psuj4fnughdw8glt5r7j38lj7rkp8",
"coins": [
{
"asset": "THOR.RUNE",
"amount": "207543687"
}
],
"gas": [
{
"asset": "THOR.RUNE",
"amount": "2000000"
}
],
"memo": "OUT:970AFA4D6E15017995B89F08F14EAE581B166A076660CB2C1D8B2D137360CED9"
}
],
"finalised_height": 5015181,
"updated_vault": true,
"outbound_height": 5015190
}These are the information store against the inbound transaction , there is structure in thornode , we call it ObservedTxVoter
When the inbound transaction get processed , a few things will happen.
- If the inbound transaction triggers outbound , then the outbound transactions will be added into
actionsfield - Once the outbound transaction get processed by bifrost , and observed back by the network , the outbound transaction will be added to
out_txsfield , and also outbound hash will be added toout_hashesfield - If an outbound need to be delayed , then
outbound_heightis the field to determinate on which block height the outbound will be issued by the network
The Problem
In this case reported by user , the outbound transaction has been processed , and sent out https://blockstream.info/tx/6651a8c09e4dc0c0bc32ee477cf583c214efc7926bd0a306e229f4f94b6e9eca , however the outbound hash didn't get added to out_hashes field, which confuse midgard to think the swap is not complete
The root cause
A while ago , in order to harden the network , a feature call scheduled outbound has been introduced , which means , if the outbound exceed some value, then the outbound will be delayed , and scheduled to be sent out many blocks later. However as we know that the gas price for external network changes often.
- At the time when inbound transaction get processed , the network construct a outbound tx , and then added it to the
actionsfield ofObservedTxVoter, max gas field will set based on the gas the network knows at the time. - the same outbound tx will be cloned and pushed into an outbound queue , and to be processed many blocks later.
- a few blocks later, the gas price changed
- When it it is time for the outbound transaction to be sent , the network issue the outbound tx for bifrost , and also update the
max_gasfield of the outbound tx in the queue, HOWEVER ,it forget to update the tx out in theactionsfield in the relevantObservedTxVoter
https://gitlab.com/thorchain/thornode/-/blob/develop/x/thorchain/manager_txout_current.go#L62
- When the outbound tx get sent out by bifrost, and observed by the network , the outbound is using different gas as the tx out in
actionsfield , thus it doesn't match, so it didn't add the outbound hash toout_hashesfield
The solution
When the network update gas rate of a scheduled outbound tx, make sure it update the tx out in actions field of ObservedTxVoter
Edited by Heimdall