AWS S3 API doesn't work with FIFO

// From: Thomas

I've tried to configure the FIFO version 0.9.1 with AWS S3 to use it for images and backups. But it doesn't work and return an error in the {{sniffle}} log:

2017-05-04 17:42:25 =ERROR REPORT====
** State machine <0.25495.0> terminating 
** Last event in was timeout
** When State == get_manifest
**      Data  == {state,undefined,<<"https://imgapi.smartos.skylime.net/images/156cac3d-2d48-489e-b12c-5ff493bffe2d">>,undefined,undefined,undefined,undefined,undefined,undefined,<0.3029.0>,#Ref<0.0.1.42207>,<<>>,0,[],undefined,false}
** Reason for termination = 
** {{badmatch,{error,{error,{http_error,400,"Bad Request",<<"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>80BEDA13F16391B1</RequestId><HostId>INMtLKaD1O0DbIcCnWUR40vMCjKidFhQM6n2AmFqlE0c+G9YEpMYlApkOJ7x9iTI/IkWelm4bz0=</HostId></Error>">>}}}},[{sniffle_dataset_download_fsm,get_manifest,2,[{file,"/root/workspace/Fifo_15.4.1_Rel/Sniffle-15.4.1-Rel/_build/prod/lib/sniffle_dataset_download/src/sniffle_dataset_download_fsm.erl"},{line,128}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,518}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}
2017-05-04 17:42:25 =CRASH REPORT====
  crasher:
    initial call: sniffle_dataset_download_fsm:init/1
    pid: <0.25495.0>
    registered_name: []
    exception exit: {{{badmatch,{error,{error,{http_error,400,"Bad Request",<<"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>80BEDA13F16391B1</RequestId><HostId>INMtLKaD1O0DbIcCnWUR40vMCjKidFhQM6n2AmFqlE0c+G9YEpMYlApkOJ7x9iTI/IkWelm4bz0=</HostId></Error>">>}}}},[{sniffle_dataset_download_fsm,get_manifest,2,[{file,"/root/workspace/Fifo_15.4.1_Rel/Sniffle-15.4.1-Rel/_build/prod/lib/sniffle_dataset_download/src/sniffle_dataset_download_fsm.erl"},{line,128}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,518}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gen_fsm,terminate,7,[{file,"gen_fsm.erl"},{line,626}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}
    ancestors: [sniffle_dataset_download_sup,<0.247.0>]
    messages: []
    links: [<0.248.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 1598
    stack_size: 27
    reductions: 14185
  neighbours:
2017-05-04 17:42:25 =SUPERVISOR REPORT====
     Supervisor: {local,sniffle_dataset_download_sup}
     Context:    child_terminated
     Reason:     {{badmatch,{error,{error,{http_error,400,"Bad Request",<<"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>80BEDA13F16391B1</RequestId><HostId>INMtLKaD1O0DbIcCnWUR40vMCjKidFhQM6n2AmFqlE0c+G9YEpMYlApkOJ7x9iTI/IkWelm4bz0=</HostId></Error>">>}}}},[{sniffle_dataset_download_fsm,get_manifest,2,[{file,"/root/workspace/Fifo_15.4.1_Rel/Sniffle-15.4.1-Rel/_build/prod/lib/sniffle_dataset_download/src/sniffle_dataset_download_fsm.erl"},{line,128}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,518}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}
     Offender:   [{pid,<0.25495.0>},{id,sniffle_dataset_download_fsm},{mfargs,{sniffle_dataset_download_fsm,start_link,undefined}},{restart_type,temporary},{shutdown,5000},{child_type,worker}]

The following configuration is used:

               storage.s3.host                      s3-eu-central-1.amazonaws.com 
               storage.s3.port                                                443 
         storage.s3.access_key                               **** 
         storage.s3.secret_key           **** 
     storage.s3.general_bucket                                               tm-fifo 
       storage.s3.image_bucket                                     tm-fifo-images 
    storage.s3.snapshot_bucket                                       tm-fifo-backups 

---

LeoFS is the suppored s3 storage used by fifo, while AWS is nice-to-have it's not part of the core functionality. Will investigate when I got some free time and nerves to mess with AWS

Maybe the current API is already supported with the newest https://github.com/erlcloud/erlcloud version? 

Yea I'm trying that right now, or at least building a dev chunter wit it for you to try 

If you want to see if it works with dev? If it does (and also works with Leo) it'd be an easy fox 

It would be nice if you could provide me with the correct URL I should use to receive the dev packages. I'm not sure if http://release.project-fifo.net/pkg/15.4.1/dev is correct.

https://docs.project-fifo.net/docs/chunter#section-installation details how to install a dev version.	

Ohh thanks, I thought it requires a new sniffle version not a chunter version 

Could be, but best to try changing as little things at once as possible 

I've updated to the dev version but I sill receive the error, so I assume I'm sorry a version bump of erlcloud doesn't fix it 

2017-05-07 12:00:56.336 [error] <0.2275.0> CRASH REPORT Process <0.2275.0> with 0 neighbours exited with reason: {error,{http_error,400,"Bad Request",<<"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>07F352323EFEA4A0</RequestId><HostId>0OLx1UF8XxkdBGgYUtVVcL+BzGuwECWxOa8EZEHKqoQUBgMu9yHIQZDpxjuEpqUi/bm37uhTvJA=</HostId></Error>">>}} in gen_server:init_it/6 line 344
2017-05-07 12:00:56.337 [error] <0.2221.0> gen_fsm <0.2221.0> in state get_manifest terminated with reason: no match of right hand value {error,{error,{http_error,400,"Bad Request",<<"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>07F352323EFEA4A0</RequestId><HostId>0OLx1UF8XxkdBGgYUtVVcL+BzGuwECWxOa8EZEHKqoQUBgMu9yHIQZDpxjuEpqUi/bm37uhTvJA=</HostId></Error>">>}}} in sniffle_dataset_download_fsm:get_manifest/2 line 128
2017-05-07 12:00:56.341 [error] <0.2221.0> CRASH REPORT Process <0.2221.0> with 0 neighbours exited with reason: no match of right hand value {error,{error,{http_error,400,"Bad Request",<<"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>07F352323EFEA4A0</RequestId><HostId>0OLx1UF8XxkdBGgYUtVVcL+BzGuwECWxOa8EZEHKqoQUBgMu9yHIQZDpxjuEpqUi/bm37uhTvJA=</HostId></Error>">>}}} in sniffle_dataset_download_fsm:get_manifest/2 line 128 in gen_fsm:terminate/7 line 626
2017-05-07 12:00:56.341 [error] <0.248.0> Supervisor sniffle_dataset_download_sup had child sniffle_dataset_download_fsm started with {sniffle_dataset_download_fsm,start_link,undefined} at <0.2221.0> exit with reason no match of right hand value {error,{error,{http_error,400,"Bad Request",<<"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>07F352323EFEA4A0</RequestId><HostId>0OLx1UF8XxkdBGgYUtVVcL+BzGuwECWxOa8EZEHKqoQUBgMu9yHIQZDpxjuEpqUi/bm37uhTvJA=</HostId></Error>">>}}} in sniffle_dataset_download_fsm:get_manifest/2 line 128 in context child_terminated

Is it somehow easy possible to add some debugging option to see the exact request?