Confirm the data is downloadable

[jennijuju]

blocked by #3

Once getting where the data is stored, run fetch on a couple of them to confirm the data is downloadable.

[jennijuju]

🟡 given missing DRI

[elizabeth-griffiths]

@jennijuju have we identified dri for this yet?

[momack2]

@dchoi can maybe do this? The aim is just "does the generated URL allow anyone to download the data" - which AFAIK anyone able to generate the URL should be able to test, right?

[jennijuju]

@Reiers @TippyFlitsUK - please randomly check the data with the updated url and confirm here whether the data is now downloadable.
(I checked two and they works - but wanna get more confidence)

[TippyFlitsUK]

Hey folks 👋
I am able to download the files by browsing to the link or using aria2c but I am running into issues using wget and curl.

aria2c -x 10 --file-allocation=none --continue -d  ~/ "https://fffa4b4363a7e5250af8357087263b3a.r2.cloudflarestorage.com/dagcargo/29b47f043d0fdca080fcc009dbdf859b_baga6ea4seaqa5fntokponqfxpzlcey44zhuuhwievxjr56wy5o2s3xl7pwfd4ky.car?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=04f131617341e6724ff861083ed5256b%2F20230307%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20230307T161630Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=0ecba06a0c72c0ed3c442986279905b84c17920bac91349e3555badb07f3823e"

03/08 14:21:09 [NOTICE] Downloading 1 item(s)
[#a50b76 5.6GiB/29GiB(18%) CN:5 DL:109MiB ETA:3m45s] 

wget https://fffa4b4363a7e5250af8357087263b3a.r2.cloudflarestorage.com/dagcargo/29b47f043d0fdca080fcc009dbdf859b_baga6ea4seaqa5fntokponqfxpzlcey44zhuuhwievxjr56wy5o2s3xl7pwfd4ky.car?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=04f131617341e6724ff861083ed5256b%2F20230307%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20230307T161630Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=0ecba06a0c72c0ed3c442986279905b84c17920bac91349e3555badb07f3823e
[1] 2634630
[2] 2634631
[3] 2634632
[4] 2634633
[5] 2634634

Redirecting output to ‘wget-log.2’.
X-Amz-Expires=604800: command not found
X-Amz-SignedHeaders=host: command not found
X-Amz-Date=20230307T161630Z: command not found
X-Amz-Signature=0ecba06a0c72c0ed3c442986279905b84c17920bac91349e3555badb07f3823e: command not found
X-Amz-Credential=04f131617341e6724ff861083ed5256b%2F20230307%2Fauto%2Fs3%2Faws4_request: command not found
[1]   Exit 8                  wget https://fffa4b4363a7e5250af8357087263b3a.r2.cloudflarestorage.com/dagcargo/29b47f043d0fdca080fcc009dbdf859b_baga6ea4seaqa5fntokponqfxpzlcey44zhuuhwievxjr56wy5o2s3xl7pwfd4ky.car?X-Amz-Algorithm=AWS4-HMAC-SHA256
[3]   Exit 127                X-Amz-Date=20230307T161630Z
[4]-  Exit 127                X-Amz-Expires=604800
[5]+  Exit 127                X-Amz-SignedHeaders=host

[dchoi27]

Think it's just a quotes thing - I just tested with quotes around the URL and it works fine
https://stackoverflow.com/questions/34101512/amazon-aws-s3-signed-url-via-wget
https://superuser.com/questions/1311516/curl-can-not-download-file-but-browser-can

[jennijuju]

We tested a .storage deal on hyperspace and the links are working