Skip to content

Commit

Permalink
Fix openssl_static_binaries download (#295)
Browse files Browse the repository at this point in the history
# Motivation

In snsdemo we download binaries with OpenSSL statically linked because
they depend less on what is installed in the environment where they are
run. These binaries used to be in a special download directory, but now
all binaries are statically linked and the separate directory no longer
exists. This means that our downloads were failing and we were not
updating our IC COMMIT. It's currently 2 weeks old at
`711369db94d9bd81d1f9127c54e644ab97a72fa3`.

# Changes

1. In `bin/dfx-software-ic-install-executable` download from the
`binaries` directory instead of the `openssl-static-binaries` directory.
2. In `bin/dfx-software-ic-latest` change the files that we try to
download:
    a. Check `binaries/` instead of `openssl-static-binaries/`
b. Remove disk images. They also no longer exist at the location we try
to download them from but also snsdemo doesn't use disk images so there
is no point in checking if they exist.
c. Add some canisters that snsdemo requires just to make sure they exist
at the used commit. This didn't seem to be necessary before but I'm
adding them in case the presence of the disk images made it more likely
for other things to also exist at the same commit.

# Tested

1. Ran `bin/dfx-software-ic-latest -x ../../ic` and got
`f45daff4e63ad2d02caa90d35a9144ba5ddddb25`
2. Installed the `ic-admin` from the `binaries` directory and checked
that it worked on my Mac. Previously only the `openssl-static-binaries`
version worked on my Mac.
  • Loading branch information
dskloetd authored Dec 11, 2023
1 parent 49eef81 commit 585c588
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 15 deletions.
4 changes: 2 additions & 2 deletions bin/dfx-software-ic-install-executable
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ clap.define short=b long=bin desc="Diectory in which to install the executeble"
source "$(clap.build)"

install_linux() {
URL="https://download.dfinity.systems/ic/${DFX_IC_COMMIT}/openssl-static-binaries/x86_64-linux/${EXEC_NAME}.gz"
URL="https://download.dfinity.systems/ic/${DFX_IC_COMMIT}/binaries/x86_64-linux/${EXEC_NAME}.gz"
echo "Downloading $URL"
curl -fL "$URL" | gunzip | install -m 755 /dev/stdin "$USER_BIN/$EXEC_NAME"
}
install_darwin() {
URL="https://download.dfinity.systems/ic/${DFX_IC_COMMIT}/openssl-static-binaries/x86_64-darwin/${EXEC_NAME}.gz"
URL="https://download.dfinity.systems/ic/${DFX_IC_COMMIT}/binaries/x86_64-darwin/${EXEC_NAME}.gz"
echo "Downloading $URL"
curl -fL "$URL" | gunzip >"$USER_BIN/$EXEC_NAME"
chmod +x "$USER_BIN/$EXEC_NAME"
Expand Down
31 changes: 18 additions & 13 deletions bin/dfx-software-ic-latest
Original file line number Diff line number Diff line change
Expand Up @@ -11,23 +11,28 @@ clap.define short=a long=after desc="First published commit after the given one"
# Source the output file ----------------------------------------------------------
source "$(clap.build)"

function disk_image_exists() {
function downloads_exist() {
(
set -euo pipefail

curl -fL --output /dev/null --silent --head --fail \
"https://download.dfinity.systems/ic/$GIT_REVISION/guest-os/disk-img/disk-img.tar.gz" ||
curl --output /dev/null --silent --head --fail \
"https://download.dfinity.systems/ic/$GIT_REVISION/guest-os/disk-img.tar.gz" || exit 1
for file in \
"binaries/x86_64-darwin/ic-admin.gz" \
"binaries/x86_64-darwin/sns.gz" \
"binaries/x86_64-linux/ic-admin.gz" \
"binaries/x86_64-linux/sns.gz" \
"canisters/bitcoin-mock-canister.wasm.gz" \
"canisters/ic-ckbtc-kyt.wasm.gz" \
"canisters/ic-ckbtc-minter.wasm.gz" \
"canisters/ic-icrc1-index-ng-u256.wasm.gz" \
"canisters/ic-icrc1-index-ng.wasm.gz" \
"canisters/ic-icrc1-ledger-u256.wasm.gz" \
"canisters/ic-icrc1-ledger.wasm.gz" \
"canisters/sns-wasm-canister.wasm.gz"; do

curl -fL --output /dev/null --silent --head --fail \
"https://download.dfinity.systems/ic/$GIT_REVISION/guest-os/update-img/SHA256SUMS" || exit 1
curl --location --output /dev/null --silent --head --fail \
"https://download.dfinity.systems/ic/${GIT_REVISION}/$file" || exit 1
done

# Mac buids can lag or be missing
curl -fL --output /dev/null --silent --head --fail \
"https://download.dfinity.systems/ic/${GIT_REVISION}/openssl-static-binaries/x86_64-darwin/ic-admin.gz" || exit 1
curl -fL --output /dev/null --silent --head --fail \
"https://download.dfinity.systems/ic/${GIT_REVISION}/openssl-static-binaries/x86_64-linux/ic-admin.gz" || exit 1
# Syncing to the public repo may be slow
curl -fL --output /dev/null --silent --head --fail \
"https://raw.githubusercontent.com/dfinity/ic/${GIT_REVISION}/rs/nns/dfx.json" || exit 1
Expand All @@ -49,4 +54,4 @@ function disk_image_exists() {
else
git log --format=format:%H "$IC_COMMIT"
fi
) | while read -r GIT_REVISION; do disk_image_exists && echo "$GIT_REVISION" && break; done
) | while read -r GIT_REVISION; do downloads_exist && echo "$GIT_REVISION" && break; done

0 comments on commit 585c588

Please sign in to comment.