Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H-3618: Add GitHub action step to authenticate with Google Cloud, for chonky tests #5687

Merged
merged 8 commits into from
Dec 4, 2024

Conversation

CiaranMn
Copy link
Member

@CiaranMn CiaranMn commented Nov 21, 2024

🌟 What is the purpose of this PR?

Adds a step prior to chonky tests which authenticates to Google Cloud Platform.

See GitHub action and GCP docs.

The service account being impersonated is limited to Vertex AI access.

Also passes the HUGGING_FACE_TOKEN to tests.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

❓ How to test this?

  1. See if the chonky tests run successfully.

@github-actions github-actions bot added area/libs Relates to first-party libraries/crates/packages (area) area/libs > chonky Affects the `chonky` crate (library) labels Dec 4, 2024
@@ -178,6 +192,10 @@ jobs:
continue-on-error: ${{ steps.tests.outputs.allow-failure == 'true' }}
env:
TEST_COVERAGE: ${{ github.event_name != 'merge_group' }}
# Variables needed for chonky tests
GOOGLE_PROJECT_ID: ${{ secrets.GOOGLE_CLOUD_HASH_PROJECT_ID }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line seems to be problematic when moving towards running the Vertex API via Hash's Google cloud account, since each project_id must be linked to the name of the project that is created by the user. This project_id is used in the URL identification for billing purposes and since it most likely does not exist on the Hash google cloud account it is failing with a 403 error. For my case I named the project hash-embed, but this should be changed when running the github testing workflow.

@@ -293,6 +316,15 @@ jobs:
rm -rf $temp_dir
echo "PDFIUM_DYNAMIC_LIB_PATH=$(pwd)/${{ matrix.directory }}/libs/" >> $GITHUB_ENV

# Sets GOOGLE_APPLICATION_CREDENTIALS in the environment, to be consumed by gcloud or client libraries
- name: Generate Google Cloud credential configuration
if: matrix.package == '@rust/chonky'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the documentation for 403 error codes, there seems to be priority in checking glcoud authentication before checking the valid URL, which implies that for the most part the rest of the workflow is functioning as intended. (link to documentation here)

Copy link
Contributor

github-actions bot commented Dec 4, 2024

Benchmark results

@rust/hash-graph-benches – Integrations

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$17.6 \mathrm{ms} \pm 225 \mathrm{μs}\left({\color{gray}-3.989 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$17.3 \mathrm{ms} \pm 233 \mathrm{μs}\left({\color{gray}-4.416 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$17.8 \mathrm{ms} \pm 238 \mathrm{μs}\left({\color{gray}-4.420 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$17.8 \mathrm{ms} \pm 224 \mathrm{μs}\left({\color{gray}3.33 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$17.1 \mathrm{ms} \pm 220 \mathrm{μs}\left({\color{gray}-2.031 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$18.9 \mathrm{ms} \pm 170 \mathrm{μs}\left({\color{gray}2.84 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$18.0 \mathrm{ms} \pm 232 \mathrm{μs}\left({\color{gray}-0.233 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$18.6 \mathrm{ms} \pm 233 \mathrm{μs}\left({\color{gray}3.84 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$16.9 \mathrm{ms} \pm 179 \mathrm{μs}\left({\color{gray}-3.076 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property depths: DT=255, PT=255, ET=255, E=255 $$69.9 \mathrm{ms} \pm 371 \mathrm{μs}\left({\color{gray}-2.007 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=0 $$41.7 \mathrm{ms} \pm 214 \mathrm{μs}\left({\color{gray}-1.719 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=2, PT=2, ET=2, E=2 $$59.7 \mathrm{ms} \pm 124 \mathrm{μs}\left({\color{gray}-2.618 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=2 $$46.1 \mathrm{ms} \pm 305 \mathrm{μs}\left({\color{gray}-0.618 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=2, E=2 $$51.3 \mathrm{ms} \pm 274 \mathrm{μs}\left({\color{gray}-1.757 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=2, ET=2, E=2 $$55.5 \mathrm{ms} \pm 211 \mathrm{μs}\left({\color{gray}-2.240 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=255, PT=255, ET=255, E=255 $$109 \mathrm{ms} \pm 337 \mathrm{μs}\left({\color{gray}-2.599 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=0 $$44.2 \mathrm{ms} \pm 217 \mathrm{μs}\left({\color{gray}-2.460 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=2, PT=2, ET=2, E=2 $$99.3 \mathrm{ms} \pm 474 \mathrm{μs}\left({\color{gray}-2.805 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=2 $$83.1 \mathrm{ms} \pm 349 \mathrm{μs}\left({\color{gray}-1.688 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=2, E=2 $$90.8 \mathrm{ms} \pm 438 \mathrm{μs}\left({\color{gray}-2.270 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=2, ET=2, E=2 $$95.3 \mathrm{ms} \pm 436 \mathrm{μs}\left({\color{gray}-2.314 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579 $$1.40 \mathrm{ms} \pm 7.93 \mathrm{μs}\left({\color{gray}0.862 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_one_depth

Function Value Mean Flame graphs
entity_by_id 50 entities $$266 \mathrm{ms} \pm 817 \mathrm{μs}\left({\color{gray}1.87 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$25.6 \mathrm{ms} \pm 159 \mathrm{μs}\left({\color{gray}0.618 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$20.4 \mathrm{ms} \pm 92.4 \mathrm{μs}\left({\color{gray}-0.984 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$52.2 \mathrm{ms} \pm 300 \mathrm{μs}\left({\color{gray}1.19 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$87.9 \mathrm{ms} \pm 334 \mathrm{μs}\left({\color{red}21.4 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$1.88 \mathrm{ms} \pm 8.37 \mathrm{μs}\left({\color{gray}-1.253 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$1.99 \mathrm{ms} \pm 9.92 \mathrm{μs}\left({\color{gray}-3.787 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$1.93 \mathrm{ms} \pm 6.95 \mathrm{μs}\left({\color{gray}-0.038 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$3.10 \mathrm{ms} \pm 14.0 \mathrm{μs}\left({\color{red}7.38 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$14.2 \mathrm{ms} \pm 92.0 \mathrm{μs}\left({\color{gray}-3.078 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_zero_depth

Function Value Mean Flame graphs
entity_by_id 50 entities $$5.43 \mathrm{ms} \pm 34.4 \mathrm{μs}\left({\color{gray}-2.492 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$1.94 \mathrm{ms} \pm 17.5 \mathrm{μs}\left({\color{gray}-1.138 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.90 \mathrm{ms} \pm 6.93 \mathrm{μs}\left({\color{gray}0.112 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$2.14 \mathrm{ms} \pm 6.81 \mathrm{μs}\left({\color{gray}0.337 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$3.33 \mathrm{ms} \pm 12.6 \mathrm{μs}\left({\color{red}18.5 \mathrm{\%}}\right) $$ Flame Graph

@JesusFileto JesusFileto merged commit c999477 into Chonky Dec 4, 2024
159 checks passed
@JesusFileto JesusFileto deleted the cm/add-google-cloud-auth branch December 4, 2024 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/infra Relates to version control, CI, CD or IaC (area) area/libs > chonky Affects the `chonky` crate (library) area/libs Relates to first-party libraries/crates/packages (area)
Development

Successfully merging this pull request may close these issues.

2 participants