Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H-3552: graph: Integrate harpc into the graph #5577

Merged
merged 28 commits into from
Nov 6, 2024

Conversation

indietyp
Copy link
Member

@indietyp indietyp commented Nov 4, 2024

🌟 What is the purpose of this PR?

This integrates harpc into the graph, the follow up will connect the implementation to the binary under a feature flag.

This ended up being a bit bigger of a PR, primarily to prepare for code generation.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • affected the execution graph, and the turbo.json's have been updated to reflect this

@github-actions github-actions bot added area/deps Relates to third-party dependencies (area) area/apps > hash* Affects HASH (a `hash-*` app) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/apps area/apps > hash-graph labels Nov 4, 2024
@indietyp indietyp requested a review from TimDiekmann November 4, 2024 11:47
@vilkinsons vilkinsons changed the title graph: integrate harpc into the graph H-3552: graph: Integrate harpc into the graph Nov 4, 2024
Copy link
Member

@TimDiekmann TimDiekmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing 🎉 Thanks a lot!
As we'll synchronize offline now I'll post my partial review so we can look at the same comments.

apps/hash-graph/libs/api/src/rpc/ping.rs Outdated Show resolved Hide resolved
apps/hash-graph/libs/api/src/rpc/ping.rs Outdated Show resolved Hide resolved
apps/hash-graph/libs/api/src/rpc/session.rs Outdated Show resolved Hide resolved
apps/hash-graph/libs/api/src/rpc/ping.rs Outdated Show resolved Hide resolved
apps/hash-graph/libs/api/src/rpc/ping.rs Outdated Show resolved Hide resolved
apps/hash-graph/libs/api/src/rpc/mod.rs Outdated Show resolved Hide resolved
apps/hash-graph/libs/api/src/rpc/account.rs Outdated Show resolved Hide resolved
apps/hash-graph/libs/api/src/rpc/account.rs Show resolved Hide resolved
apps/hash-graph/libs/api/src/rpc/account.rs Outdated Show resolved Hide resolved
Copy link
Member

@TimDiekmann TimDiekmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So exciting to see this happening!

@indietyp indietyp enabled auto-merge November 6, 2024 18:10
Copy link

codecov bot commented Nov 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 20.09%. Comparing base (f67bc88) to head (d158a1b).
Report is 20 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5577      +/-   ##
==========================================
- Coverage   20.15%   20.09%   -0.06%     
==========================================
  Files         509      514       +5     
  Lines       17319    17368      +49     
  Branches     2538     2545       +7     
==========================================
  Hits         3490     3490              
- Misses      13791    13840      +49     
  Partials       38       38              
Flag Coverage Δ
apps.hash-ai-worker-ts 1.38% <ø> (ø)
apps.hash-api 1.17% <ø> (-0.02%) ⬇️
local.hash-backend-utils 8.80% <ø> (ø)
local.hash-isomorphic-utils 1.05% <ø> (-0.01%) ⬇️
local.hash-subgraph 24.54% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

github-actions bot commented Nov 6, 2024

Benchmark results

@rust/graph-benches – Integrations

scaling_read_entity_complete_one_depth

Function Value Mean Flame graphs
entity_by_id 25 entities $$73.9 \mathrm{ms} \pm 500 \mathrm{μs}\left({\color{gray}-1.720 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$26.2 \mathrm{ms} \pm 195 \mathrm{μs}\left({\color{red}5.04 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$20.3 \mathrm{ms} \pm 70.0 \mathrm{μs}\left({\color{gray}2.50 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$47.8 \mathrm{ms} \pm 2.52 \mathrm{ms}\left({\color{red}52.7 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 50 entities $$565 \mathrm{ms} \pm 5.09 \mathrm{ms}\left({\color{red}108 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$16.6 \mathrm{ms} \pm 239 \mathrm{μs}\left({\color{gray}4.67 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$16.8 \mathrm{ms} \pm 217 \mathrm{μs}\left({\color{gray}2.45 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$17.4 \mathrm{ms} \pm 214 \mathrm{μs}\left({\color{gray}4.70 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$16.3 \mathrm{ms} \pm 202 \mathrm{μs}\left({\color{gray}0.578 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$16.0 \mathrm{ms} \pm 168 \mathrm{μs}\left({\color{gray}-3.142 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$16.5 \mathrm{ms} \pm 175 \mathrm{μs}\left({\color{gray}-4.825 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$17.1 \mathrm{ms} \pm 199 \mathrm{μs}\left({\color{red}8.85 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$15.8 \mathrm{ms} \pm 193 \mathrm{μs}\left({\color{lightgreen}-6.329 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$17.0 \mathrm{ms} \pm 200 \mathrm{μs}\left({\color{gray}1.57 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_zero_depth

Function Value Mean Flame graphs
entity_by_id 25 entities $$3.29 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}0.285 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$1.92 \mathrm{ms} \pm 10.5 \mathrm{μs}\left({\color{gray}0.256 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.86 \mathrm{ms} \pm 11.0 \mathrm{μs}\left({\color{gray}-0.099 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$2.12 \mathrm{ms} \pm 10.6 \mathrm{μs}\left({\color{gray}1.56 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 50 entities $$5.33 \mathrm{ms} \pm 28.2 \mathrm{μs}\left({\color{red}24.4 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$1.87 \mathrm{ms} \pm 6.20 \mathrm{μs}\left({\color{gray}-0.015 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$13.5 \mathrm{ms} \pm 90.2 \mathrm{μs}\left({\color{gray}0.065 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$1.97 \mathrm{ms} \pm 10.9 \mathrm{μs}\left({\color{gray}-4.253 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$2.78 \mathrm{ms} \pm 13.0 \mathrm{μs}\left({\color{gray}-4.922 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$1.88 \mathrm{ms} \pm 6.05 \mathrm{μs}\left({\color{gray}-0.267 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
link_by_source_by_property depths: DT=255, PT=255, ET=255, E=255 $$107 \mathrm{ms} \pm 781 \mathrm{μs}\left({\color{gray}0.532 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=2, E=2 $$88.0 \mathrm{ms} \pm 439 \mathrm{μs}\left({\color{gray}-0.789 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=2, PT=2, ET=2, E=2 $$97.3 \mathrm{ms} \pm 483 \mathrm{μs}\left({\color{gray}0.618 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=0 $$42.5 \mathrm{ms} \pm 216 \mathrm{μs}\left({\color{gray}-0.434 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=2, ET=2, E=2 $$92.7 \mathrm{ms} \pm 543 \mathrm{μs}\left({\color{gray}0.227 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=2 $$80.1 \mathrm{ms} \pm 396 \mathrm{μs}\left({\color{gray}0.713 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=255, PT=255, ET=255, E=255 $$68.3 \mathrm{ms} \pm 426 \mathrm{μs}\left({\color{gray}0.469 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=2, E=2 $$48.9 \mathrm{ms} \pm 266 \mathrm{μs}\left({\color{gray}-0.554 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=2, PT=2, ET=2, E=2 $$58.0 \mathrm{ms} \pm 454 \mathrm{μs}\left({\color{gray}0.690 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=0 $$39.7 \mathrm{ms} \pm 287 \mathrm{μs}\left({\color{gray}-1.797 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=2, ET=2, E=2 $$53.5 \mathrm{ms} \pm 261 \mathrm{μs}\left({\color{gray}-0.217 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=2 $$43.4 \mathrm{ms} \pm 196 \mathrm{μs}\left({\color{gray}-1.112 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579 $$1.38 \mathrm{ms} \pm 5.15 \mathrm{μs}\left({\color{gray}-0.476 \mathrm{\%}}\right) $$ Flame Graph

Merged via the queue into main with commit 10447a2 Nov 6, 2024
164 checks passed
@indietyp indietyp deleted the bm/harpc/graph-integration branch November 6, 2024 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-graph area/apps area/deps Relates to third-party dependencies (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team
Development

Successfully merging this pull request may close these issues.

2 participants