Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H-3778: Resolve DNS entries on connection re-use #5859

Conversation

indietyp
Copy link
Member

🌟 What is the purpose of this PR?

When connecting via libp2p over DNS the addresses that are saved on the peer are the ipv4 ones, not the dns entry, this lead to harpc forcing and maintaining a new connection.

This is (of course) not ideal, as DNS isn't "built-in" (it simply exploits the fact that node:net can resolve dns entries) we need to resolve the DNS entries ourselves and match them with the ones libp2p uses.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

@github-actions github-actions bot added area/deps Relates to third-party dependencies (area) area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-api Affects the HASH API (app) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/apps labels Dec 11, 2024
Copy link

codecov bot commented Dec 11, 2024

Codecov Report

Attention: Patch coverage is 14.41441% with 95 lines in your changes missing coverage. Please review.

Project coverage is 22.90%. Comparing base (cb79928) to head (dce8277).
Report is 11 commits behind head on main.

Files with missing lines Patch % Lines
...pc/client/typescript/src/net/internal/transport.ts 13.09% 73 Missing ⚠️
...pc/client/typescript/src/net/internal/multiaddr.ts 18.51% 22 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5859      +/-   ##
==========================================
- Coverage   23.01%   22.90%   -0.11%     
==========================================
  Files         569      572       +3     
  Lines       19087    19389     +302     
  Branches     2709     2745      +36     
==========================================
+ Hits         4392     4442      +50     
- Misses      14643    14894     +251     
- Partials       52       53       +1     
Flag Coverage Δ
apps.hash-ai-worker-ts 1.32% <ø> (ø)
apps.hash-api 1.16% <ø> (ø)
local.harpc-client 58.98% <14.41%> (-9.16%) ⬇️
local.hash-backend-utils 8.81% <ø> (ø)
local.hash-graph-sdk 58.62% <ø> (ø)
local.hash-isomorphic-utils 1.00% <ø> (ø)
local.hash-subgraph 24.54% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

TimDiekmann
TimDiekmann previously approved these changes Dec 11, 2024
Co-authored-by: Tim Diekmann <21277928+TimDiekmann@users.noreply.github.com>
Copy link
Contributor

Benchmark results

@rust/hash-graph-benches – Integrations

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$18.1 \mathrm{ms} \pm 228 \mathrm{μs}\left({\color{lightgreen}-27.259 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$18.0 \mathrm{ms} \pm 190 \mathrm{μs}\left({\color{gray}4.35 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$18.8 \mathrm{ms} \pm 237 \mathrm{μs}\left({\color{gray}-2.095 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$19.1 \mathrm{ms} \pm 229 \mathrm{μs}\left({\color{red}8.48 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$18.2 \mathrm{ms} \pm 242 \mathrm{μs}\left({\color{gray}2.57 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$17.6 \mathrm{ms} \pm 223 \mathrm{μs}\left({\color{gray}1.49 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$18.4 \mathrm{ms} \pm 205 \mathrm{μs}\left({\color{gray}2.55 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$18.0 \mathrm{ms} \pm 245 \mathrm{μs}\left({\color{lightgreen}-28.162 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$17.9 \mathrm{ms} \pm 243 \mathrm{μs}\left({\color{red}10.6 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property depths: DT=255, PT=255, ET=255, E=255 $$68.4 \mathrm{ms} \pm 273 \mathrm{μs}\left({\color{gray}-1.499 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=0 $$41.4 \mathrm{ms} \pm 91.9 \mathrm{μs}\left({\color{gray}1.88 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=2, PT=2, ET=2, E=2 $$59.6 \mathrm{ms} \pm 223 \mathrm{μs}\left({\color{gray}0.422 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=2 $$45.6 \mathrm{ms} \pm 140 \mathrm{μs}\left({\color{gray}0.896 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=2, E=2 $$51.2 \mathrm{ms} \pm 168 \mathrm{μs}\left({\color{gray}-1.268 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=2, ET=2, E=2 $$55.3 \mathrm{ms} \pm 164 \mathrm{μs}\left({\color{gray}-0.962 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=255, PT=255, ET=255, E=255 $$114 \mathrm{ms} \pm 416 \mathrm{μs}\left({\color{gray}0.686 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=0 $$45.1 \mathrm{ms} \pm 222 \mathrm{μs}\left({\color{gray}-0.488 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=2, PT=2, ET=2, E=2 $$106 \mathrm{ms} \pm 517 \mathrm{μs}\left({\color{gray}1.49 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=2 $$86.5 \mathrm{ms} \pm 412 \mathrm{μs}\left({\color{gray}-0.691 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=2, E=2 $$96.2 \mathrm{ms} \pm 272 \mathrm{μs}\left({\color{gray}0.178 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=2, ET=2, E=2 $$101 \mathrm{ms} \pm 269 \mathrm{μs}\left({\color{gray}-0.001 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579 $$1.42 \mathrm{ms} \pm 5.96 \mathrm{μs}\left({\color{gray}0.081 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_one_depth

Function Value Mean Flame graphs
entity_by_id 50 entities $$280 \mathrm{ms} \pm 1.54 \mathrm{ms}\left({\color{gray}3.11 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$27.2 \mathrm{ms} \pm 116 \mathrm{μs}\left({\color{gray}2.77 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$20.6 \mathrm{ms} \pm 61.4 \mathrm{μs}\left({\color{gray}1.05 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$54.6 \mathrm{ms} \pm 2.56 \mathrm{ms}\left({\color{gray}-4.331 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$181 \mathrm{ms} \pm 816 \mathrm{μs}\left({\color{gray}0.946 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$1.99 \mathrm{ms} \pm 6.71 \mathrm{μs}\left({\color{gray}1.74 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$2.22 \mathrm{ms} \pm 7.83 \mathrm{μs}\left({\color{gray}4.04 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$1.99 \mathrm{ms} \pm 5.47 \mathrm{μs}\left({\color{gray}1.91 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$3.09 \mathrm{ms} \pm 16.8 \mathrm{μs}\left({\color{red}5.78 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$14.9 \mathrm{ms} \pm 43.3 \mathrm{μs}\left({\color{red}48.9 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_zero_depth

Function Value Mean Flame graphs
entity_by_id 50 entities $$4.28 \mathrm{ms} \pm 26.0 \mathrm{μs}\left({\color{lightgreen}-16.454 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$2.01 \mathrm{ms} \pm 11.2 \mathrm{μs}\left({\color{gray}2.10 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.97 \mathrm{ms} \pm 9.73 \mathrm{μs}\left({\color{gray}1.07 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$2.23 \mathrm{ms} \pm 11.3 \mathrm{μs}\left({\color{gray}2.87 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$3.50 \mathrm{ms} \pm 18.3 \mathrm{μs}\left({\color{gray}4.25 \mathrm{\%}}\right) $$ Flame Graph

Merged via the queue into main with commit 134b75f Dec 11, 2024
166 checks passed
@indietyp indietyp deleted the bm/h-3778-investigate-connections-being-opened-indiscriminately-with branch December 11, 2024 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-api Affects the HASH API (app) area/apps area/deps Relates to third-party dependencies (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team
Development

Successfully merging this pull request may close these issues.

2 participants