Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H-2976: Improve validation for ontology types #5827

Merged
merged 2 commits into from
Dec 9, 2024

Conversation

TimDiekmann
Copy link
Member

🌟 What is the purpose of this PR?

Some ontology types don't make sense, e.g. having a minimum higher than the maximum or an allOf: [number, string]. An error should be emitted in this case.

Entity/property types should be validated: e.g. required only specifies actual properties.

The references don't need to be validated up-front as they will be checked when inserted into the graph anyway.

🔍 What does this change?

By now, most of the schemas are either validated when created or when creating the closed variant of it.
This PR adds a validation for array property types to ensure that minItems must be less than or equal to maxItems.
Further, it allows required to point to a property which is specified in a parent entity type (by removing this constraint on the EntityType but keep it in ClosedEntityType).

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

@TimDiekmann TimDiekmann self-assigned this Dec 6, 2024
@github-actions github-actions bot added the area/libs Relates to first-party libraries/crates/packages (area) label Dec 6, 2024
@TimDiekmann TimDiekmann requested a review from CiaranMn December 6, 2024 14:37
@TimDiekmann TimDiekmann marked this pull request as ready for review December 6, 2024 14:37
@TimDiekmann TimDiekmann enabled auto-merge December 6, 2024 14:37
@TimDiekmann TimDiekmann changed the title Improve validation for ontology types H-2976: Improve validation for ontology types Dec 6, 2024
Copy link

codecov bot commented Dec 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 22.88%. Comparing base (42c1bfa) to head (7565585).
Report is 14 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5827      +/-   ##
==========================================
- Coverage   23.00%   22.88%   -0.12%     
==========================================
  Files         568      568              
  Lines       19165    19201      +36     
  Branches     2715     2732      +17     
==========================================
- Hits         4408     4395      -13     
- Misses      14705    14754      +49     
  Partials       52       52              
Flag Coverage Δ
apps.hash-ai-worker-ts 1.32% <ø> (ø)
apps.hash-api 1.16% <ø> (+<0.01%) ⬆️
blockprotocol.type-system 46.42% <ø> (ø)
local.hash-backend-utils 8.80% <ø> (ø)
local.hash-graph-sdk 100.00% <ø> (ø)
local.hash-isomorphic-utils 0.99% <ø> (ø)
local.hash-subgraph 24.54% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

github-actions bot commented Dec 9, 2024

Benchmark results

@rust/hash-graph-benches – Integrations

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$16.3 \mathrm{ms} \pm 179 \mathrm{μs}\left({\color{lightgreen}-30.335 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$16.8 \mathrm{ms} \pm 185 \mathrm{μs}\left({\color{gray}4.16 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$16.1 \mathrm{ms} \pm 171 \mathrm{μs}\left({\color{gray}1.30 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$16.8 \mathrm{ms} \pm 201 \mathrm{μs}\left({\color{gray}-0.443 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$16.1 \mathrm{ms} \pm 158 \mathrm{μs}\left({\color{gray}-1.644 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$16.3 \mathrm{ms} \pm 177 \mathrm{μs}\left({\color{red}5.37 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$16.2 \mathrm{ms} \pm 196 \mathrm{μs}\left({\color{gray}3.71 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$15.7 \mathrm{ms} \pm 171 \mathrm{μs}\left({\color{lightgreen}-17.527 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$16.0 \mathrm{ms} \pm 175 \mathrm{μs}\left({\color{gray}-1.248 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property depths: DT=255, PT=255, ET=255, E=255 $$64.6 \mathrm{ms} \pm 230 \mathrm{μs}\left({\color{gray}-0.604 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=0 $$37.1 \mathrm{ms} \pm 166 \mathrm{μs}\left({\color{gray}0.524 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=2, PT=2, ET=2, E=2 $$55.6 \mathrm{ms} \pm 250 \mathrm{μs}\left({\color{gray}0.689 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=2 $$41.2 \mathrm{ms} \pm 266 \mathrm{μs}\left({\color{gray}0.055 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=2, E=2 $$47.1 \mathrm{ms} \pm 374 \mathrm{μs}\left({\color{gray}0.597 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=2, ET=2, E=2 $$51.8 \mathrm{ms} \pm 237 \mathrm{μs}\left({\color{gray}0.828 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=255, PT=255, ET=255, E=255 $$106 \mathrm{ms} \pm 419 \mathrm{μs}\left({\color{gray}-0.953 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=0 $$41.6 \mathrm{ms} \pm 374 \mathrm{μs}\left({\color{gray}0.856 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=2, PT=2, ET=2, E=2 $$97.7 \mathrm{ms} \pm 453 \mathrm{μs}\left({\color{gray}0.633 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=2 $$79.9 \mathrm{ms} \pm 329 \mathrm{μs}\left({\color{gray}0.398 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=2, E=2 $$88.9 \mathrm{ms} \pm 509 \mathrm{μs}\left({\color{gray}0.323 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=2, ET=2, E=2 $$93.2 \mathrm{ms} \pm 487 \mathrm{μs}\left({\color{gray}-0.538 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579 $$1.39 \mathrm{ms} \pm 8.71 \mathrm{μs}\left({\color{gray}1.02 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_one_depth

Function Value Mean Flame graphs
entity_by_id 50 entities $$267 \mathrm{ms} \pm 2.99 \mathrm{ms}\left({\color{gray}-0.017 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$26.4 \mathrm{ms} \pm 189 \mathrm{μs}\left({\color{gray}0.959 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$20.2 \mathrm{ms} \pm 142 \mathrm{μs}\left({\color{gray}0.724 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$56.3 \mathrm{ms} \pm 210 \mathrm{μs}\left({\color{red}54.8 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$177 \mathrm{ms} \pm 950 \mathrm{μs}\left({\color{gray}1.73 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$1.89 \mathrm{ms} \pm 6.13 \mathrm{μs}\left({\color{gray}-1.170 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$2.07 \mathrm{ms} \pm 8.75 \mathrm{μs}\left({\color{gray}-0.739 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$1.92 \mathrm{ms} \pm 5.52 \mathrm{μs}\left({\color{gray}-0.134 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$3.12 \mathrm{ms} \pm 10.6 \mathrm{μs}\left({\color{gray}4.60 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$13.4 \mathrm{ms} \pm 40.4 \mathrm{μs}\left({\color{gray}0.118 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_zero_depth

Function Value Mean Flame graphs
entity_by_id 50 entities $$5.04 \mathrm{ms} \pm 24.8 \mathrm{μs}\left({\color{red}26.6 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$1.93 \mathrm{ms} \pm 11.8 \mathrm{μs}\left({\color{gray}0.019 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.90 \mathrm{ms} \pm 5.30 \mathrm{μs}\left({\color{gray}-0.787 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$2.13 \mathrm{ms} \pm 8.23 \mathrm{μs}\left({\color{gray}-0.135 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$3.28 \mathrm{ms} \pm 19.5 \mathrm{μs}\left({\color{gray}-0.936 \mathrm{\%}}\right) $$ Flame Graph

Merged via the queue into main with commit 43abc52 Dec 9, 2024
101 of 102 checks passed
@TimDiekmann TimDiekmann deleted the t/h-2976-validate-ontology-types-on-creation branch December 9, 2024 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/libs Relates to first-party libraries/crates/packages (area)
Development

Successfully merging this pull request may close these issues.

2 participants