You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Get a baseline duration of current tiles and points jobs on C5 when full capacity is allocated.
Remove the year and basis of record and capture the timings, with same compute capacity, ensuring it is all allocated.
This can be done by changing the SQL to singular values to e.g. SELECT ... 2024 AS year, 'HUMAN_OBSERVATION' AS basisOfRecord in the relevant queries.
Rationale:
We are aware of growing capacity constraints, and removing these controls on the revised GBIF.org interface can be considered (possibly desirable). Users can still filter by year and basis of record driven from the adhoc mapping searches. It would of course require a deprecation and removal of the v2 map API and a simpler v3 version. Removal of pre-calculated views may also be less wasteful (electricity) for rarely used views. An alternative could be to stop precalculating altogether but make use of a fast aggregation store, such as clickhouse (see demo and blog)
For now though, let's just understand the potential savings in compute with existing code.
The text was updated successfully, but these errors were encountered:
A performance test to evaluate potential gains.
Get a baseline duration of current tiles and points jobs on C5 when full capacity is allocated.
Remove the year and basis of record and capture the timings, with same compute capacity, ensuring it is all allocated.
This can be done by changing the SQL to singular values to e.g.
SELECT ... 2024 AS year, 'HUMAN_OBSERVATION' AS basisOfRecord
in the relevant queries.Rationale:
We are aware of growing capacity constraints, and removing these controls on the revised GBIF.org interface can be considered (possibly desirable). Users can still filter by year and basis of record driven from the adhoc mapping searches. It would of course require a deprecation and removal of the
v2
map API and a simplerv3
version. Removal of pre-calculated views may also be less wasteful (electricity) for rarely used views. An alternative could be to stop precalculating altogether but make use of a fast aggregation store, such as clickhouse (see demo and blog)For now though, let's just understand the potential savings in compute with existing code.
The text was updated successfully, but these errors were encountered: