Hammer/max downloads #909

aaronchongth · 2024-02-28T16:25:22Z

What's new

Exporting CSV now queries in chunks instead of all in one go
New global backdrop loading screen and events to trigger it, lets users know that the exporting is underway (which could take sometime)
Made export and refresh buttons more explicit

export-2024-03-08_15.23.58.mp4

Normal resolution

Low resolution

Self-checks

I have prototyped this new feature (if necessary) on Figma
I'm familiar with and follow this Typescript guideline
I added unit-tests for new components
I tried testing edge cases
I tested the behavior of the components that interact with the backend, with an e2e test

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

codecov · 2024-02-28T16:30:14Z

Codecov Report

Attention: Patch coverage is 0% with 54 lines in your changes are missing coverage. Please review.

Project coverage is 46.76%. Comparing base (fe0e808) to head (efa1cd5).
Report is 68 commits behind head on deploy/hammer.

Files	Patch %	Lines
...kages/dashboard/src/components/tasks/tasks-app.tsx	0.00%	45 Missing ⚠️
packages/dashboard/src/components/app-base.tsx	0.00%	8 Missing ⚠️
packages/dashboard/src/components/tasks/utils.ts	0.00%	1 Missing ⚠️

Additional details and impacted files

@@                Coverage Diff                @@
##           deploy/hammer     #909      +/-   ##
=================================================
- Coverage          49.35%   46.76%   -2.60%     
=================================================
  Files                285      204      -81     
  Lines               7564     6266    -1298     
  Branches            1050      833     -217     
=================================================
- Hits                3733     2930     -803     
+ Misses              3682     3330     -352     
+ Partials             149        6     -143

Flag	Coverage Δ
dashboard	`13.74% <0.00%> (-1.32%)`	⬇️
react-components	`?`
rmf-auth	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

packages/api-server/api_server/dependencies.py

packages/api-client/schema/index.ts

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

packages/api-server/api_server/dependencies.py

packages/api-server/api_server/routes/tasks/tasks.py

packages/dashboard/src/components/tasks/tasks-app.tsx

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

…ck to 31 days of querying Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

packages/dashboard/src/components/tasks/tasks-app.tsx

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

…t month Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

koonpeng · 2024-03-11T04:45:28Z

packages/dashboard/src/components/tasks/tasks-app.tsx

          return;
        }
        if (minimal) {
-          downloadCsvMinimal(now, allTasks, allTaskRequests);
+          downloadCsvMinimal(now, pastMonthTasks, pastMonthTaskRequests);


"download*" is misleading since it doesn't actually download, suggest to rename it to "save*" or add docs.

It's weird that "minimal" contains info that is not in "full" (from what I see, pickup and dropoff), I will say to include the task request in full so it has the same data.

Parsing the task request for pick and dropoff is also a red flag, it is out of scope of this PR so just add a FIXME. In future PR we can look at using task tags to store this information and just dump it, it also solves the issue of requiring double query to get all the required data.

Changing it to export, 651562d

Technically, full contains all the information of minimal and much more, since any number of pickups and dropoffs can be parsed from the task phases if needed. And since we will be adding pick and dropoff information in labels soon, I will not make any changes to this.

FIXME was added previously d126767

Technically, full contains all the information of minimal and much more, since any number of pickups and dropoffs can be parsed from the task phases if needed. And since we will be adding pick and dropoff information in labels soon, I will not make any changes to this.

Then we don't need to fetch task requests for "minimal"? Or do you mean that it will be included after we add the tags? If so then add a TODO.

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

koonpeng

Technically, full contains all the information of minimal and much more, since any number of pickups and dropoffs can be parsed from the task phases if needed. And since we will be adding pick and dropoff information in labels soon, I will not make any changes to this.

Then we don't need to fetch task requests for "minimal"? Or do you mean that it will be included after we add the tags? If so then add a TODO.

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

aaronchongth · 2024-03-11T07:33:41Z

Technically, full contains all the information of minimal and much more, since any number of pickups and dropoffs can be parsed from the task phases if needed. And since we will be adding pick and dropoff information in labels soon, I will not make any changes to this.

Then we don't need to fetch task requests for "minimal"? Or do you mean that it will be included after we add the tags? If so then add a TODO.

The ticket detailing this is already in our deployment project board, will not be adding additional TODOs here.

aaronchongth added 2 commits February 29, 2024 00:17

Use a limit of 1000 for all-task queries

32eb4ad

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Raising limit to 5000

f40b701

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

aaronchongth requested a review from koonpeng February 29, 2024 02:39

koonpeng requested changes Feb 29, 2024

View reviewed changes

packages/api-server/api_server/dependencies.py Outdated Show resolved Hide resolved

packages/api-client/schema/index.ts Outdated Show resolved Hide resolved

aaronchongth added 6 commits March 8, 2024 01:22

Added new API to get number of task states

c2dae72

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

First attempt at querying by chunks with a loading backdrop

35f61ff

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Count tasks based on query as well

6014ead

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Chunk-based query with global loading backdrop

1cc0fa2

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Query requests in chunks as well

9b68089

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Clean up stale comments

16617a8

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

aaronchongth requested a review from koonpeng March 8, 2024 07:24

koonpeng requested changes Mar 8, 2024

View reviewed changes

aaronchongth added 5 commits March 9, 2024 00:12

Remove count endpoint, revert changes to API

40b91f0

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Revert routes file

0353728

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Using normal pagination queries to query until the tasks run out, sti…

1de251a

…ck to 31 days of querying Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Use start time for query

d911940

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Use Button instead of IconButton for export and refresh

18a721f

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

aaronchongth requested a review from koonpeng March 8, 2024 18:08

Remove stale import

f5ed5e9

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

koonpeng requested changes Mar 11, 2024

View reviewed changes

packages/dashboard/src/components/tasks/tasks-app.tsx Show resolved Hide resolved

aaronchongth added 2 commits March 11, 2024 10:44

Add a fixme for still querying for task requests during csv export

d126767

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

Rename variables more accurately since it will only query for the pas…

ac4a6c7

…t month Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

aaronchongth requested a review from koonpeng March 11, 2024 03:18

koonpeng requested changes Mar 11, 2024

View reviewed changes

Changing download to export

651562d

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

aaronchongth requested a review from koonpeng March 11, 2024 06:39

koonpeng approved these changes Mar 11, 2024

View reviewed changes

Moving task request query for only minimal export

efa1cd5

Signed-off-by: Aaron Chong <aaronchongth@gmail.com>

aaronchongth merged commit a0e2f8b into deploy/hammer Mar 11, 2024
2 checks passed

aaronchongth deleted the hammer/max-downloads branch March 11, 2024 07:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hammer/max downloads #909

Hammer/max downloads #909

aaronchongth commented Feb 28, 2024 •

edited

Loading

codecov bot commented Feb 28, 2024 •

edited

Loading

koonpeng Mar 11, 2024

aaronchongth Mar 11, 2024

koonpeng Mar 11, 2024 •

edited

Loading

koonpeng left a comment

aaronchongth commented Mar 11, 2024

Hammer/max downloads #909

Hammer/max downloads #909

Conversation

aaronchongth commented Feb 28, 2024 • edited Loading

What's new

Self-checks

codecov bot commented Feb 28, 2024 • edited Loading

Codecov Report

koonpeng Mar 11, 2024

Choose a reason for hiding this comment

aaronchongth Mar 11, 2024

Choose a reason for hiding this comment

koonpeng Mar 11, 2024 • edited Loading

Choose a reason for hiding this comment

koonpeng left a comment

Choose a reason for hiding this comment

aaronchongth commented Mar 11, 2024

aaronchongth commented Feb 28, 2024 •

edited

Loading

codecov bot commented Feb 28, 2024 •

edited

Loading

koonpeng Mar 11, 2024 •

edited

Loading