Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datafile Parse Latency Bug #2778

Merged
merged 2 commits into from
Dec 13, 2023
Merged

Datafile Parse Latency Bug #2778

merged 2 commits into from
Dec 13, 2023

Conversation

elipe17
Copy link

@elipe17 elipe17 commented Dec 12, 2023

Summary of Changes

How to Test

cd tdrs-frontend && docker-compose up
cd tdrs-backend && docker-compose up
  • Open http://localhost:3000/ and sign in.
  • Submit a datafile with lots of ParserErrors
  • Verify you don't see a bunch of ParserErrors being printed to the console

Deliverables

More details on how deliverables herein are assessed included here.

Deliverable 1: Accepted Features

Checklist of ACs:

  • Less latency in datafile parsing

Deliverable 2: Tested Code

  • Are all areas of code introduced in this PR meaningfully tested?
    • If this PR introduces backend code changes, are they meaningfully tested?
    • If this PR introduces frontend code changes, are they meaningfully tested?
  • Are code coverage minimums met?
    • Frontend coverage: [insert coverage %] (see CodeCov Report comment in PR)
    • Backend coverage: [insert coverage %] (see CodeCov Report comment in PR)

Deliverable 3: Properly Styled Code

  • Are backend code style checks passing on CircleCI?
  • Are frontend code style checks passing on CircleCI?
  • Are code maintainability principles being followed?

Deliverable 4: Accessible

  • Does this PR complete the epic?
  • Are links included to any other gov-approved PRs associated with epic?
  • Does PR include documentation for Raft's a11y review?
  • Did automated and manual testing with iamjolly and ttran-hub using Accessibility Insights reveal any errors introduced in this PR?

Deliverable 5: Deployed

  • Was the code successfully deployed via automated CircleCI process to development on Cloud.gov?

Deliverable 6: Documented

  • Does this PR provide background for why coding decisions were made?
  • If this PR introduces backend code, is that code easy to understand and sufficiently documented, both inline and overall?
  • If this PR introduces frontend code, is that code easy to understand and sufficiently documented, both inline and overall?
  • If this PR introduces dependencies, are their licenses documented?
  • Can reviewer explain and take ownership of these elements presented in this code review?

Deliverable 7: Secure

  • Does the OWASP Scan pass on CircleCI?
  • Do manual code review and manual testing detect any new security issues?
  • If new issues detected, is investigation and/or remediation plan documented?

Deliverable 8: User Research

Research product(s) clearly articulate(s):

  • the purpose of the research
  • methods used to conduct the research
  • who participated in the research
  • what was tested and how
  • impact of research on TDP
  • (if applicable) final design mockups produced for TDP development

Copy link

codecov bot commented Dec 12, 2023

Codecov Report

Merging #2778 (d30efc5) into develop (e5ccc2c) will decrease coverage by 0.02%.
Report is 1 commits behind head on develop.
The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2778      +/-   ##
===========================================
- Coverage    92.80%   92.78%   -0.02%     
===========================================
  Files          246      246              
  Lines         5576     5575       -1     
  Branches       480      479       -1     
===========================================
- Hits          5175     5173       -2     
- Misses         308      309       +1     
  Partials        93       93              
Flag Coverage Δ
dev-backend 92.77% <ø> (-0.03%) ⬇️
dev-frontend 92.83% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
tdrs-backend/tdpservice/parsers/models.py 92.06% <ø> (-1.69%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b11e1a9...d30efc5. Read the comment docs.

@elipe17 elipe17 requested a review from ADPennington December 12, 2023 21:04
@ADPennington ADPennington added the Deploy with CircleCI-qasp Deploy to https://tdp-frontend-qasp.app.cloud.gov through CircleCI label Dec 12, 2023
Copy link
Collaborator

@ADPennington ADPennington left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the quick fix @elipe17

logs a lot less cluttered when hitting the data files page in DAC 🚀

14:06:08.747243+00:00
09:06:08.748: [APP/PROC/WEB.0] 10.255.222.82 - - [13/Dec/2023:14:06:08 +0000] "GET /v1/auth_check HTTP/1.0" 200 1097 "https://tdp-frontend-qasp.app.cloud.gov/data-files" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
09:06:09.017: [APP/PROC/WEB.0] 10.255.222.82 - - [13/Dec/2023:14:06:09 +0000] "GET /admin/data_files/datafile/ HTTP/1.0" 200 122223 "https://tdp-frontend-qasp.app.cloud.gov/admin/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
09:06:10.482: [APP/PROC/WEB.0] [2023-12-13 14:06:10 +0000] [130] [DEBUG] GET /admin/parsers/parsererror/
09:06:10.605: [APP/PROC/WEB.0] 10.255.222.82 - - [13/Dec/2023:14:06:10 +0000] "GET /admin/parsers/parsererror/?file=55 HTTP/1.0" 200 74901 "https://tdp-frontend-qasp.app.cloud.gov/admin/data_files/datafile/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
09:06:10.727: [APP/PROC/WEB.0] [2023-12-13 14:06:10 +0000] [130] [DEBUG] GET /admin/jsi18n/
09:06:10.766: [APP/PROC/WEB.0] 10.255.222.82 - - [13/Dec/2023:14:06:10 +0000] "GET /admin/jsi18n/ HTTP/1.0" 200 3195 "https://tdp-frontend-qasp.app.cloud.gov/admin/parsers/parsererror/?file=55" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
09:06:25.976: [APP/PROC/WEB.0] [2023-12-13 14:06:25 +0000] [132] [DEBUG] GET /admin/data_files/datafile/
09:06:26.880: [APP/PROC/WEB.0] 10.255.222.82 - - [13/Dec/2023:14:06:26 +0000] "GET /admin/data_files/datafile/ HTTP/1.0" 200 122223 "https://tdp-frontend-qasp.app.cloud.gov/admin/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"

@ADPennington ADPennington added Ready to Merge and removed QASP Review Deploy with CircleCI-qasp Deploy to https://tdp-frontend-qasp.app.cloud.gov through CircleCI labels Dec 13, 2023
@andrew-jameson andrew-jameson merged commit 68e2aba into develop Dec 13, 2023
26 checks passed
@andrew-jameson andrew-jameson deleted the 2777-print-bug branch December 13, 2023 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[SPIKE] Investigate latency in data files model
5 participants