-
-
Notifications
You must be signed in to change notification settings - Fork 464
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixed: Speed up Automated Tests by sharing apks. #3747
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #3747 +/- ##
============================================
+ Coverage 52.08% 53.16% +1.08%
- Complexity 1268 1291 +23
============================================
Files 292 292
Lines 10983 10983
Branches 1455 1455
============================================
+ Hits 5720 5839 +119
+ Misses 4306 4188 -118
+ Partials 957 956 -1 ☔ View full report in Codecov by Sentry. |
f6e1b55
to
0526cbb
Compare
@kelson42 After these all changes, now it seems CI is stable becuase CI passed twice in a row: |
0526cbb
to
f2b62d9
Compare
* Improved our CI to generate the test apk, we are using this in our all test cases. * Added apk to `testInstrumentationRunnerArguments` option which takes the test apk when running the test cases.
…tions.sleep` in our test cases. Because `BaristaSleepInteractions.sleep` runs everytime even the view is displaying on the screen which takes the extra time to run the test case, but now we are using `uiDevice.waitForIdle()` method which perform the further operation if desired view is already displaying otherwise wait for that view. * Improved the deleting of zim files in localibrary, since we already pause for a second to check if the delete dialog is displayed or not, and again we are waiting before clicking on it. The same issue for while clicking on the PlayStoreRestrictionDialogTest where we are first waiting for the dialog to visible and then waiting for clicking on it. So we have removed the second wait since this dialog is displaying in both test cases. * Improved the TopLevelDestinationTest. Because we were testing the drawer options fragments (Settings, Bookmarks etc) from the download screen and when we pressed the back button it came to the download screen and refreshed its data due to this our next functionality had to wait to finish this. Now we are testing all the navigation screens from the LocalLibraryScreen which makes this test case faster.
…saved bookmarks. * Refresh the data before checking the loaded data in DownloadTest. * Increasing the Retry count for loading data and downloading start to properly work with low internet connection.
* Improved our `ZimHostFragmentTest` test to boost the test case speed. It was running in 2 minutes, but after this change, it now completes in 27 seconds.
…e failures occurred multiple times due to the heavy `google_apis`. so to address this we have increased it to 5 to properly run the test cases. * We are removing the `ZimHostFragment` test from the API level 33 because, most of the time, the emulator does not have WiFi service. Running this test on this emulator is not worthwhile, as we are testing this code on WiFi.
89657df
to
16e8cad
Compare
34e4667
to
026dcac
Compare
…nally crashes during testing on API level 33 due to emulator lag on google_apis. The extensive data migration exacerbates the emulator's performance issues, leading to crashes during subsequent test processes. To address this issue, we have reduced the migration volume from 10K to 1K. * Increased the disk size of API level 33 emulator since emulator is crashing on this api level while performing the test cases so it might require more resource to run smoothly. * Clearing the previous installed application data when launching the test cases for the first time to properly run the test cases. * Removed the enabling wifi for adb as it is unused now because sometimes API level 33 emulator does not have the wifi, and we have introduced this for API level 33 emulator. * Removed the APK sharing in test cases since if a text case fails it crashed all the test cases even we re-run that test case in second run.
026dcac
to
9107c4a
Compare
@MohitMaliDeveloper So we can remove the draft? |
f1bc0a2
to
dbbd02b
Compare
…uery` test cases on API level 33 since numerous security updates have been included, preventing us from modifying the default behavior of ContentResolver. So running the test case on this APi level leads to test failure.
@kelson42 Not now, CI is not stable right now, the last run was unsuccessful(Before the last run CI was green), I have made some new changes in CI. Hope this will stabilize the CI. |
… sometimes the emulator treats app-specific directory as `READ_ONLY_FILESYSTEM`.
…ault it has 10 second but the API level 33 emulator is heavy and takes more time to become idle so we have increased the timeout to properly come in the idle state.
* Improved the functionality of closing dialog where any system dialog is visible on the window.
…, the app-specific path is occasionally treated as READ_ONLY. Additionally, for certain test cases where we need to load the zim file in the LocalLibraryScreen, we've included the test directory in our scanning process, as it will be automatically handled by the OS.
…ame data directory is not supported.
…ame data directory is not supported.
… data before running the test cases.
Duplicate of #3758. |
Fixes #2096
uiDevice.waitForIdle()
method instead ofBaristaSleepInteractions.sleep
in our test cases. BecauseBaristaSleepInteractions.sleep
runs everytime even the view is displaying on the screen which takes the extra time to run the test case, but now we are usinguiDevice.waitForIdle()
method which perform the further operation if desired view is already displaying otherwise wait for that view.ObjectBoxToLibkiwixMigratorTest
for clearing the previously saved bookmarks.ZimHostFragmentTest
test to boost the test case speed. It was running in 2 minutes, but after this change, it now completes in 27 seconds.google_apis
. so to address this we have increased it to 5 to properly run the test cases.ZimHostFragment
test from the API level 33 because, most of the time, the emulator does not have WiFi service. Running this test on this emulator is not worthwhile, as we are testing this code on WiFi see https://github.com/kiwix/kiwix-android/actions/runs/8265444407/job/22611097563?pr=3747#step:6:1980.READ_ONLY_FILESYSTEM
see https://github.com/kiwix/kiwix-android/actions/runs/8308488756/job/22738768095?pr=3747#step:5:12547.Before test cases were completed on API level 33(Android 13) in 18 Minutes and 57 seconds and by doing the above changes now it is completed in 14 minutes on the same API level so almost 5 minutes have been reduced.