Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix_: bad node config migration #21952

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Conversation

qfrank
Copy link
Contributor

@qfrank qfrank commented Jan 20, 2025

For details, pls see description in status-go PR

General test steps:

  • install release apk v1.10.1 (I haven't tried earlier version)
  • create a new account, set log level to debug so we can track the log
  • install release apk v2.30.0
  • try the first login with correct password after upgrading which will trigger MigrateNodeConfig, you will get something like failed to open database: table node_config has no column named connector_enabled
  • try the second login with correct password, will get follow error:
 loginAccount failed                      error="failed to start node: Key: 'NodeConfig.NetworkID' Error:Field validation for 'NetworkID' failed on the 'required' tag\nKey: 'NodeConfig.LogLevel' Error:Field validation for 'LogLevel' failed on the 'eq=ERROR|eq=WARN|eq=INFO|eq=DEBUG|eq=TRACE' tag"
  • install release build of this PR and see if user can login successfully

Affected Platform:

  • iOS
  • android

@qfrank qfrank requested review from ilmotta and flexsurfer January 20, 2025 01:16
@qfrank qfrank self-assigned this Jan 20, 2025
@qfrank qfrank marked this pull request as ready for review January 20, 2025 01:17
@status-im-auto
Copy link
Member

status-im-auto commented Jan 20, 2025

Jenkins Builds

Click to see older builds (8)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 1f2e11e #1 2025-01-20 01:22:41 ~6 min tests 📄log
✔️ 1f2e11e #1 2025-01-20 01:26:04 ~9 min android-e2e 🤖apk 📲
✔️ 1f2e11e #1 2025-01-20 01:26:23 ~10 min android 🤖apk 📲
✔️ 1f2e11e #1 2025-01-20 01:27:57 ~11 min ios 📱ipa 📲
✔️ 70768b6 #2 2025-01-20 14:34:34 ~6 min tests 📄log
✔️ 70768b6 #2 2025-01-20 14:37:31 ~9 min android-e2e 🤖apk 📲
✔️ 70768b6 #2 2025-01-20 14:37:58 ~9 min android 🤖apk 📲
✔️ 70768b6 #2 2025-01-20 14:40:55 ~12 min ios 📱ipa 📲
Commit #️⃣ Finished (UTC) Duration Platform Result
0531a80 #3 2025-01-21 11:30:53 ~5 min tests 📄log
✔️ 0531a80 #3 2025-01-21 11:33:19 ~7 min android-e2e 🤖apk 📲
✔️ 0531a80 #3 2025-01-21 11:37:54 ~12 min android 🤖apk 📲
✔️ 0531a80 #3 2025-01-21 11:38:13 ~12 min ios 📱ipa 📲
✔️ 37ef6f3 #4 2025-01-23 06:53:39 ~4 min tests 📄log
✔️ 37ef6f3 #4 2025-01-23 06:57:18 ~8 min android 🤖apk 📲
✔️ 37ef6f3 #4 2025-01-23 06:57:25 ~8 min android-e2e 🤖apk 📲
✔️ 37ef6f3 #4 2025-01-23 06:59:22 ~10 min ios 📱ipa 📲

Copy link
Contributor

@ilmotta ilmotta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@qfrank, I followed the steps from this PR description in a real device with Android 13. I got locked out as expected and this PR did fix the migration bug!

In order to test this PR I had to manually generate a release build for Android https://ci.infra.status.im/job/status-mobile/job/platforms/job/android/2084/ because I installed 1.10.1 and 2.30.0 using release builds, so the upgrade had to be a release build.

I'm wondering about the safest way for @3esmit to test the release build in a way that doesn't risk making things worse for him. Probably the database mutation caused by the call to SaveConfigWithTx is fine if he needs to downgrade to his current build after testing this PR's release build, correct?

src/status_im/contexts/profile/config.cljs Outdated Show resolved Hide resolved
@qfrank qfrank force-pushed the fix/node_config_migration branch from 0531a80 to 37ef6f3 Compare January 23, 2025 06:48
@pavloburykh pavloburykh self-assigned this Jan 23, 2025
@pavloburykh
Copy link
Contributor

@qfrank Thank you for this PR, great work! While waiting for reviews and approvals of both this PR and the corresponding go PR, I’ve started some initial testing.

I’ve identified that the latest Status v1 version resulting in the issue after upgrading to Status v2 is v1.18.0. The issue does not appear to reproduce when upgrading from versions >v1.18.0, though this doesn’t entirely rule out the possibility, as we’ve previously seen it occur after an upgrade from v1.20.6. However, those cases were random, and we don’t yet have clear steps to reproduce them. In contrast, upgrading from versions ≤v1.18.0 consistently leads to the issue after upgrading to Status v2 (100% reproducibility).

Interestingly, if a user previously upgraded from v1.18.0 or earlier to v1.19.0 or later, they can successfully log in after upgrading to any Status v2 version (e.g., v2.30, v2.31, or v2.32). In such cases, the issue does not reproduce.

Based on current results, it looks like this PR resolves the issue. Below is a table summarizing the upgrade flows I’ve tested so far:

Upgrade Flow Result
v1.18.0 → v2.30 Cannot log in (issue reproducible)
v1.18.0 → v2.32 Cannot log in (issue reproducible)
v1.18.0 → v2.32 → PR version Can log in (PR build fixes issue)
v1.18.0 → v2.30 → PR version Can log in (PR build fixes issue)
v1.19.0 → v2.30 Can log in (issue not reproducible)
v1.18.0 → v1.19.0 → v2.30 Can log in (issue not reproducible)
v1.18.0 → v1.20.6 → v2.30 Can log in (issue not reproducible)

@qfrank Please ping me once these PRs receive approval, and I’ll proceed with further testing. Ideally, both the go and mobile PRs should have at least two approving reviews before moving forward. cc @ilmotta

@ilmotta
Copy link
Contributor

ilmotta commented Jan 23, 2025

Awesome work @pavloburykh. Interesting findings!

I approved the PR now. Our original plan with @qfrank was to provide a release build to @3esmit without having to necessarily force him to wait until 2.33 goes out, especially because we don't actually know if this PR will solve the problem he's facing.

But things aren't that straightforward with Apple, or maybe it is and I don't know. I believe @3esmit is locked out in an iPhone, where he cannot install a manual release build from this PR to upgrade and test the fix like any APK would do in Android. I thought TestFlight builds couldn't upgrade production builds from App Store and I'm not even sure if it's safe regarding data loss.

@siddarthkay do you know if there's a way for a user to safely "upgrade" from the production build from App Store using TestFlight?

@3esmit
Copy link
Member

3esmit commented Jan 23, 2025

I have the backup of the database, I can test it without fear. If anything goes wrong, I can restore (the whole iPhone, including Status app private data).

@pavloburykh
Copy link
Contributor

pavloburykh commented Jan 23, 2025

While waiting for reviews and approvals of both this PR and the corresponding status-im/status-go#6270, I’ve started some initial testing.

Performed additional testing including upgrade of keycard user, user with enabled biometrics.

Checked:

  1. Creating of new wallet accounts
  2. Importing new key pair
  3. Accounts created in v.1 are working as expected
  4. Transaction (send flow)
  5. Creating 1-1 chat, sending/receiving messages

No issues from my side. I still need to trigger full e2e run (currently pipeline is broken) and in case there are no unexpected failures there will be green light from QA side. Will keep you updated.

@qfrank as I have in fact finished testing the PR prior to additional reviews of status-go PR please let me know in case there will be any additional commits so I perform re-testing. Thank you.

@status-im-auto
Copy link
Member

96% of end-end tests have passed

Total executed tests: 56
Failed tests: 1
Expected to fail tests: 1
Passed tests: 54
IDs of failed tests: 702731 
IDs of expected to fail tests: 702844 

Failed tests (1)

Click to expand
  • Rerun failed tests

  • Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_pin_messages, id: 702731

    Device 2: Find `Button` by `xpath`: `//*[@content-desc='pins-count']//android.widget.TextView`
    Device 2: Element Button text is equal to 2

    critical/chats/test_1_1_public_chats.py:278: in test_1_1_chat_pin_messages
        self.errors.verify_no_errors()
    base_test_case.py:179: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     Device 1: Message 4 is not pinned in chat after unpinning previous one
    E    Device 2: Message 4 is not pinned in chat after unpinning previous one
    



    Device sessions

    Expected to fail tests (1)

    Click to expand

    Class TestCommunityMultipleDeviceMerged:

    1. test_community_links_with_previews_github_youtube_twitter_gif_send_enable, id: 702844

    Device 2: Find EmojisNumber by xpath: //*[starts-with(@text,'https://m.youtube.com/watch?v=Je7yErjEVt4')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']/../..//*[@content-desc='emoji-reaction-4']/android.widget.TextView[2]
    Device 2: Element EmojisNumber text is equal to 1

    critical/chats/test_public_chat_browsing.py:662: in test_community_links_with_previews_github_youtube_twitter_gif_send_enable
        self.errors.verify_no_errors()
    base_test_case.py:179: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     Device 1: No preview is loaded for url https://youtu.be/Je7yErjEVt4
    E    Device 1: No preview is loaded for url https://www.youtube.com/watch?v=XN-SVmuJH2g&list=PLbrz7IuP1hrgNtYe9g6YHwHO6F3OqNMao
    E    Device 1: No preview is loaded for url https://m.youtube.com/watch?v=Je7yErjEVt4 
    

    [[Youtube links preview is not loaded on LambdaTest emulators, needs investigation]]

    Device sessions

    Passed tests (54)

    Click to expand

    Class TestDeepLinksOneDevice:

    1. test_links_open_universal_links_from_chat, id: 704613
    Device sessions

    2. test_links_deep_links_profile, id: 702775
    Device sessions

    3. test_deep_links_communities, id: 739307
    Device sessions

    Class TestCommunityOneDeviceMerged:

    1. test_restore_multiaccount_with_waku_backup_remove_profile_switch, id: 703133
    Device sessions

    2. test_community_copy_and_paste_message_in_chat_input, id: 702742
    Device sessions

    3. test_community_navigate_to_channel_when_relaunch, id: 702846
    Device sessions

    4. test_community_undo_delete_message, id: 702869
    Device sessions

    5. test_community_mute_community_and_channel, id: 703382
    Device sessions

    6. test_community_discovery, id: 703503
    Device sessions

    Class TestGroupChatMultipleDeviceMergedNewUI:

    1. test_group_chat_reactions, id: 703202
    Device sessions

    2. test_group_chat_join_send_text_messages_push, id: 702807
    Device sessions

    3. test_group_chat_offline_pn, id: 702808
    Device sessions

    4. test_group_chat_pin_messages, id: 702732
    Device sessions

    5. test_group_chat_send_image_save_and_share, id: 703297
    Device sessions

    6. test_group_chat_mute_chat, id: 703495
    Device sessions

    Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_edit_message, id: 702855
    Device sessions

    2. test_1_1_chat_message_reaction, id: 702730
    Device sessions

    3. test_1_1_chat_non_latin_messages_stack_update_profile_photo, id: 702745
    Device sessions

    4. test_1_1_chat_text_message_delete_push_disappear, id: 702733
    Device sessions

    5. test_1_1_chat_push_emoji, id: 702813
    Device sessions

    6. test_1_1_chat_emoji_send_reply_and_open_link, id: 702782
    Device sessions

    7. test_1_1_chat_send_image_save_and_share, id: 703391
    Device sessions

    Class TestWalletOneDevice:

    1. test_wallet_add_remove_regular_account, id: 727231
    2. test_wallet_balance_mainnet, id: 740490

    Class TestCommunityMultipleDeviceMerged:

    1. test_community_emoji_send_copy_paste_reply, id: 702840
    Device sessions

    2. test_community_contact_block_unblock_offline, id: 702894
    Device sessions

    3. test_community_mark_all_messages_as_read, id: 703086
    Device sessions

    4. test_community_unread_messages_badge, id: 702841
    Device sessions

    5. test_community_message_delete, id: 702839
    Device sessions

    6. test_community_message_send_check_timestamps_sender_username, id: 702838
    Device sessions

    7. test_community_edit_delete_message_when_offline, id: 704615
    Device sessions

    8. test_community_one_image_send_reply, id: 702859
    Device sessions

    9. test_community_message_edit, id: 702843
    Device sessions

    10. test_community_several_images_send_reply, id: 703194
    Device sessions

    Class TestActivityCenterContactRequestMultipleDevicePR:

    1. test_activity_center_contact_request_accept_swipe_mark_all_as_read, id: 702851
    Device sessions

    2. test_activity_center_contact_request_decline, id: 702850
    Device sessions

    3. test_add_contact_field_validation, id: 702777
    Device sessions

    Class TestFallbackMultipleDevice:

    1. test_fallback_add_key_pair, id: 741054
    2. test_fallback_sync_with_error, id: 740220
    3. test_fallback_with_correct_seed_phrase, id: 740221
    4. test_fallback_validate_seed_phrase, id: 740222

    Class TestCommunityMultipleDeviceMergedTwo:

    1. test_community_leave, id: 702845
    Device sessions

    2. test_community_mentions_push_notification, id: 702786
    Device sessions

    3. test_community_markdown_support, id: 702809
    Device sessions

    4. test_community_hashtag_links_to_community_channels, id: 702948
    Device sessions

    5. test_community_join_when_node_owner_offline, id: 703629
    Device sessions

    Class TestOneToOneChatMultipleSharedDevicesNewUiTwo:

    1. test_1_1_chat_mute_chat, id: 703496
    Device sessions

    2. test_1_1_chat_is_shown_message_sent_delivered_from_offline, id: 702783
    Device sessions

    3. test_1_1_chat_delete_via_long_press_relogin, id: 702784
    Device sessions

    Class TestActivityMultipleDevicePRTwo:

    1. test_activity_center_admin_notification_accept_swipe, id: 702958
    Device sessions

    2. test_activity_center_mentions, id: 702957
    Device sessions

    Class TestActivityMultipleDevicePR:

    1. test_activity_center_reply_read_unread_delete_filter_swipe, id: 702947
    Device sessions

    Class TestWalletMultipleDevice:

    1. test_wallet_send_asset_from_drawer, id: 727230
    2. test_wallet_send_eth, id: 727229

    @pavloburykh
    Copy link
    Contributor

    No issues from my side. I still need to trigger full e2e run (currently pipeline is broken) and in case there are no unexpected failures there will be green light from QA side. Will keep you updated.

    UPDATE: failed e2e is not PR related. PR is good to merge in case there will be no additional changes.

    @qfrank
    Copy link
    Contributor Author

    qfrank commented Jan 24, 2025

    @qfrank Thank you for this PR, great work! While waiting for reviews and approvals of both this PR and the corresponding go PR, I’ve started some initial testing.

    I’ve identified that the latest Status v1 version resulting in the issue after upgrading to Status v2 is v1.18.0. The issue does not appear to reproduce when upgrading from versions >v1.18.0, though this doesn’t entirely rule out the possibility, as we’ve previously seen it occur after an upgrade from v1.20.6. However, those cases were random, and we don’t yet have clear steps to reproduce them. In contrast, upgrading from versions ≤v1.18.0 consistently leads to the issue after upgrading to Status v2 (100% reproducibility).

    Interestingly, if a user previously upgraded from v1.18.0 or earlier to v1.19.0 or later, they can successfully log in after upgrading to any Status v2 version (e.g., v2.30, v2.31, or v2.32). In such cases, the issue does not reproduce.

    Based on current results, it looks like this PR resolves the issue. Below is a table summarizing the upgrade flows I’ve tested so far:

    Upgrade Flow Result
    v1.18.0 → v2.30 Cannot log in (issue reproducible)
    v1.18.0 → v2.32 Cannot log in (issue reproducible)
    v1.18.0 → v2.32 → PR version Can log in (PR build fixes issue)
    v1.18.0 → v2.30 → PR version Can log in (PR build fixes issue)
    v1.19.0 → v2.30 Can log in (issue not reproducible)
    v1.18.0 → v1.19.0 → v2.30 Can log in (issue not reproducible)
    v1.18.0 → v1.20.6 → v2.30 Can log in (issue not reproducible)
    @qfrank Please ping me once these PRs receive approval, and I’ll proceed with further testing. Ideally, both the go and mobile PRs should have at least two approving reviews before moving forward. cc @ilmotta

    Thank you for your feedback! The interesting thing is reasonable. In short, the trap of node config migration was introduced from v1.19.0 @pavloburykh

    @churik
    Copy link
    Member

    churik commented Jan 24, 2025

    I’d say we should not support such cases in general. Testing upgrades from a previous version while skipping several intermediate versions is not something I would include in our testing scope, as the possible combinations are almost endless.

    Amazing work, @qfrank and @3esmit, on identifying this issue, but it’s an invalid user path, and I believe it shouldn’t be supported. Ideally, we should consider addressing this at the program level—if feasible—by detecting the installed version of Status and prohibiting installation if it’s older than the latest previous version.

    cc @ilmotta

    @ilmotta
    Copy link
    Contributor

    ilmotta commented Jan 24, 2025

    I’d say we should not support such cases in general. Testing upgrades from a previous version while skipping several intermediate versions is not something I would include in our testing scope, as the possible combinations are almost endless.

    Amazing work, @qfrank and @3esmit, on identifying this issue, but it’s an invalid user path, and I believe it shouldn’t be supported. Ideally, we should consider addressing this at the program level—if feasible—by detecting the installed version of Status and prohibiting installation if it’s older than the latest previous version.

    cc @ilmotta

    @churik Maybe I misunderstood your suggestion. This would force the user to upgrade one version at a time. The point of using migrations the way we do (which is a conventional solution, nothing special about Status) is that the user can skip releases as much as they want. Status app is meant to be upgradable from any version A to B (where B > A). In other words, if the user skips releases B, C, and D, and if each of these releases had a corresponding migration, when they migrate from A to E, all migrations for B, C, and D will be applied in order and the end result should be the same as if the user upgraded one version at a time. Therefore, if we prevent the user from skipping versions we wouldn't be helping them. I hope this makes sense.

    We may prefer to not test certain combinations of app upgrades and I very much agree with you on that, but this is simply a limitation of our capacity, not about the software.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    Status: MERGE
    Development

    Successfully merging this pull request may close these issues.

    6 participants