Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving .zim files for importation is inefficient #4182

Open
wildasin opened this issue Jan 20, 2025 · 4 comments
Open

Moving .zim files for importation is inefficient #4182

wildasin opened this issue Jan 20, 2025 · 4 comments
Assignees
Milestone

Comments

@wildasin
Copy link

Describe the bug
When importing a .zim file from another directory on an SD card to the org.kiwix.kiwixmobile directory on the same SD card, the "move" operation appears to be O(n) in file size, when it should be instant regardless of file size. It appears to be doing a copy+remove rather than rename despite being on the same filesystem. This wastes hours when importing large files.

Expected behavior
Moving a file within the same filesystem should be instant regardless of size, as it is with the mv command.

Steps to reproduce the behavior:

  1. Select for importation a .zim file on the same SD card as the Kiwix data directory.
  2. A dialog box appears: "Kiwix requires the ZIM file to be in its own data directory." Click MOVE.
  3. The duration of the move operation appears to be proportional to the size of the file - hours for the largest files.

Environment

  • Version of Kiwix Android : 3.13.0
  • Device : Galaxy Tab A9+
  • OS version : 14

P.S.
Please advise if there is a workaround or best practice for efficient importation from an SD card. I tried putting files directly into the kiwixmobile directory, but they were not recognized.

@kelson42
Copy link
Collaborator

@MohitMaliFtechiz Issue seems reasonable. To me the question is can we have a robust solution to identify if the move happens on the same physical storage based on the common part of the paths)? If "yes", then we could implement a real move in place of copy+remove (and keep copy+remove) if source and target paths are on different storage.

@wildasin
Copy link
Author

I am not an Android engineer and there may be other complications, but I think Files.move() handles this in plain Java. Certainly the modern implementations of the mv(1) command handle it.

@wildasin
Copy link
Author

Another possibility - again forgive my ignorance of Android APIs - might be to attempt a proper move and then fall back to copy+remove if it fails. Less elegant but IMO better than trying to guess filesystem from path.

@MohitMaliFtechiz
Copy link
Collaborator

@MohitMaliFtechiz Issue seems reasonable. To me the question is can we have a robust solution to identify if the move happens on the same physical storage based on the common part of the paths)? If "yes", then we could implement a real move in place of copy+remove (and keep copy+remove) if source and target paths are on different storage.

@kelson42 We are using the DocumentsContract.moveDocument method which is recommended by the android to move document files, and it works for all connected devices e.g. internal, external and OTG. Also, it throws an error if it fails to move the document(ZIM file) for any reason for this scenario we have added the (copy+remove) fallback method to at least move the ZIM file.

I am not an Android engineer and there may be other complications, but I think Files.move() handles this in plain Java. Certainly the modern implementations of the mv(1) command handle it.

@wildasin For Android 11 and above devices, this method will not work since Android restricts direct access to files, it only gives access to Uri and provides its own APIs to work with these Uris.

Another possibility - again forgive my ignorance of Android APIs - might be to attempt a proper move and then fall back to copy+remove if it fails. Less elegant but IMO better than trying to guess filesystem from path.

@wildasin Yes, we have implemented this approach. See

and

It uses the Android provided APIs to move the document and if any error or restriction it faced it throws the error and our fallback method runs which copy+remove the file.

Please advise if there is a workaround or best practice for efficient importation from an SD card. I tried putting files directly into the kiwixmobile directory, but they were not recognized.

@wildasin Have you put the files in the Android/media/org.kiwix.kiwixmobile/ directory in the SD card?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants