Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can we have synchronous API for log.ReadAsync #932

Open
dkantyadev opened this issue Nov 18, 2024 · 1 comment
Open

Can we have synchronous API for log.ReadAsync #932

dkantyadev opened this issue Nov 18, 2024 · 1 comment

Comments

@dkantyadev
Copy link

dkantyadev commented Nov 18, 2024

Thank you so much for this amazing project!

I’m currently working on building an in-memory database that can handle datasets larger than the available RAM.
For persistence, I’m using Faster Log, which seems very promising. The concurrency control relies on the ReaderWriterLockSlim, which keeps the implementation simple, particularly for synchronous APIs (without using async/await).

The challenge appears when considering how to fetch data from the log once the data record has been flushed to disk (to maintain a small memory footprint). I noticed that the log only provides an asynchronous method for this:

(result, length) = await log.ReadAsync(iter.CurrentAddress);

However, this creates an issue for my implementation, which relies on ReaderWriterLockSlim. It breaks serializable execution because I can’t use async/await within a transaction.

Having a synchronous version of log.ReadAsync would be incredibly helpful. That way, during a transaction, I could verify that a record is missing from RAM but still fetch it from disk using its address.

Maybe I can use log.ReadAsync(iter.CurrentAddress).GetAwaiter().GetResult(), but I am not sure about deadlocks that may happen with such an approach.

I would appreciate any suggestions.

@dkantyadev dkantyadev changed the title Can I we have synchronous API for log.ReadAsync Can we have synchronous API for log.ReadAsync Nov 19, 2024
@AntyaDev
Copy link

@badrishc Any ideas?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants