You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I’m currently working on building an in-memory database that can handle datasets larger than the available RAM.
For persistence, I’m using Faster Log, which seems very promising. The concurrency control relies on the ReaderWriterLockSlim, which keeps the implementation simple, particularly for synchronous APIs (without using async/await).
The challenge appears when considering how to fetch data from the log once the data record has been flushed to disk (to maintain a small memory footprint). I noticed that the log only provides an asynchronous method for this:
However, this creates an issue for my implementation, which relies on ReaderWriterLockSlim. It breaks serializable execution because I can’t use async/await within a transaction.
Having a synchronous version of log.ReadAsync would be incredibly helpful. That way, during a transaction, I could verify that a record is missing from RAM but still fetch it from disk using its address.
Maybe I can use log.ReadAsync(iter.CurrentAddress).GetAwaiter().GetResult(), but I am not sure about deadlocks that may happen with such an approach.
I would appreciate any suggestions.
The text was updated successfully, but these errors were encountered:
dkantyadev
changed the title
Can I we have synchronous API for log.ReadAsync
Can we have synchronous API for log.ReadAsync
Nov 19, 2024
Thank you so much for this amazing project!
I’m currently working on building an in-memory database that can handle datasets larger than the available RAM.
For persistence, I’m using Faster Log, which seems very promising. The concurrency control relies on the ReaderWriterLockSlim, which keeps the implementation simple, particularly for synchronous APIs (without using async/await).
The challenge appears when considering how to fetch data from the log once the data record has been flushed to disk (to maintain a small memory footprint). I noticed that the log only provides an asynchronous method for this:
However, this creates an issue for my implementation, which relies on ReaderWriterLockSlim. It breaks serializable execution because I can’t use async/await within a transaction.
Having a synchronous version of
log.ReadAsync
would be incredibly helpful. That way, during a transaction, I could verify that a record is missing from RAM but still fetch it from disk using its address.Maybe I can use
log.ReadAsync(iter.CurrentAddress).GetAwaiter().GetResult()
, but I am not sure about deadlocks that may happen with such an approach.I would appreciate any suggestions.
The text was updated successfully, but these errors were encountered: