-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix memory leak in queue #4192
Fix memory leak in queue #4192
Conversation
f61bad6
to
255050e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting! 👍 Btw, the older commits from that other PR are still showing up in this PR.
Does that mean that the Go GC isn't able to collect the memory allocations that are no longer pointed by any slice? 🤔 I guess that after: eh.queue.read = eh.queue.read[1:] The memory address where Instead, I guess that (based on the change on this PR), it is only able to collect a range of memory addresses that's not pointed at all by any slice. Is that reasoning right? 🤔 |
@joanlopez the GC will only cleanup the element that the 0th entry is pointing to if nothing is referencing it. So the underlying array remains in place i believe. In grafana/xk6-browser#1488, we found this the hard way when the reference was being kept even though there was no way to get back to the 0th element after performing eh.queue.read = eh.queue.read[1:] Therefore by making the change in this PR, we are sure that the underlying array is empty and the previous array is cleaned up by the GC. |
d8a6212
to
638eb16
Compare
The base branch was changed.
When we read off the queue.read slice, it essentially pops the first available element by doing this: case eh.ch <- eh.queue.read[0]: eh.queue.read[0] = Event{} eh.queue.read = eh.queue.read[1:] So the entry where the event used to exist has been overwritten with a fresh instance of Event which will avoid a memory leak. However the issue is that the underlying slice keeps growing. During a long running test the slice could grow to an unnecessarily length. To avoid this issue, when queue.read is empty and is swapped with queue.write, just before swapping them around the existing queue.read is overwritten with a new slice, freeing up the old potentially long slice. This should help avoid memory leaks.
What?
Overwrite the empty
queue.read
with a new slice.Why?
When we read off the
queue.read
slice, it essentially pops the first available element by doing this:(link to code)
So the entry where the event used to exist has been overwritten with a fresh instance of
Event
which will help avoid a memory leak (which was recently fixed). However the issue is that the underlying slice keeps growing. During a long running test the slice could grow to an unnecessarily length.To avoid this issue, when
queue.read
is empty and is swapped withqueue.write
, just before swapping them around the existingqueue.read
is overwritten with a new slice, freeing up the old potentially long slice. This should help avoid further memory leaks.Checklist
make lint
) and all checks pass.make tests
) and all tests pass.Related PR(s)/Issue(s)