Add support for Native multithreaded execution #4201

durban · 2024-12-15T13:22:22Z

This is on top of @djspiewak's wip/multithreaded-wstp branch. I did nothing so far, except default to SleepSystem in 489fbb9. This makes it possible to run IOSpec on scala-native; it (sometimes) passes on my machine ;-). It's probably too early to have this as a PR, but I'm doing it anyway to avoid duplicating work, and maybe to have a discussion about EpollSystem (see below). (@djspiewak feel free to close this PR if you have other plans with your branch.)

durban · 2024-12-15T13:32:00Z

So, I'm running this on Linux, and without 489fbb9 it tries to use EpollSystem. But starting the tests (e.g., testsNative/testOnly cats.effect.IOSpec) only leads to a hanging process. It seems to me, that the WSTP threads are waiting in epoll_wait, and a (the?) GC thread seems to wait for mutator threads to reach a safepoint(?). (Details: the GC is in thread_yield, called by Synchronizer_acquire.) I'm speculating here, but maybe Thread.sleep() does "something", which EpollSystem doesn't, before epoll_waiting?

EDIT: what I wrote above is probably completely wrong... 🤷‍♂️

djspiewak · 2024-12-15T15:45:20Z

This is great! Thank you for pushing this forward to the next obstacle.

One of the things that occurred to me as I poked at my branch originally is SN's tooling for introspecting thread state is really really limited so far as I understand it. Maybe this is just my ignorance and LLVM has some magic we could turn on, but I strongly suspect we're going to need better introspection to run down some of these problems.

What I'm thinking is we're probably going to end up building that, or at least leaning in heavily to do so, and that's probably a large part of what we'll need to do to get this off the ground. We should chat with the SN folks.

armanbilge · 2024-12-15T21:15:14Z

The reason it's hanging is because we haven't implemented interruption yet for the Native I/O-polling systems. This wasn't necessary when it was single-threaded, but now it's critical :)

cats-effect/core/native/src/main/scala/cats/effect/unsafe/EpollSystem.scala

Line 67 in 489fbb9

def interrupt(targetThread: Thread, targetPoller: Poller): Unit = ()

Compare with:

cats-effect/core/jvm/src/main/scala/cats/effect/unsafe/SelectorSystem.scala

Lines 97 to 100 in 9ce05f2

    
           def interrupt(targetThread: Thread, targetPoller: Poller): Unit = { 
        
             targetPoller.selector.wakeup() 
        
             () 
        
           }

armanbilge · 2024-12-15T21:19:00Z

Oh, the other reason it may be hanging is indeed related to GC. On Scala Native, blocking native calls need to be annotated explicitly with the @blocking annotation, so that it does the necessary book-keeping so it's possible to GC while a thread is stuck in that blocking call.

https://github.com/scala-native/scala-native/blob/c7b54a18e3ff11d8b2792f16fbb6e97780314014/nativelib/src/main/scala/scala/scalanative/unsafe/package.scala#L103-L106

For now it's fine to just mark it @blocking, but b/c this comes at a performance cost, we should actually make two separate epoll_wait bindings. One will use @blocking for when the timeout is > 0, and the other will not, for when the timeout == 0.

djspiewak · 2024-12-16T15:20:53Z

The reason it's hanging is because we haven't implemented interruption yet for the Native I/O-polling systems. This wasn't necessary when it was single-threaded, but now it's critical :)

You know, I didn't even think about this. Makes loads of sense though. Pipes time!

…SN blocking

durban · 2024-12-20T14:40:40Z

@armanbilge Thanks, I've tried to do the 2 things you mentioned. In 62b8141 I turned on the EpollSystem again, and tried implementing interrupt, and added the scala-native blocking annotation. This way testsNative/testOnly cats.effect.IOSpec passes on my machine. It obviously needs more work (e.g., I think interrupt is not threadsafe), but at least it's a step in the right direction.

armanbilge · 2024-12-20T15:52:15Z

core/native/src/main/scala/cats/effect/unsafe/EpollSystem.scala

+      // TODO: this is not threadsafe, we're reading `interruptFd` without synchronization:
+      if (unistd.write(this.interruptFd, buf, 8.toCSize) == -1) {


It doesn't need to be, it will be synchronized by workerThreadPublisher at its callsites.

cats-effect/core/jvm/src/main/scala/cats/effect/unsafe/WorkStealingThreadPool.scala

Lines 324 to 326 in 8d92651

workerThreadPublisher.get()

val worker = workerThreads(index)

system.interrupt(worker, pollers(index))

djspiewak · 2024-12-26T22:35:39Z

Interesting. So I merged your branch with series/3.x and now I'm getting the following:

[error] Unknown DWARF abbrev code: 26
[error] 
[error] STACKTRACE
[error] 
[error] java.lang.RuntimeException: Unknown DWARF abbrev code: 26
[error] 
[error] 
[error] This looks like a specs2 exception...
[error] Please report it with the preceding stacktrace at http://github.com/etorreborre/specs2/issues
[error]  
[error] Error: Total 1, Failed 0, Errors 1, Passed 0
[error] Error during tests:
[error] 	cats.effect.IOSpec

Edit: Appears to be a macOS only thing. Compiles and runs on Linux. Lovely.

djspiewak · 2024-12-27T15:33:37Z

Okay got around the issue with Lorenzo's help. It's fixed in SN main, so I updated to a local snapshot (lol) on my branch and made progress. I'll dig into interruption for kqueue

djspiewak · 2024-12-27T22:17:03Z

Update:

We no longer need the snapshot. I've pushed the magic compiler settings incantation on my branch
Also pushed is an initial stab at using EVFILT_USER to handle kqueue interrupts. I implemented it by basically putting that event permanently into the front of the changes array and then triggering it on the kqfd on interrupt(). In principle, this makes sense, but it doesn't actually work. The threads wake up but things spin forever. I'm probably being dumb. Have fun.

Will get back to this later.

djspiewak · 2024-12-28T03:17:29Z

I pushed more. Kqueue is pretty close to working I think.

djspiewak

We definitely need to figure out how to start sharing code across platforms more sanely. :P

armanbilge · 2025-01-02T23:46:26Z

Latest from CI ... this is reminiscent of the issues we are having in Cats, where test suites just error out 😕

Sporadic test errors since upgrading to Native 0.5 cats#4647

[error] Error: Total 2141, Failed 0, Errors 32, Passed 2108, Skipped 1, Pending 1
[error] Error during tests:
[error] 	cats.effect.unsafe.IORuntimeConfigSpec
[error] 	cats.effect.std.internal.BinomialHeapSpec
[error] 	cats.effect.ExitCodeSpec
[error] 	cats.effect.IOPropSpec
[error] 	cats.effect.std.DroppingQueueSpec
[error] 	cats.effect.std.MapRefSpec
[error] 	cats.effect.std.BackpressureSpec
[error] 	cats.effect.std.UnboundedDequeueSpec
[error] 	cats.effect.OptionTIOSpec
[error] 	cats.effect.tracing.TracingSpec
[error] 	cats.effect.std.UnboundedPQueueSpec
[error] 	cats.effect.std.SystemPropertiesSpec
[error] 	cats.effect.KleisliIOSpec
[error] 	cats.effect.std.DeferredParallelism2Spec
[error] 	cats.effect.std.DeferredParallelism4Spec
[error] 	cats.effect.unsafe.IORuntimeBuilderSpec
[error] 	cats.effect.kernel.DeferredSpec
[error] 	cats.effect.IOSpec
[error] 	not.cats.effect.IOParImplicitSpec
[error] 	cats.effect.std.BoundedQueueSpec
[error] 	cats.effect.ContSpec
[error] 	cats.effect.std.internal.BankersQueueSpec
[error] 	cats.effect.IOMtlLocalSpec
[error] 	cats.effect.std.MutexSpec
[error] 	cats.effect.kernel.ParallelFSpec
[error] 	cats.effect.FileDescriptorPollerSpec
[error] 	cats.effect.std.EnvSpec
[error] 	cats.effect.ThunkSpec
[error] 	cats.effect.std.ConsoleSpec
[error] 	cats.effect.DefaultContSpec
[error] 	cats.effect.testkit.TestControlSpec

armanbilge · 2025-01-02T23:51:40Z

Locally is a bit more promising 🤔

[error] Error: Total 2860, Failed 0, Errors 2, Passed 2857, Skipped 1, Pending 2
[error] Error during tests:
[error]         cats.effect.ContSpec
[error]         cats.effect.DefaultContSpec

djspiewak · 2025-01-02T23:52:16Z

I saw that happen when any individual test segfaults: it seems to cause a whole cascade of ghost errors.

djspiewak · 2025-01-03T15:14:55Z

I no longer see segfaults locally on mac. Haven't tried Linux yet. stackalloc is really fucky on SN 0.5 afaict, even for really simple cases. I don't think anyone should be using the no-args version for anything ever.

Oh as an aside, the problems I had pushing to this remote were being caused by git-lfs. In case anyone else sees these types of issues, the solution is to use the --no-verify flag with git push.

armanbilge · 2025-01-03T17:59:21Z

I no longer see segfaults locally on mac

I still do 😕

sbt:cats-effect> testsNative/testOnly *.FileDescriptorPollerSpec
[info] Build skipped: No changes detected in build configuration and class path contents since last build.
[info] Starting process '/Users/arman/code/cats-effect/tests/native/target/scala-2.13/cats-effect-tests-test' on port '63437'.
[info] Starting process '/Users/arman/code/cats-effect/tests/native/target/scala-2.13/cats-effect-tests-test' on port '63439'.
[error] Test runner interrupted by fatal signal 11
[warn] Force close java.lang.RuntimeException: Process /Users/arman/code/cats-effect/tests/native/target/scala-2.13/cats-effect-tests-test finished with non-zero value 139 (0x8b)
[info] FileDescriptorPollerSpec
[info] FileDescriptorPoller should
[info]   + notify read-ready events
[error] Error: Total 2, Failed 0, Errors 1, Passed 1
[error] Error during tests:
[error] 	cats.effect.FileDescriptorPollerSpec
[error] (testsNative / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 5 s, completed Jan 3, 2025, 9:59:12 AM
sbt:cats-effect>

djspiewak · 2025-01-04T14:55:49Z

Nuts. I wonder if I just didn't run it enough.

djspiewak and others added 5 commits December 8, 2024 09:56

Got the ball rolling on SN 0.5

caaac9b

Fixed more weird 0.4 -> 0.5 things

cc600ac

Got WSTP-related stuffs compiling on native

d7306c7

Reimplemented pats of AtomicIntegerFieldUpdater; now it's all linking

426776b

Use SleepSystem by default on native

489fbb9

djspiewak mentioned this pull request Dec 18, 2024

Release for Scala Native 0.5.x #4076

Open

durban added 2 commits December 20, 2024 15:36

Reënable EpollSystem, implement interrupt for it, mark epoll_wait as …

62b8141

…SN blocking

More helpful TestTimeoutException

7f0a559

armanbilge reviewed Dec 20, 2024

View reviewed changes

Merge branch 'series/3.x' into wip/multithreaded-wstp

8ebf321

djspiewak added 2 commits December 27, 2024 11:46

Adjusted DWARF version to avoid issues with 0.5.6 on macOS

afba12f

Started noodling with kqueue interrupts

3a88218

Rewrote most of the kqueue stuff to be simpler

e01f25b

djspiewak added 7 commits December 28, 2024 10:19

Forgot to bump the base version

06b0a65

Enabled concurrent Ref on native

ca7c89d

Make kqueue compatible with parallel GC on SN

081bc56

Enabled higher iterations from ContSpec on native

b0caa11

Made Deferred parallelism specs common across JVM and native

6e75e13

Shifted JVM IO functionality to share with native

65fd3e2

Made Dispatcher functionality common across JVM and native

e818c50

armanbilge added 4 commits January 2, 2025 13:39

Use sizeof with stackalloc

77379c2

mulithreadify IOApp

48d92dd

Delete single-thread native runtimes

4dcb142

Regenerate workflow

d5568bd

djspiewak reviewed Jan 2, 2025

View reviewed changes

armanbilge added 8 commits January 2, 2025 20:42

organize imports

76e2df6

Share SleepSystem

7859974

Fix bincompat

833ce4a

Fix Scala 3 compile

cefddb2

EpollSystem fixups

767c465

use constant val defns in LocalQueueConstants

b0e4ccf

Restore native+macos+JDK21 jobs to matrix

4856714

Tweak filter

651be36

djspiewak added the 🍄 enhancement label Jan 3, 2025

djspiewak changed the title ~~WIP, doesn't even compile~~ Add support for Native multithreaded execution Jan 3, 2025

stackalloc considered harmful

53abc4d

Split epoll_wait into blocking and non-blocking variants

22271cc

djspiewak force-pushed the wip/multithreaded-wstp branch from 89c862c to 22271cc Compare January 3, 2025 15:37

Restore encodeKevent

81ace6e

armanbilge added 5 commits January 4, 2025 16:20

Restore kqueue event buffering

024fe66

Avoid stackalloc and sizeof

c723bbc

Formatting

09bc339

Fix epoll_wait invocation

704c914

Fix more stackallocs / sizeofs

2eee55d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Native multithreaded execution #4201

Add support for Native multithreaded execution #4201

durban commented Dec 15, 2024

durban commented Dec 15, 2024 •

edited

Loading

djspiewak commented Dec 15, 2024 •

edited

Loading

armanbilge commented Dec 15, 2024 •

edited

Loading

armanbilge commented Dec 15, 2024

djspiewak commented Dec 16, 2024

durban commented Dec 20, 2024

armanbilge Dec 20, 2024

djspiewak commented Dec 26, 2024 •

edited

Loading

djspiewak commented Dec 27, 2024

djspiewak commented Dec 27, 2024

djspiewak commented Dec 28, 2024

djspiewak left a comment

armanbilge commented Jan 2, 2025

armanbilge commented Jan 2, 2025

djspiewak commented Jan 2, 2025

djspiewak commented Jan 3, 2025

armanbilge commented Jan 3, 2025

djspiewak commented Jan 4, 2025

		// TODO: this is not threadsafe, we're reading `interruptFd` without synchronization:
		if (unistd.write(this.interruptFd, buf, 8.toCSize) == -1) {

	workerThreadPublisher.get()
	val worker = workerThreads(index)
	system.interrupt(worker, pollers(index))

Add support for Native multithreaded execution #4201

Are you sure you want to change the base?

Add support for Native multithreaded execution #4201

Conversation

durban commented Dec 15, 2024

durban commented Dec 15, 2024 • edited Loading

djspiewak commented Dec 15, 2024 • edited Loading

armanbilge commented Dec 15, 2024 • edited Loading

armanbilge commented Dec 15, 2024

djspiewak commented Dec 16, 2024

durban commented Dec 20, 2024

armanbilge Dec 20, 2024

Choose a reason for hiding this comment

djspiewak commented Dec 26, 2024 • edited Loading

djspiewak commented Dec 27, 2024

djspiewak commented Dec 27, 2024

djspiewak commented Dec 28, 2024

djspiewak left a comment

Choose a reason for hiding this comment

armanbilge commented Jan 2, 2025

armanbilge commented Jan 2, 2025

djspiewak commented Jan 2, 2025

djspiewak commented Jan 3, 2025

armanbilge commented Jan 3, 2025

djspiewak commented Jan 4, 2025

durban commented Dec 15, 2024 •

edited

Loading

djspiewak commented Dec 15, 2024 •

edited

Loading

armanbilge commented Dec 15, 2024 •

edited

Loading

djspiewak commented Dec 26, 2024 •

edited

Loading