how to dlopen the so file during wasm init and preload the so files?

[aisnote]

Q1: in mymodule.wasm, it will do dlopen("mylib.so"),

do I need to use :
emscripten_async_wget to download the mylib.so firstly?

Also I tried to debug into the mymodule.js, seems the __dlopen_js will fallback to download the so file, but seems there is no chance to put the url on the mylib.so.

if (ENVIRONMENT_IS_WORKER) {
                    readBinary = (url) => {
                        var xhr = new XMLHttpRequest();
                        xhr.open('GET', url, false);
                        xhr.responseType = 'arraybuffer';
                        xhr.send(null);
                        return new Uint8Array(xhr.response);
                    };
                }

the url now only the filename: mylib.so, how to make the url like: https://mysite.com/wasm/mylib.so.

Q2. how to preload the so files as this link: https://emscripten.org/docs/porting/files/packaging_files.html#preloading-files

[aisnote]

@sbc100

[sbc100]

dlopen will first look in the emscripten filesystem, and then attempt to download. If it attempts to download its will simply use the filename as the URL and there is no way to control that.

If you want to use emscripten_async_wget that should be fine. I think you can then call emscripten_run_preload_plugins or emscripten_run_preload_plugins_data on the result before calling dlopen

If you want to preload at build time (W1) that is done via the --preload-file or (better) --embed-file command line flags (See https://emscripten.org/docs/tools_reference/emcc.html)

[aisnote]

I changed as below:
return readBinary(locateFile(libFile));

locateFile is passed from JS:

const Lib = {
        locateFile: (file) => fullHttpsPath+ '/' + file,
        onRuntimeInitialized: () => {
            console.log('onRuntimenitialized');
        },
        mainScriptUrlOrBlob: fullHttpsPath + '/mywasm.js',
    };

But meet the issue as:
sdk/upstream/emscripten/system/lib/libc/dynlink.c,250,_emscripten_thread_sync_code)

I saw the XHR get the mylib.so resp successful, but seems

t Object.invokeEntryPoint (

caused some issue.

The background is: mywasm worker running on A domain, but need to load wasm from other domain. thanks for you help.

[aisnote]

@sbc100 not sure what inside mywasm code, the assert happened here:

stream/emscripten/system/lib/libc/dynlink.c,250,_emscripten_thread_sync_code

    if (!success) {
      fprintf(stderr, "dlerror: %s\n", dlerror());
    }
    assert(success);

#ifdef _REENTRANT
void _emscripten_thread_sync_code() {
  // This function is called from emscripten_yeild which itself is called
  // whenever we block on a futex.  We need to check to avoid infinite
  // recursion when taking the lock below.
  static thread_local bool syncing = false;
  if (syncing) {
    return;
  }
  syncing = true;
  ensure_init();
  if (thread_local_tail == tail) {
#ifdef DYLINK_DEBUG
    fprintf(stderr, "%p: emscripten_thread_sync_code: already in sync\n", pthread_self());
#endif
    goto done;
  }
  pthread_rwlock_rdlock(&lock);
  if (!thread_local_tail) {
    thread_local_tail = head;
  }
  while (thread_local_tail->next) {
    struct dso* p = thread_local_tail->next;
#ifdef DYLINK_DEBUG
    fprintf(stderr, "%p: emscripten_thread_sync_code: %s mem_addr=%p mem_size=%d table_addr=%p table_size=%d\n", pthread_self(), p->name, p->mem_addr, p->mem_size, p->table_addr, p->table_size);
#endif
    void* success = _dlopen_js(p);
    if (!success) {
      fprintf(stderr, "dlerror: %s\n", dlerror());
    }
    assert(success);
    thread_local_tail = p;
  }

[sbc100]

Hmm.. I guess the problem is that each thread would need to perform the preload in that case. I guess that is not going to work (at least not today).

[aisnote]

actually I do make it worked before. But now just need to cross origin to load the wasm into the worker and this issues happened.

[aisnote]

I found that:

thread = std::thread 

seems will callback to JS and try to load the SO file, I am not sure why?

[sbc100]

Because only JS can download new code and create new module. That part must happen on the JS side right now.

[aisnote]

but actually this mylib.so already downloaded before. Also I found at this time try to XHR to download this so file. the Module['locateFile'] is null.

var scriptDirectory = '';
        function locateFile(path) {
            if (Module['locateFile']) {
                return Module['locateFile'](path, scriptDirectory);
            }
            return scriptDirectory + path;
        }

[aisnote]

Recently, I found that the file be embed into wasm file with --embed-file and map to MEMFS like "/myfile.so", it can be found by FS in the mainthread. But it cannot be found by the worker thread.
Q: can be the worker thread and main thread share the FS? Thanks. @sbc100

[aisnote]

now in worker my code will automatically to download the SO file. I want dlopen to load the SO file from the FS, possible doable?

[sbc100]

Yes, dlopen should work fine in both the worker and the main thread. When calling dlopen from a worker the filesystem request will get proxied back to the main thread.

[aisnote]

The scenario on above code is:
The call stack is from a worker.
flags.fs.findObject(libFile) return fail, errno is 44, no file and directory.

In main thread, the libFile will be found. Nor sure what I missed?

[sbc100]

I believe all the code was removed as part of #19310. If you use version 3.1.39 or above that should no longer be there. What version are you using?

[aisnote]

I used 3.1.27. Will try 3.1.39 to see and update here. thanks.