Update to webpack 5 #26229

Johennes · 2023-09-23T20:11:59Z

Requires: matrix-org/matrix-react-sdk#11869

I have deliberately left most of the optional changes and deprecation fixes out of this pull request to keep the diff small. Those should be easy to follow up on afterwards.

Things tested

Open questions

The --output-filename and --output-chunk-filename flags that stopped working in Update webpack to 4.47.0 / webpack-cli to 4.10.0 / webpack-dev-server 4.15.1 #26216 have been restored in this pull request. However, I'm unsure what effect these options have. I would have expected to see a _dev_ directory but didn't.
Webpack 5 no longer polyfills core node modules. Missing module issues seem to only strike at runtime, sadly. I've fixed the ones I came across but there might be more and I'm not sure how to systematically find them or prevent future issues. There is https://github.com/Richienb/node-polyfill-webpack-plugin which would let us easily reproduce what Webpack 4 did. However, I assume there's some overhead to polyfilling modules we don't actually use.
Instead of putting it into index.ts, could setimmediate be injected via a ProvidePlugin, too?
What things other than the above need testing?

Granular tasks

Webpack 5 migration guide: https://webpack.js.org/migrate/5/

Fixes: https://github.com/vector-im/wat-internal/issues/67

This change is marked as an internal change (Task), so will not be included in the changelog.

… import paths

THOUSAND-SKY · 2023-10-03T13:27:28Z

I tried this but it gets stuck for me on [webpack-dev-middleware] wait until bundle finished: /, I have yarn link'ed matrix-{js,react}-sdk+@vector-im/compound-web

Just saying so this gets tested with yarn links too, in case it wasn't.

t3chguy · 2023-10-03T13:28:05Z

@THOUSAND-SKY CI is failing and it is still in draft. Its clearly not ready.

THOUSAND-SKY · 2023-10-03T13:29:11Z

I know, I wasn't implying that...

Johennes · 2023-10-04T08:47:59Z

@THOUSAND-SKY I have tested this with matrix-js-sdk and matrix-react-sdk being yarn linked. I haven't particularly tested doing the same with compound-web but it should work equally.

Note that until element-hq/compound#224 is consumed in matrix-react-sdk, you'll need to manually change this line in compound-web.js to avoid the use of useId:

With that change done it builds and runs fine for me.

THOUSAND-SKY · 2023-10-05T12:32:09Z

Alright, I must've messed up somewhere, I wasn't on the same commits as you probably, also I assume yarn link'ing compound-web from main already got me that patch, but didn't actually check.

I noticed that yarn link stuff wasn't on the checklist in OP, so I decided to comment because of that 👍 . I was testing this in hopes that webpack 5 would speed up recompile times.

Johennes · 2023-10-05T13:43:44Z

Yes, fair point and testing this is very much appreciated! I've added testing yarn link to the check list in the description. In terms of build performance, I haven't done any formal measurements but from a gut it does not feel faster to me. Possibly even slower. There are no further optimizations (e.g. replacing the deprecated things) on this branch though. It's really just the bare minimum to get things running with webpack 5.

Johennes · 2023-10-16T06:24:50Z

This still fails after merging in develop because the matrix-react-sdk dependency hasn't yet been updated in yarn.lock and still requires compound-web 0.4.0.

https://github.com/vector-im/element-web/blob/develop/yarn.lock#L8964

Johennes · 2023-10-24T11:56:51Z

compound-web has been updated on develop and merged into this branch now. However, the build still appears to choke on useId. 🙄

In node_modules, the file compound-web.js contains the useId pony-fill:

import * as React from "react";

...

const react18UseId = React.useId;
const getUniqueId = (() => {
  let index = 1;
  return () => `:r${index++}:`;
})();
const useIdPonyFill = () => {
  return React.useMemo(getUniqueId, []);
};
const useId = typeof react18UseId === "function" ? react18UseId : useIdPonyFill;

but the build breaks with

ERROR in ./node_modules/@vector-im/compound-web/dist/compound-web.js 185:21-32
export 'useId' (imported as 'React') was not found in 'react' (possible exports: Children, Component, Fragment, Profiler, PureComponent, StrictMode, Suspense, __SECRET_INTERNALS_DO_NOT_USE_OR_YOU_WILL_BE_FIRED, cloneElement, createContext, createElement, createFactory, createRef, forwardRef, isValidElement, lazy, memo, useCallback, useContext, useDebugValue, useEffect, useImperativeHandle, useLayoutEffect, useMemo, useReducer, useRef, useState, version)
 @ ./node_modules/matrix-react-sdk/src/Avatar.ts
 @ ./src/vector/platform/ElectronPlatform.tsx 40:0-81 97:19-35 98:17-33
 @ ./src/vector/init.tsx 33:0-59 44:24-40
 @ ./src/vector/index.ts

Johennes · 2023-10-24T12:02:56Z

Upstream webpack bug: webpack/webpack#14814

It looks like our options are either making the import unanalyzable (React['useId'.toString()]) or set strictExportPresence: false.

Johennes · 2023-10-24T12:16:37Z

And of course strictExportPresence doesn't do anything at all. 😫

Attempting to make the usage unanalyzable in element-hq/compound-web#116.

…-plugin

Johennes · 2023-11-08T08:24:07Z

This builds and runs now but still fails the type check.


Run yarn run lint:types
yarn run v1.22.19
$ yarn lint:types:src && yarn lint:types:module_system
$ tsc --noEmit --jsx react
Error: node_modules/@types/loader-utils/index.d.ts(3,10): error TS2305: Module '"webpack"' has no exported member 'loader'.
Error: node_modules/@types/webpack/index.d.ts(44,5): error TS2305: Module '"tapable"' has no exported member 'Tapable'.
Error: node_modules/@types/webpack/index.d.ts(1152,23): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1153,22): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1154,28): error TS2707: Generic type 'AsyncSeriesWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1155,27): error TS2707: Generic type 'AsyncSeriesWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1156,27): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1157,21): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1158,27): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1160,30): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1161,24): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1170,33): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1171,27): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1172,37): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1173,44): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1174,47): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1175,28): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1176,30): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1177,24): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1178,25): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1179,34): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1180,25): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1181,31): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1182,36): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1183,35): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1184,34): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1185,40): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1186,33): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1187,36): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1188,38): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1189,36): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1190,28): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1191,25): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1192,27): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1193,25): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1194,29): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1195,23): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1196,32): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1197,22): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1198,29): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1199,38): error TS2314: Generic type 'HookMap<H>' requires 1 type argument(s).
Error: node_modules/@types/webpack/index.d.ts(1200,48): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1201,44): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1202,26): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1211,28): error TS2707: Generic type 'AsyncSeriesWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1212,27): error TS2707: Generic type 'AsyncSeriesWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1213,33): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1214,27): error TS2707: Generic type 'AsyncSeriesWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1234,21): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1235,19): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1237,40): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1238,35): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1239,43): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1242,23): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1244,35): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1245,30): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1246,38): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1261,27): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1280,25): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1281,24): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1287,33): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1288,40): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1289,32): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1294,31): error TS2707: Generic type 'AsyncSeriesHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1300,33): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1301,24): error TS2707: Generic type 'AsyncSeriesHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1309,33): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1310,28): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1314,43): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1315,38): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1316,46): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1329,31): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1330,26): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1331,36): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1332,32): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1333,28): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1334,31): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1353,22): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1354,21): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1355,21): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1356,22): error TS2707: Generic type 'SyncWaterfallHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1357,19): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1456,25): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1458,29): error TS2707: Generic type 'AsyncSeriesHook<T, AdditionalOptions>' requires between 1 and 2 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1474,25): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1475,26): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1476,31): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1479,26): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1499,25): error TS2707: Generic type 'SyncHook<T, R, AdditionalOptions>' requires between 1 and 3 type arguments.
Error: node_modules/@types/webpack/index.d.ts(1566,24): error TS2707: Generic type 'SyncBailHook<T, R, AdditionalOptions>' requires between 2 and 3 type arguments.
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Error: Process completed with exit code 2.

Johennes · 2023-11-09T15:57:53Z

warning " > worklet-loader@2.0.0" has incorrect peer dependency "webpack@^3.0.0 || ^4.0.0-alpha.0 || ^4.0.0".

Looks sane other than this and the full screen errors being shown even if an error is caught by an ErrorBoundary

I've eliminated worklet-loader but this will only start working once matrix-org/matrix-react-sdk#11845 lands.

For the error overlays, I think we could set a flag on the errors as they pass through ErrorBoundary and then evaluate it via the webpack config.

devServer: {
    client: {
      overlay: {
        runtimeErrors: (error) => {
          if (error[...]) {
            return false;
          }
          return true;
        },
      },
    },

When I briefly tried this out, I hit a CSP issue on evaluating the function though. Given that this overlay may easily get frustrating rather than helpful, I opted for just disabling it for all runtime errors for now. I kept it on for build errors because noticing in the browser that you actually haven't successfully rebuilt after a change seemed helpful.

commit 069c1bc Author: Johannes Marbach <johannesm@element.io> Date: Sat Nov 11 16:08:30 2023 +0100 Replace worker-loader with built-in Webpack 5 support for web workers

Johennes · 2023-11-13T19:23:26Z

Latest status: With matrix-org/matrix-react-sdk#11860 workers and worklets work without worker-loader and worklet-loader. However, worklets require publicPath: "" which appears to break workers. So they don't yet work at the same time. I'll need to find a way to set the public path differently for both or figure out why automatic public path doesn't work for worklets.

Johennes · 2023-11-16T21:06:37Z

I'm, sadly, starting to reach the end of my "wisdom" here in terms of making the audio recording worklet function.

Webpack 5 built-in worklet support

The built-in Webpack 5 support for worklets is what's currently implemented in this PR. When attempting to do a voice recording, it fails with Error: Automatic publicPath is not supported in this browser. This is because Webpack injects this into the compiled worklet

/******/ 	/* webpack/runtime/publicPath */
/******/ 	(() => {
/******/ 		var scriptUrl;
/******/ 		if (__webpack_require__.g.importScripts) scriptUrl = __webpack_require__.g.location + "";
/******/ 		var document = __webpack_require__.g.document;
/******/ 		if (!scriptUrl && document) {
/******/ 			if (document.currentScript)
/******/ 				scriptUrl = document.currentScript.src;
/******/ 			if (!scriptUrl) {
/******/ 				var scripts = document.getElementsByTagName("script");
/******/ 				if(scripts.length) {
/******/ 					var i = scripts.length - 1;
/******/ 					while (i > -1 && !scriptUrl) scriptUrl = scripts[i--].src;
/******/ 				}
/******/ 			}
/******/ 		}
/******/ 		// When supporting browsers where an automatic publicPath is not supported you must specify an output.publicPath manually via configuration
/******/ 		// or pass an empty string ("") and set the __webpack_public_path__ variable from your code to use your own logic.
/******/ 		if (!scriptUrl) throw new Error("Automatic publicPath is not supported in this browser");
/******/ 		scriptUrl = scriptUrl.replace(/#.*$/, "").replace(/\?.*$/, "").replace(/\/[^\/]+$/, "/");
/******/ 		__webpack_require__.p = scriptUrl + "../../";
/******/ 	})();

due to our use of the default output.publicPath: "auto". I wasn't able to make this go away because setting output.publicPath to anything else broke the rest of the app.

Separate entry point for the worklet

I also tried adding a dedicated entry for the worklet with

--- a/webpack.config.js
+++ b/webpack.config.js
@@ -158,10 +158,15 @@ module.exports = (env, argv) => {
         entry: {
             bundle: "./src/vector/index.ts",
             mobileguide: "./src/vector/mobile_guide/index.ts",
             jitsi: "./src/vector/jitsi/index.ts",
             usercontent: "./node_modules/matrix-react-sdk/src/usercontent/index.ts",
+            worklet: {
+                import: "./node_modules/matrix-react-sdk/src/audio/RecorderWorklet.ts",
+                filename: "worklet.js",
+                publicPath: "."
+            },

and loading the worklet via context.audioWorklet.addModule("worklet.js").

Judging from the source, this is what https://github.com/reklawnos/worklet-loader does. However, it fails with ReferenceError: self is not defined which is due to Webpack injecting this piece:

/******/ 		self["webpackHotUpdateelement_web"] = (chunkId, moreModules, runtime) => {
/******/ 			for(var moduleId in moreModules) {
...

When I disable hot reload entirely, this error goes away but it then hits self somewhere later in the generated file.

Separate dependent project for the worklet

I also tried setting up a dedicated project for the worklet.

diff --cc webpack.config.js
index c65ebef99,fc525fddc..000000000
--- a/webpack.config.js
+++ b/webpack.config.js
@@@ -150,7 -149,107 +150,107 @@@ module.exports = (env, argv) => 
          `;
      }
  
-     return {
+     return [
+         {
+             name: "worklet",
+             mode: "production",
+             target: "webworker",
+             entry: "./node_modules/matrix-react-sdk/src/audio/RecorderWorklet.ts",
+             output: {
+               path: path.join(__dirname, "webapp"),
 -              filename: "the-recorder.worklet.js",
 -              publicPath: ""
++              filename: "worklet.js",
++              publicPath: "."
+             },
+             resolve: {
+                 ... duplicated from the app project ...
+             },
+             module: {
+                 rules: [
+                     {
+                         test: /\.(ts|js)x?$/,
+                         include: (f) => {
+                             ... duplicated from the app project ...
+                         },
+                         loader: "babel-loader",
+                         options: {
+                             cacheDirectory: true,
+                         },
+                     },
+                 ]
+           },
+         },
+     {
+         name: "webapp",
+         dependencies: ["worklet"],
+ 
          ...development,
  
          bail: true,

This, again, requires loading the worklet via context.audioWorklet.addModule("worklet.js").

Doing this, the worklet runs but only when I yarn build once before yarn starting. The build step here produces webapp/worklet.js which for some reason the dev server doesn't appear to serve on its own. I have not been able to fix that so far.

3rd-party audio worklet loader

Finally, I also found and tried https://github.com/leviance/audio-worklet-loader which works but is a fairly young and yet unused project.

Johennes · 2023-11-18T20:42:46Z

I ended up going the cheap route by creating a tiny custom loader based on https://github.com/reklawnos/worklet-loader specifically for the recorder worklet. This doesn't allow us to use the new Webpack 5 syntax but it does work and it does cause any peer-dependency warnings.

Update to webpack 5

af3659a

Johennes added the T-Task Tasks for the team like planning label Sep 23, 2023

Johennes added 3 commits September 29, 2023 11:00

Merge branch 'johannes/webpack-cli-4.10.0' into johannes/webpack-5

c844d80

Set context manually in NormalModuleReplacementPlugin to fix relative…

9b8d6e7

… import paths

Remove . since path is already relative

2984c3b

Johennes added 2 commits October 16, 2023 07:56

Merge branch 'johannes/webpack-cli-4.10.0' into johannes/webpack-5

0b706bb

Also polyfill buffer and process/browser

88666ee

Merge branch 'johannes/webpack-cli-4.10.0' into johannes/webpack-5

98c27c8

This was referenced Oct 24, 2023

Make usage of useId unanalyzable element-hq/compound-web#116

Merged

Update webpack to 4.47.0 / webpack-cli to 4.10.0 / webpack-dev-server 4.15.1 #26216

Merged

Base automatically changed from johannes/webpack-cli-4.10.0 to develop October 25, 2023 14:47

Johennes added 7 commits October 25, 2023 16:51

Merge branch 'develop' into johannes/webpack-5

3a73f6b

Appease the linter

d8f46ab

Merge branch 'develop' into johannes/webpack-5

17147ad

Merge branch 'develop' into johannes/webpack-5

4ffac57

Upgrade webpack to 5.89.0

e271f2c

Remove unused extract-text-webpack-plugin

fe46b05

Replace optimize-css-assets-webpack-plugin with css-minimizer-webpack…

25bb2ae

…-plugin

Eliminate use of loader-utils

0a36934

Johennes added 2 commits November 9, 2023 16:11

Only show overlay on build errors

42da986

Eliminate worklet-loader

37632d8

t3chguy approved these changes Nov 9, 2023

View reviewed changes

Johennes mentioned this pull request Nov 9, 2023

Load audioWorklet in a Webpack 5 compatible way matrix-org/matrix-react-sdk#11845

Closed

Johennes added 3 commits November 9, 2023 17:05

Appease the linter

3e03b2e

Add defaults and fix comment

0584c39

Disable automatic publicPath to unbreak worklets

91f46cc

Johennes mentioned this pull request Nov 13, 2023

Load web workers and worklets in a way that Webpack 5 can bundle them matrix-org/matrix-react-sdk#11860

Closed

3 tasks

Johennes added 2 commits November 13, 2023 20:02

Merge branch 'develop' into johannes/webpack-5

bbd449f

Squashed commit of the following:

432c1d6

commit 069c1bc Author: Johannes Marbach <johannesm@element.io> Date: Sat Nov 11 16:08:30 2023 +0100 Replace worker-loader with built-in Webpack 5 support for web workers

Johennes mentioned this pull request Nov 13, 2023

Replace worker-loader with built-in Webpack 5 support for web workers #26551

Closed

Update mocks for workers / worklets

5e9763e

Johennes mentioned this pull request Nov 14, 2023

Load web workers and worklets in a way that Webpack 5 can bundle them matrix-org/matrix-react-sdk#11869

Merged

3 tasks

Johennes marked this pull request as draft November 14, 2023 13:15

Johennes added 3 commits November 16, 2023 19:20

Merge branch 'develop' into johannes/webpack-5

a56a226

Update copy-webpack-plugin to latest

4e32b6f

Undo auto public path disablement as it doesn't work with workers

8dbefcc

Johennes added 2 commits November 18, 2023 21:37

Add custom loader for recorder worklet

e2fddda

Move worklet rule up to where it used to be

6093cd8

Johennes added 2 commits November 18, 2023 21:52

Merge branch 'develop' into johannes/webpack-5

f46cdd0

Set es2022 globally but force CommonJS when using ts-node for scripts

37c1c4e

Johennes marked this pull request as ready for review November 19, 2023 20:15

Johennes requested a review from t3chguy November 19, 2023 20:15

t3chguy approved these changes Nov 20, 2023

View reviewed changes

Johennes merged commit 619f36b into develop Nov 20, 2023
20 checks passed

Johennes deleted the johannes/webpack-5 branch November 20, 2023 12:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to webpack 5 #26229

Update to webpack 5 #26229

Johennes commented Sep 23, 2023 •

edited by github-actions bot

Loading

THOUSAND-SKY commented Oct 3, 2023

t3chguy commented Oct 3, 2023

THOUSAND-SKY commented Oct 3, 2023

Johennes commented Oct 4, 2023

THOUSAND-SKY commented Oct 5, 2023

Johennes commented Oct 5, 2023

Johennes commented Oct 16, 2023

Johennes commented Oct 24, 2023

Johennes commented Oct 24, 2023

Johennes commented Oct 24, 2023 •

edited

Loading

Johennes commented Nov 8, 2023

Johennes commented Nov 9, 2023

Johennes commented Nov 13, 2023

Johennes commented Nov 16, 2023 •

edited

Loading

Johennes commented Nov 18, 2023

Update to webpack 5 #26229

Update to webpack 5 #26229

Conversation

Johennes commented Sep 23, 2023 • edited by github-actions bot Loading

Things tested

Open questions

Granular tasks

THOUSAND-SKY commented Oct 3, 2023

t3chguy commented Oct 3, 2023

THOUSAND-SKY commented Oct 3, 2023

Johennes commented Oct 4, 2023

THOUSAND-SKY commented Oct 5, 2023

Johennes commented Oct 5, 2023

Johennes commented Oct 16, 2023

Johennes commented Oct 24, 2023

Johennes commented Oct 24, 2023

Johennes commented Oct 24, 2023 • edited Loading

Johennes commented Nov 8, 2023

Johennes commented Nov 9, 2023

Johennes commented Nov 13, 2023

Johennes commented Nov 16, 2023 • edited Loading

Webpack 5 built-in worklet support

Separate entry point for the worklet

Separate dependent project for the worklet

3rd-party audio worklet loader

Johennes commented Nov 18, 2023

Johennes commented Sep 23, 2023 •

edited by github-actions bot

Loading

Johennes commented Oct 24, 2023 •

edited

Loading

Johennes commented Nov 16, 2023 •

edited

Loading