Make site work with the Cloudflare OpenNext adapter #7383

dario-piotrowicz · 2025-01-02T18:50:56Z

This PR applies changes to make it so that the site can be deployed to Cloudflare workers using the open-next Cloudflare adapter

The app does seem to work as intended for the most part:

Deployment URL: https://nodejs-website.web-experiments.workers.dev

There are still a few pressing things that need to be looked at, which are documented at the bottom of the apps/site/CLOUDFLARE.md file

Note

This is very experimental and full of hacks it's very much a work-in-progress right now, we probably need to iterate a bunch of this PR before potentially merging it

Warning

The PR is currently a single commit, I am planning to force push (with --force-with-lease) any new changes and keep this a single commit PR for now (so that it's easier to manage and rebase), if that's too annoying problematic for reviewers please let me know and I can just push standard commits if strongly preferred

Check List

I have read the Contributing Guidelines and made commit messages that follow the guideline.
I have run npm run format to ensure the code follows the style guide.
I have run npm run test to check if all tests are passing.
I have run npx turbo build to check if the website builds without errors.
I've covered new added functionality with unit tests if necessary.

vercel · 2025-01-02T18:50:59Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
nodejs-org	✅ Ready (Inspect)	Visit Preview	Jan 2, 2025 7:10pm

update the site application so that it can be build using the Cloudflare OpenNext adapter (`@opennextjs/cloudflare`) and thus deployed on Cloudflare Workers > [!Note] > This is very experimental and full of hacks > it's very much a work-in-progress right now ___ Co-authored-by: Brian Muenzenmeyer <brian.muenzenmeyer@gmail.com> Co-authored-by: Igor Minar <i@igor.dev>

dario-piotrowicz · 2025-01-02T18:53:49Z

apps/site/next-data/providers/releaseData.ts

-const releaseData = await generateReleaseData();
-
-const provideReleaseData = cache(() => releaseData);
+const provideReleaseData = cache(() => generateReleaseData());


I made this change because in workerd async operations can't be executed at the top level of a dynamically imported module, and that's what was happening here

(I can add a code comment about it if it'd be helpful)

dario-piotrowicz · 2025-01-02T18:57:47Z

apps/site/scripts/prebuild.generator.mjs

this file takes normal files such as '../../next.helpers.mjs' that read from the file system and generates modified copies of them in the .generated/ directory, these copies don't use the filesystem thus can be used in Cloudflare deployments

this whole setup came about naturally as I was iterating over the various hacks, we can probably come up with something much cleverer/cleaner/robust (I also haven't used the standard dev command much here, we might need to also evaluate how this current solution effects that)

dario-piotrowicz · 2025-01-02T18:59:36Z

apps/site/.cloudflare/opentelemetry.mjs

+// IMPORTANT: we already do that in the open-next Cloudflare adapter, it shouldn't be necessary here too
+//            (https://github.com/opennextjs/opennextjs-cloudflare/issues/219 seems to be the same issue)


This whole file shouldn't be needed, I'll look into opennextjs/opennextjs-cloudflare#219 and hopefully find a solution to fix this in the open-next Cloudflare adapter

dario-piotrowicz · 2025-01-02T19:04:01Z

apps/site/instrumentation.ts

I think that this was removed because we currently (probably) don't support instrumentation (opennextjs/opennextjs-cloudflare#171)

I don't recall deleting this, probably @bmuenzenmeyer or @IgorMinar did

dario-piotrowicz · 2025-01-02T19:06:45Z

apps/site/next.config.mjs

+  // Adds custom WebPack configuration to our Next.js setup
+  webpack: function (config) {
+    // CF hacking: temporarily disable minification to make debugging easier
+    config.optimization = {
+      minimize: false,
+    };
+
+    return config;
+  },


this block can really be removed at any time 👍

dario-piotrowicz · 2025-01-02T19:09:10Z

apps/site/next.dynamic.mjs

-    if (pathnameToFilename.has(normalizedPathname)) {
-      const filename = pathnameToFilename.get(normalizedPathname);
-      const filepath = join(process.cwd(), 'pages', locale, filename);
+    if (


TODO for myself: by debugging this I was that parts of the filepath were missing that's why the changes here were applied, I need to double check this and see why we need to do that now... maybe something somewhere else got changed incorrectly?

dario-piotrowicz · 2025-01-02T19:12:04Z

apps/site/wrangler.toml

+"jimp" = "./.cloudflare/empty.mjs"
+"probe-image-size" = "./.cloudflare/empty.mjs"


no idea why these are needed, I just added these since the respective imports were failing at runtime, I need to look deeper into these 😕

dario-piotrowicz · 2025-01-02T19:13:05Z

apps/site/wrangler.toml

+# There is a `require("react-dom/server.edge")` in the `pages.runtime.prod.js` file
+# but there isn't a `server.edge` entry in the `react-dom` package, so we just replace
+# it with the existing `server.browser` (we should investigate why we have this behavior
+# in the first place)
+"react-dom/server.edge" = "react-dom/server.browser"


this feels to me like an opennextjs-cloudflare issue...

Yes, we have a closed issue regarding this 😕 : opennextjs/opennextjs-cloudflare#119

According to this comment: opennextjs/opennextjs-cloudflare#119 (comment) this can be fixed by using react 19 instead of 18 🤔

Yes it was on us: opennextjs/opennextjs-cloudflare#235

ovflowd · 2025-01-03T15:49:14Z

apps/site/tsconfig.json

@@ -14,7 +14,7 @@
    "isolatedModules": true,
    "jsx": "preserve",
    "incremental": true,
-    "paths": { "@/*": ["./*"] },
+    "paths": { "@/*": ["./*"], "@generated/*": ["./.generated/*"] },


Is this correct?

yeah it is I just thought it'd be useful to add a small distinction for these generated files, I'm happy to use @/generated as you suggested below

Although this is part of the current solution which as you put it, is hacky as hell 😅, so it's probably worth rethinking anyways

ovflowd · 2025-01-03T15:50:50Z

apps/site/scripts/prebuild.generator.mjs

I definitely cannot sanction this change, or at least I find it extremely hard to justify. Although things such as blogData, and others (from the generators) folder are generated at build time, they are re-generated on runtime from time-to-time when the revalidation expires.

I believe all of this must be discarded before we can proceed with this PR, as soon as OpenNext or CF Workers respects the revalidate and the other static/caching rules from the Next.js routes and page exports.

(I've hidden my comment as I understood that I don't have a full picture here and I don't understand how the files are getting re-generated at runtime)

ovflowd · 2025-01-03T15:52:09Z

apps/site/package.json

+    "test": "turbo test:unit",
+    "cf": "npm run prebuild && npm run cf:build && npm run cf:preview",
+    "precf:build": "npm run prebuild",
+    "cf:build": "cross-env opennextjs-cloudflare && ln -s ../../pages ./.open-next/assets/",


I'd rather avoid adding platform/vendor-specific commands to the package.json. Feel free to document these commands in a file. There are many new commands, and I do not want to complicate the DevEx of the average contributor/user.

yeah this is quite work-in-progress, an ideal implementation, would override the serve, start, deploy, etc... scripts with Cloudflare ones, at this very experimental/hacks stage maybe it does make sense to keep the two distinct? 🤔 I don't have a huge preference on this actually 🤔

ovflowd · 2025-01-03T15:53:20Z

apps/site/open-next.config.ts

+  },
+
+  dangerous: {
+    enableCacheInterception: false,


Can you explain what these settings are? I would appreciate inline documentations for what each one of these Open Next settings are/mean or at least some type definitions of the shape of config.

These are open-next configs that are actually required to be set in this way by the Cloudflare adapter, which even validates and enforces this precise setup: https://github.com/opennextjs/opennextjs-cloudflare/blob/4b6a50b63912ba2a6d0a7cd168e13da1b7c50536/packages/cloudflare/src/cli/build/index.ts#L140-L153

I don't think that users need to really care much about these and I think that most if not all of these could/should be abstracted away from the user (we actually have an issue for that: opennextjs/opennextjs-cloudflare#114)

I can add comments saying what these configs do (after reading some docs for the ones that I don't know 😅), but maybe the easiest thing here would be to proceed with opennextjs/opennextjs-cloudflare#114 and practically just remove most if not all of these from this file.

ovflowd · 2025-01-03T15:53:47Z

apps/site/next.helpers.mjs

@@ -53,8 +40,27 @@ export const getMarkdownFiles = async (root, cwd, ignore = []) => {
  const cacheKey = `${root}${cwd}${ignore.join('')}`;

  if (!globCacheByPath.has(cacheKey)) {
-    globCacheByPath.set(cacheKey, glob('**/*.{md,mdx}', { root, cwd, ignore }));
+    const keyFiles = files


This is hacky as hell and complex as hell. Please let's not do this.

yeah definitely... but as I mentioned in my other comment I think that currently we can't really do much better than this (besides cleaning it up and move it in a less hacky/poorely designed way, etc...), since we cannot really run glob() in a deployed Worker...

(basically I would cathegorize this as the same issue as #7383 (comment))

ovflowd · 2025-01-03T15:59:22Z

apps/site/next.helpers.mjs

  }

  return globCacheByPath.get(cacheKey);
 };
+
+export const files = [
+  /* generated at build time */


Nor this. If you want a generated list of files, create a JSON or TXT file, but please do not mix source files with pseudo-generated files or make copies of these files within a generated folder.

That's hacky and poor design. Want to generate a list of files to be read during runtime or data to be read during runtime? Make a JSON file and update it. It can also be imported.

However, this extremely defeats the concept of providers that we created. Providers during build time read data from FS and during runtime from the API endpoint. The API endpoint itself then regenerates from the FS when it needs to invalidate the data

This static build-time generation not only defeats the purpose of said API endpoints and providers but also prevents the data from being updated without another build and deployment to be done.

Regarding blog data, that will only change on a new build because new blog posts are directly related to changes in the blog post files (file update, new file, file deleted), so the endpoint does not need to be regenerated during runtime, et all.

I'd recommend that you try to find alternatives here.

Regarding creating different files and avoid this hacky/poorely designed solution I totally agree with you and as I also mentioned in my comment this was just practically the first thing that came from hacking and trying to get the thing to deploy. This can 1000% be changed and made nice and robust.

But in my opinion this is pretty minor, as the real concern is the rest of your comment, whether something in this vein this can in any way be acceptable or not, which, sounds like it cannot (so there's really no point in trying to improve the hackyness if the core concept is bad/unacceptable).

Again I would categorize this as the same issue as #7383, i.e. pre-generating content is not acceptable

ovflowd · 2025-01-03T16:00:30Z

apps/site/next-data/providers/downloadSnippets.ts

@@ -1,6 +1,6 @@
 import { cache } from 'react';

-import generateDownloadSnippets from '@/next-data/generators/downloadSnippets.mjs';
+import generateDownloadSnippets from '@generated/downloadSnippets.mjs';


This should be:

Suggested change

import generateDownloadSnippets from '@generated/downloadSnippets.mjs';

import generateDownloadSnippets from '@/generated/downloadSnippets.mjs';

vercel bot deployed to Preview January 2, 2025 18:52 View deployment

dario-piotrowicz force-pushed the dario/opennextjs-cloudflare branch from b5e1217 to 0463451 Compare January 2, 2025 19:09

vercel bot deployed to Preview January 2, 2025 19:10 View deployment

dario-piotrowicz commented Jan 2, 2025

View reviewed changes

ovflowd reviewed Jan 3, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make site work with the Cloudflare OpenNext adapter #7383

Make site work with the Cloudflare OpenNext adapter #7383

dario-piotrowicz commented Jan 2, 2025 •

edited

Loading

vercel bot commented Jan 2, 2025 •

edited

Loading

dario-piotrowicz Jan 2, 2025

dario-piotrowicz Jan 2, 2025

dario-piotrowicz Jan 2, 2025

dario-piotrowicz Jan 2, 2025

dario-piotrowicz Jan 2, 2025

dario-piotrowicz Jan 2, 2025

dario-piotrowicz Jan 2, 2025

dario-piotrowicz Jan 2, 2025

dario-piotrowicz Jan 4, 2025

dario-piotrowicz Jan 8, 2025

ovflowd Jan 3, 2025

dario-piotrowicz Jan 3, 2025

ovflowd Jan 3, 2025

This comment was marked as off-topic.

dario-piotrowicz Jan 8, 2025

ovflowd Jan 3, 2025

dario-piotrowicz Jan 3, 2025

ovflowd Jan 3, 2025

dario-piotrowicz Jan 3, 2025

ovflowd Jan 3, 2025

dario-piotrowicz Jan 3, 2025 •

edited

Loading

ovflowd Jan 3, 2025

dario-piotrowicz Jan 3, 2025

ovflowd Jan 3, 2025 •

edited

Loading

		// IMPORTANT: we already do that in the open-next Cloudflare adapter, it shouldn't be necessary here too
		// (https://github.com/opennextjs/opennextjs-cloudflare/issues/219 seems to be the same issue)

		"jimp" = "./.cloudflare/empty.mjs"
		"probe-image-size" = "./.cloudflare/empty.mjs"

	import generateDownloadSnippets from '@generated/downloadSnippets.mjs';
	import generateDownloadSnippets from '@/generated/downloadSnippets.mjs';

Make site work with the Cloudflare OpenNext adapter #7383

Are you sure you want to change the base?

Make site work with the Cloudflare OpenNext adapter #7383

Conversation

dario-piotrowicz commented Jan 2, 2025 • edited Loading

Check List

vercel bot commented Jan 2, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as off-topic.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dario-piotrowicz Jan 3, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ovflowd Jan 3, 2025 • edited Loading

Choose a reason for hiding this comment

dario-piotrowicz commented Jan 2, 2025 •

edited

Loading

vercel bot commented Jan 2, 2025 •

edited

Loading

dario-piotrowicz Jan 3, 2025 •

edited

Loading

ovflowd Jan 3, 2025 •

edited

Loading