Upstreaming the Node-API host into React Native core ☂️ #447

A tracking issue for a long-term ambition: propose the core of this project — creating a Node-API environment and loading an addon into it — for inclusion in React Native itself, and keep the build tooling here.

Nothing here is committed to. The point is to have one place to argue about the seam, and to record what has to be true before a proposal is worth writing.

Why the cut is worth making

The repo currently spans two quite different kinds of work:

1. A runtime capability. A TurboModule (CxxNodeApiHostModule) that creates a napi_env per addon, loads a dynamic library into it, and supplies the host primitives Node-API needs but React Native has no equivalent of — async work, thread-safe function dispatch, fatal-error routing (HostContext in cpp/HermesNapiHost.cpp, and cpp/RuntimeNodeApi.cpp, which refactor(host): drop the shadowing napi_fatal_error #492 removes).
2. A build system. cmake-rn, ferric, gyp-to-cmake, the autolinking CLI, the Babel plugin, the prebuild naming and packaging conventions.

(1) is small, general, and belongs anywhere React Native runs. (2) is opinionated, iterates fast, and is a much bigger ask to hand to another project. Asking core to adopt both is likely to sink the whole proposal; asking for (1) alone is close to a one-sentence pitch.

The seam: loading an addon

The natural boundary is the loading primitive itself — a process.dlopen analogue (#498). Core would own "given a reference to an addon, load it and give me its exports"; everything that decides the reference stays here.

Two things to get right in a proposal:

It should behave like process.dlopen(module, filename), with nothing added. The filename is what gets passed to the system's dlopen: lib<name>.so on Android and @rpath/<name>.framework/<name> on Apple. Building that string stays on our side, so core knows nothing about our layouts, and caching stays in a higher-level function, the way require sits above process.dlopen in Node (#498).

The naming must not leak across the seam. Whatever core accepts has to be free of our library-name mangling (see #222, #91 and the manifest discussion in #368). If the strategy leaks upstream, we can never change it.

What already argues in favor

• Hermes implements Node-API itself. The engine at our pinned commit ships the whole surface, plus hermes_napi_create_env and hermes_napi_load_module, which we now use directly (Load addons through Hermes' hermes_napi_load_module #445). The upstream ask reduces to "expose what the engine already does", not "implement Node-API".
• Remove Hermes vendoring once React Native ships a Hermes with Node-API #412 — once React Native ships a Hermes with Node-API, we stop vendoring, and the remaining host code is small enough to read in one sitting.
• The frameworks are no longer preloaded (Apple link without cocoapods #351), so addons load on demand and the loading path is ordinary, not a startup-order special case.

The weak-node-api problem

This is the main obstacle, and it is not build tooling, so it does not fall cleanly on our side of the cut.

Addons are compiled independently of any app, so they have no specific Hermes to link against. Today they link weak-node-api: generated forwarding stubs for every Node-API symbol, backed by a function table the host injects at startup. If the host moves to core, addons still need something to link, and "core gains one TurboModule method" turns into "core gains a method and takes on a shim package" — a much harder sell.

The long-term dissolution is nodejs/node#60916 ("node-api: use v-table to reverse module dependencies", @vmoroz), which inverts the dependency: the runtime exposes its API through vtables reachable from napi_env, and an addon built with NODE_API_MODULE_USE_VTABLE links against nothing at all. That removes the reason weak-node-api exists, and with it the awkward part of the cut. It is upstream, open, and not something to plan a schedule around — but it is the shape of the end state, and worth tracking here.

Until then, options worth thinking through:

• Keep weak-node-api as a peer package here and propose only the host to core, accepting that addons still need our shim to build.
• Have addons link a generated stub of Hermes, so they call the app's Hermes directly and need no shim. This is the direction we're taking: Link addons against a generated Hermes stub instead of weak-node-api #494 adds it, Spike: move the napi host and addon loading into React Native core #495 tests it with the host in core, and How should addons get Node-API from the app's Hermes? #493 has the background.
• Wait for the vtable work and propose the whole thing once addons link nothing.

Prerequisites on our side

Roughly in order, and all independently useful:

• A raw, naming-free loading API in JS — the shape core would eventually own (Build addon paths in TypeScript and expose dlopen(module, filename) #498).
• Stop vendoring Hermes (Remove Hermes vendoring once React Native ships a Hermes with Node-API #412).
• Settle the addon-declared Node-API version question, which needs an upstream Hermes change either way (Support Node-API modules declaring their "Node-API version" #4).

RFC notes

Decisions so far, to draft the RFC from. Add to this list as they're made.

• API: import { dlopen } from "react-native/node-api", with the signature dlopen(module, filename). A subpath because dlopen is too ambiguous as a top-level react-native export. Same semantics as Node's process.dlopen, with no cache. flags is left out for now and can be added later (Build addon paths in TypeScript and expose dlopen(module, filename) #498).
• Stays in this project: path construction, requireNodeAddon and its cache, the naming scheme, and the build tooling.
• Addons: compile against node-api-headers, which defines the interface. A mismatch with the headers is a Hermes bug. They can link the real Hermes, or choose to link a generated stub (Link addons against a generated Hermes stub instead of weak-node-api #494). The main benefit of the stub is that a developer can build an addon without linking directly against Hermes from a React Native app. Either way, weak-node-api is out of the proposal.
• Examples: Node-API examples embedded in the rn-tester app, linking directly against the real Hermes, so they likely need no cmake-rn and no stub, which keeps the RFC free of this project's tooling.
• Asks of Hermes and React Native: ship Hermes with Node-API exported (Remove Hermes vendoring once React Native ships a Hermes with Node-API #412), and keep a stable library name for it. hermes_napi.h stays internal.
• Evidence: Spike: move the napi host and addon loading into React Native core #495 is meant to supply a working diff against core.

Open questions

• Does HostContext's threading model (a process-global worker pool plus CallInvoker dispatch) survive contact with core's own scheduling, or would core want to back those primitives differently?
• Is the per-addon napi_env (as in Node) the right model for core, given a React Native app can have several runtimes over its lifetime?
• What is the minimum viable proposal — the TurboModule alone, or does it only make sense together with autolinking, so that an app can actually get an addon into its bundle?

Related

#222, #91, #368, #412, #4, #3, #104, #493, #494, #495, #496, #497, #498, #263