Modules are how you organize code into self-contained chunks that you can reuse in different codebases and import as necessary. JavaScript didn’t have a standard module system until ECMAScript 2015 (with support for ES modules arriving in major browsers by 2018). Up until then, developers using JavaScript for larger and more complex development turned to community module systems like AMD (Asynchronous Module Definition) plus RequireJS, Webpack, or CommonJS (the module system in Node.js).
The ES module system was developed to be an approach that both AMD and CommonJS users would be comfortable with. Node.js added experimental support in v8.5.0 in 2017, with stable support arriving in mid-2021 in Node v15.3.0. Importantly, that gave Node developers access to the full range of published modules.
“ES modules was wildly successful as an authoring format,” TC39 co-chair Rob Palmer told The New Stack. “It’s the dominant form [in] which you see source code being authored, if you’re on GitHub. It will all look like ES modules.”
Most of the World Runs on CommonJS
Despite the popularity of ES modules for authoring packages, when those modules get executed, they may actually get compiled (either by Node or TypeScript) and run as CommonJS.
“Most of the world is still running on CommonJS, even if it looks like it’s being authored in ES modules,” Palmer explained.
By default, Node.js treats the JavaScript code you import as CommonJS modules, for backward compatibility. That means developers who think they’re writing ES modules actually aren’t, Palmer warned, meaning they don’t get access to ES modules features like the new top-level await coming in ECMAScript 2022. “That’s an ergonomic feature that most developers really love,” he added.
ES modules enable some other important Node.js features, like package exports — where you can encapsulate a package so that developers only get access to specific entry points, rather than being able to call anything in the package. This means that someone could take a dependency on the way a feature is implemented, which might change in a later version.
“Previously it was a free-for-all in Node packages,” Palmer said. “You could just load any arbitrary file inside the package you like: just write the directory and the file name and you can load it. Whereas now, with package exports, you can say no, only these particular entry points are the ones that the public are allowed and everything inside is private to that package. This is a key piece that’s needed for the ecosystem and it’s one of the really wonderful treats we’ll get once TypeScript has support for Node’s flavor of ES modules.”
A Challenging Change
Supporting developers through the transition to ES modules has been “an ongoing pain point,” explained Daniel Rosenwasser, senior program manager for TypeScript at Microsoft. It is the complexity that developers will have to deal with, whether or not they use TypeScript.
“JavaScript developers have built up a lot of expectations of how things should work based on their experiences with CommonJS, bundlers, and browser ES module support,” he said, “and things don’t work that way in Node.”
He described CommonJS as “extremely convenient to use, but ultimately different from what browsers needed”.
With its focus on developer productivity, adding ECMAScript module support for Node.js in TypeScript seems like an obvious step. It was originally planned for TypeScript 4.5 as module: node12 and moduleResolution: nodenext modes, that would either match Node.js behavior or use TypeScript features to deliver the same functionality.
“To provide a bridge for developers, we’re experimenting with their support for ES modules and seeing where we can reduce some of the issues developers may run into,” Rosenwasser said.
But the new mode wasn’t ready for TypeScript 4.5, partly because of complexities in the ecosystem of tools like Deno, ts-node, Webpack and Vite; and partly because it’s already too easy for package authors to misconfigure packages in ways that make it hard for developers to work with them. So the TypeScript team doesn’t want to make that worse. There’s also some debate about whether the default for using the new mode should be node12 or nodenext, since Node 12 doesn’t have top-level await.
“Now it’s time for TypeScript to support this, because the rest of the ecosystem has adopted it and people want access to the features that Node has provided here,” Palmer told us. “But it’s really challenging to do that. This is nothing to do with TypeScript; the challenges are inherent in the compatibility between the old format and the new.
The Difficulties of Making the Two Systems Interoperate
The two module systems don’t just have different syntax, they also work rather differently.
“The root of that conflict is that CommonJS is synchronous,” Palmer continued. “When it runs, it just runs straight line code: there’s no gaps, there’s no time delays. ES modules supports asynchronous work, where you can split your work up over time.”
That’s important, because when you write code that will run in a browser, you have to be able to make sure that the rest of the page isn’t held up waiting for a long-running script.
Because they’re synchronous, CommonJS modules are parsed dynamically during execution — so you can call modules from inside an IF statement, because the dependency graph will be built while the program is running. Because ES modules are asynchronous, the interpreter builds a dependency graph before they run, which means that it can optimize the code (so code that won’t be called isn’t included). Note: tree shaking to remove unnecessary code is possible with CommonJS, but bundlers don’t usually do it by default.
There’s an additional complication in the way module files are named. The .mjs and .cjs extensions make it explicit whether you’re using ECMAScript or CommonJS modules (the equivalent for TypeScript would be .mts and .cts). However, they’re optional because some developers feel strongly that JavaScript code should always use .js as the extension, even for modules.
Where TypeScript Can Help
With the new Node.js ES module support, TypeScript will understand these extensions. But if developers don’t use them, TypeScript will need to figure out if a file of code is a module or not, as well as what import syntax to use. Picking the wrong one could end up fetching a different file from the package, making it tricky for the TypeScript team to choose the right import default. If it’s complex for the people designing TypeScript to understand what the right behaviour will be, explaining to developers who aren’t experts in module systems what they should be doing will be even harder.
Those differences add to the difficulty of making the two systems interoperate. So far, the new module support is only available as an experimental flag in the nightly builds of TypeScript, so developers have to explicitly opt in to it.
“People would love TypeScript to just solve this,” Palmer noted. “In some ways, it’s an unfair expectation that TypeScript will provide a magic solution, because it’s not possible for them to play that role entirely.”
But despite all of this complexity, he praises the way TypeScript is tackling support for Node.js-style ECMAScript modules. “What I really love is that they started to implement this and then, because they’re famous for developer experience, they found [that] when people try to use this […] the experience is not what they hope it to be.”
“It’s maturing and feedback is going in and they’re responding,” Palmer concluded. “But this is a real example where they didn’t just throw it out the door; they’re working very hard to make up for problems they did not create, problems that some other part of the ecosystem created.”
Conclusion
What TypeScript is hoping to offer is a large feature that will deliver a lot of the improvements developers have been asking for. The JavaScript community is so broad that there continues to be debate about what TypeScript should do here, so work in the broader ecosystem of JavaScript tools to support the new module options is ongoing. But many of the issues that led to postponing the feature from TypeScript 4.5 have been addressed, and the current plan is to include it in TypeScript 4.7 (which is due in May).
Currently, it’s just too easy for developers to get things wrong when configuring ES modules in Node.js. With the extra discussion that resulted from the delay, what ships should be a feature that makes working with ES modules easier to understand and debug.
The post Can TypeScript Help More Developers Adopt ECMAScript Modules? appeared first on The New Stack.
