Explore

modules (in javascript and beyond)

this document is in progress!

these are notes about:

mechanisms for splitting JavaScript programs up into separate modules that can be imported when needed

(source:
MDN⁠
)

there are several such mechanisms, which can be confusing if 1) you never sit down and clarify their syntaxes and the differences among all of them, or 2) you’re a baby programmer with a hazy understanding of what modules even are, generally and conceptually. both of these apply to me, which is why these notes exist :)

the what and why of modules

this

post⁠

from the blog Mozilla Hacks motivates modules this way: the vast majority of code is assigning, modifying, and keeping track of variables, or bindings. scope, provided by functions, helps a great deal with this: “functions can’t access variables that are defined in other functions.” when debugging, for example, you can often isolate issues to a specific function, rather than having huge sections of your codebase be your search space.

however, we often do want to share and re-use logic across functions. how can we do this? one idea is to move things out of functions and into the global scope, but several issues arise:

scripts must be loaded in the right order and this order must be delicately maintained (e.g. imagine the <script /> tags on an html page)

dependencies are implicit: anyone can make use of anything in the global environment, so when you’re trying to understand or change a function foo, you can’t easily tell who is depending on foo

a variable in the global scope can be modified by any code inside the global scope, which makes debugging an even larger task

this naive solution will not suffice, so we design modules instead. the book eloquent javascript, whose text is available online, has an excellent introduction to

modules⁠

. drawing from that source, some key properties that would be ideal to have in a modules system include:

a module has an interface: objects in the module can only be interacted with by outsiders in a restricted, controlled way specified by the module’s author(s) — that “way” is the interface. the flip side of this encapsulation is that the module’s internals are protected from accidental or intentional tampering by other code.

a module specifies its dependencies: this lets whatever is executing our module load any code required by the module. it’s also useful for composability — if you need some commonplace functionality, you can probably find and use a pre-existing, well-tested package (i.e. on

npm⁠

) instead of coding it yourself.

there are, of course, other definitions of modules, but the description in eloquent javascript strikes me as more full-fledged than some others i’ve seen and clicks with me — i can easily integrate it into existing mental frameworks i happen to have (namely those from my

software engineering class⁠

inventing modules in javascript

given this wish list of things we want to be true about modules, we can consider: how would we implement this functionality using vanilla javascript? it turns out that we can harness functions for local scoping, and objects come in handy for making module interfaces.

i really like the following code example from eloquent javascript — it seemed very foreign when i read it for the first time, because i don’t often encounter code that uses these built-in language features in this way. but upon closer inspection, it feels like a creative, reasonable starting point for building module support from scratch.

// from Eloquent JavaScript, chapter 10

const weekDay = function() {

const names = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"];

return {

name(number) { return names[number]; },

number(name) { return names.indexOf(name); }

};

}();

console.log(weekDay.name(weekDay.number("Sunday")));

// → Sunday

studying the code:

the weekDay variable is assigned to an anonymous function that is immediately invoked (see also:

IIFEs⁠

)

when invoked, the function returns an object with two methods: name and number

to reiterate, all that’s being used here are plain ol’ functions and objects.

this approach achieves the first of the characteristics we mentioned earlier. names is scoped to the function — local, rather than global. the methods in the returned object are made available to other code — we have an interface. however, to handle dependencies, each module needs to “export” its interface to the global scope and “import” anything it depends on from there too (trusting that everyone else has diligently followed the former convention), which isn’t ideal.

“evaluating data as code”

we can imagine an improved dependency situation, closer to what we’re used to seeing these days: our module identifies other modules it relies on; identification is in the form of strings (called “module identifiers”) which are relative or absolute file paths, web urls, npm package names, etc. provided that we have ways to load the contents of these referenced files — nothing more than strings of code — what we need next is a way a program to run these strings as part of itself/the current program.

eval() is a naive way to evaluate things, as its name suggests, but since it “execute[s] a string in the current scope,” it would introduce some undesired behavior. functions come in clutch again to provide local scope. in particular, we can use the Function constructor, which “wraps the code [of the function body] in a function value so that it gets its own scope and won’t do odd things with other scopes.”

(TODO: transition) with this context at our disposal, the rest of this document leans largely practical, with notes and examples about using two module systems: CommonJS and ES Modules.

the workarounds era (ft. commonjs)

as javascript graduated from the realm of tractable scripts to a status of ubiquity on the web, several efforts conspired to meet programmers’ demands for modules. i will only talk about commonjs, but there are other systems such as amd and umd.

CommonJS effort sets JavaScript on path for world domination —

Ars Technica⁠

, 2009

commonjs was a grassroots effort to produce a collection of standards for modules, binary data, interacting with the file system, and other things for which the javascript language and ecosystem lacked a good-enough standard. the module spec defined in commonjs is what nodejs originally used for modules, implemented in its module core module. below i will talk about commonjs modules in node.

basics of commonjs

each file is treated as a separate module

some ways to enable this module system or spot that commonjs modules are being used

the .cjs file extension

setting "type": "commonjs" at the top level of the nearest package.json

(neither of these are required, however, so it’s possible for a project to use commonjs modules without having any .cjs files or having "type": "module" in its package.json)

import modules with the require() function

// example

const http = require("http");

imports come from the node_modules directory or the file that a given relative path points to

destructuring is often useful (in both cjs and ecm)

how exporting works

the Module system creates an object and assigns it to module.exports

exports (not to be confused with module.exports) is read-only/non-reassignable and references the same object as the initial module.exports, serving as a shorthand

you could export things (i.e. “[add] functions and objects... to the root of a module”) by setting properties on that pre-made object, e.g. exports.myProperty = someValue;

often, however, people overwrite the object completely, e.g. module.exports = { myProperty: someValue, anotherProp: value2 };

nota bene: this breaks the exports shorthand, since exports will still point to the pre-made object

a mental model for require()

function require(/* ... */) {

// we will define and call an anonymous function

// this will get passed into the anonymous function

const module = { exports: {} };

((module, exports) => {

// begin module code

// for example, define a function

function someFunc() {}

exports = someFunc;

// at this point, `exports` is no longer a shortcut for module.exports,

// and this module will still export an empty default object

module.exports = someFunc;

// now, however, the module will export someFunc

// end module code

})(module, module.exports);

}

another example:

function require(/* ... */) {

const module = { exports: {} };

((module, exports) => {

function someFunc() {}

exports.someFunc = someFunc;

// now the module will export an object that looks like:

// { someFunc: /* the function someFunc */ }

})(module, module.exports);

}

module.exports, exports, and require are not actually globals. we have access to them because nodejs (also?) wraps the module code in a function like this:

(function(exports, require, module, __filename, __dirname) {

// module code lives here

});

this “keeps top-level variables (defined [in the module] with var, const, or let) scoped to the module” and “provide[s] some global-looking variables that are actually specific to the module.”

why does node keep track of module parents and children?

other mechanisms of interest

caching

require.cache

main module (entry script when nodejs process is launched, or undefined if the program is not a cjs module)

fun archaeological sites:

⁠

https://www.commonjs.org/⁠

⁠

https://wiki.commonjs.org/wiki/CommonJS⁠

⁠

https://arstechnica.com/information-technology/2009/12/commonjs-effort-sets-javascript-on-path-for-world-domination/⁠