8. An overview of Node: Modules and npm

Node.js has a good amount of functionality built in. Let's look at the Table of Contents for the API documentation and try to group it into manageable chunks (italic = not covered here):

Fundamentals Globals STDIO Timers Modules Events Buffers Streams C/C++ Addons	Network I/O HTTP HTTPS URL Query Strings Net UDP/Datagram DNS	File system I/O File System Path Process I/O and V8 VM Process VM Child Processes Cluster
Terminal/console REPL Readline TTY	Testing & debugging Assertion Testing Debugger Utilities	Misc Crypto TLS/SSL _String Decoder * ZLIB * OS_

I’ll go through the parts of the Node API that you’ll use the most when writing web applications. The rest of the API is best looked up from nodejs.org/api/.

Fundamentals The current chapter and Chapter 9.	Network I/O HTTP and HTTPS are covered in Chapter 10.	File system I/O The file system module is covered in Chapter 11.
Process I/O and V8 VM Covered in Chapter TODO.	Terminal/console REPL is discussed in Chapter TODO.	Testing and debugging Coverage TODO.

8.1 Node.js modules

Let's talk about the module system in Node.

Modules make it possible to include other Javascript files into your applications. In fact, a vast majority of Node’s core functionality is implemented using modules written in Javascript - which means you can read the source code for the core libraries on Github.

Modules are crucial to building applications in Node, as they allow you to include external libraries, such as database access libraries - and they help in organizing your code into separate parts with limited responsibilities. You should try to identify reusable parts in your own code and turn them into separate modules to reduce the amount of code per file and to make it easier to read and maintain your code.

Using modules is simple: you use the require() function, which takes one argument: the name of a core library or a file system path to the module you want to load. You’ve seen this before in the simple messaging application example, where I used require() to use several core modules.

To make a module yourself, you need to specify what objects you want to export. The exports object is available in the top level scope in Node for this purpose:

exports.funcname = function() {
  return ‘Hello World’;
};

Any properties assigned to the exports object will be accessible from the return value of the require() function:

var hello = require('./hello.js');
console.log(hello.funcname()); // Print “Hello World”

You can also use module.exports instead of exports:

function funcname() { return 'Hello World'; }
module.exports = { funcname: funcname };

This alternative syntax makes it possible to assign a single object to exports (such as a class). We’ve previously discussed how you can build classes using prototypal inheritance. By making your classes separate modules, you can easily include them in your application:

// in class.js:
var Class = function() { … }
Class.prototype.funcname = function() {...}
module.exports = Class;

Then you can include your file using require() and make a new instance of your class:

// in another file:
var Class = require('./class.js');
var object = new Class(); // create new instance

Sharing variables between modules

Note that there is no global context in Node. Each script has it's own context, so including multiple modules does not pollute the current scope. var foo = 'bar'; in the top level scope of another module will not define foo in other modules.

What this means is that the only way to share variables and values between node modules is to include the same module in multiple files. Since modules are cached, you can use a shared module to store common data, such as configuration options:

// in config.js
var config = {
  foo: 'bar'
};
module.exports = config;

In a different module:

// in server.js
var config = require('./config.js');
console.log(config.foo);

However, Node module has a number of variables which are available by default. These are documented in the API docs: globals and process.

Some of the more interesting ones are:

filename	The filename of the code being executed.
dirname	The name of the directory that the currently executing script resides in.
process	A object which is associated with the currently running process. In addition to variables, it has methods such as process.exit, process.cwd and process.uptime.
process.argv.	An array containing the command line arguments. The first element will be 'node', the second element will be the name of the JavaScript file. The next elements will be any additional command line arguments.
process.stdin, process.stout, process.stderr.	Streams which correspond to the standard input, standard output and standard error output for the current process.
process.env	An object containing the user environment of the current process.
require.main	When a file is run directly from Node, require.main is set to its module.

The code below will print the values for the current script:

console.log('__filename', __filename);
console.log('__dirname', __dirname);
console.log('process.argv', process.argv);
console.log('process.env', process.env);
if(module === require.main) {
  console.log('This is the main module being run.');
}

require.main can be used to detect whether the module being currently run is the main module. This is useful when you want to do something else when a module is run standalone. For example, I make my test files runnable via node filename.js by including something like this:

// if this module is the script being run, then run the tests:
if (module === require.main) {
  var nodeunit_runner = require('nodeunit-runner');
  nodeunit_runner.run(__filename);
}

process.stdin, process.stdout and process.stderr are briefly discussed in the next chapter, where we discuss readable and writable streams.

Organizing modules

There are three ways in which you can require() files:

• using a relative path: foo = require('./lib/bar.js');
• using an absolute path: foo = require('/home/foo/lib/bar.js')
• using a search: foo = require('bar')

The first two are easy to understand. In the third case, Node starts at the current directory, and adds ./node_modules/, and attempts to load the module from that location. If the module is not found, then it moves to the parent directory and performs the same check, until the root of the filesystem is reached.</p

For example, if require('bar') would called in /home/foo/, the following locations would be searched until a match a found:

• /home/foo/node_modules/bar.js
• /home/node_modules/bar.js and
• /node_modules/bar.js

Loading modules in this way is easier than specifying a relative path, since you can move the files without worrying about the paths changing.

Directories as modules

You can organize your modules into directories, as long as you provide a point of entry for Node.

The easiest way to do this is to create the directory ./node_modules/mymodulename/, and put an index.js file in that directory. The index.js file will be loaded by default.

Alternatively, you can put a package.json file in the mymodulename folder, specifying the name and main file of the module:

{
  "name": "mymodulename",
  "main": "./lib/foo.js"
}

This would cause the file ./node_modules/mymodulename/lib/foo.js to be returned from require('mymodulename').

Generally, you want to keep a single ./node_modules folder in the base directory of your app. You can install new modules by adding files or directories to ./node_modules. The best way to manage these modules is to use npm, which is covered briefly in the next section.

8.2 npm

npm is the package manager used to distribute Node modules. I won't cover it in detail here, because the Internet does that already.

npm is awesome, and you should use it. Below are a couple of use cases.

8.2.1 Installing packages

The most common use case for npm is to use it for installing modules from other people:

npm search packagename

npm view packagename

npm install packagename

npm outdated

npm update packagename

Packages are installed under ./node_modules/ in the current directory.

8.2.2 Specifying and installing dependencies for your own app

npm makes installing your application on a new machine easy, because you can specify what modules you want to have in your application by adding a package.json file in the root of your application.

Here is a minimal package.json:

{ "name": "modulename",
  "description": "Foo for bar",
  "version": "0.0.1",
  "repository": {
    "type": "git",
    "url":  "git://github.com/mixu/modulename.git" },
  "dependencies": {
    "underscore": "1.1.x",
    "foo": "git+ssh://[email protected]:mixu/foo.git#0.4.1",
    "bar": "git+ssh://[email protected]:mixu/bar.git#master"
  },
  "private": true
}

This makes getting the right versions of the dependencies of your application a lot easier. To install the dependencies specified for your application, run:

npm install

8.2.3 Loading dependencies from a remote git repository

One of my favorite features is the ability to use git+ssh URLs to fetch remote git repositories. By specifying a URL like git+ssh://github.com:mixu/nwm.git#master, you can install a dependency directly from a remote git repository. The part after the hash refers to a tag or branch on the repository.

To list the installed dependencies, use:

npm ls

8.2.4 Specifying custom start, stop and test scripts

You can also use the "scripts" member of package.json to specify actions to be taken during various stages:

{ "scripts" :
  { "preinstall" : "./configure",
    "install" : "make && make install",
    "test" : "make test",
    "start": "scripts/start.js",
    "stop": "scripts/stop.js"
  }
}

In the example above, we specify what should happen before install and during install. We also define what happens when npm test, npm start and npm stop are called.