This week we continue our series on building better web services with Node.js by taking a look at the Express web application framework. Unfamiliar with Node? Take a look at last week’s blog post to find out what you’re missing.
Why Express?
Express provides all of the tools you need to immediately be productive working on web applications in Node.js. While Node provides a number of built-in networking APIs, they are as a group cumbersome and somewhat unintuitive to work with, forcing you to do a large amount of repeated setup any time you want to set up another endpoint, route, port, or HTTP verb. This is where Express comes in. We can use Express to handle all of the repeat setup tasks associated with building a HTTP server, and spend more time focus on getting the core logic and UI of our applications correct.
Getting set up
To start using Express, just do a normal NPM install:
yourname@domain > npm install express -g
Optionally (but highly encouraged!), you can also install the Express-generator package to provide a convenient terminal command to set up a well-structured Express project. Again, NPM is the bomb.
yourname@domain > npm install express-generator -g
We’ll probably have to install some more dependencies for the generated project later, but this is enough to start rolling.
Making your Express project
Now that Express and Express-generator have been set up, let’s generate an Express project. Express-generator comes with a number of command line options, but for the most part uses sensible defaults. One notable point of contention is the default template engine: Jade. Express-generator can also generate projects using EJS, Handlebars, or Hogan.js. While I personally prefer Handlebars for most purposes (it does a great job of separating logic and templates in a clean fashion), for the purposes of this article I’ll be using EJS in interest of staying as close to Javascript as possible. To generate an Express project using EJS templates, navigate to your project’s intended parent directory and run:
yourname@domain > express --ejs {{YOUR PROJECT NAME GOES HERE}}
This will place a simple Express project configured to template its web pages using EJS in the specified subdirectory. Next, you’ll need to navigate to that directory and run NPM install to resolve your dependencies.
yourname@domain > cd {{YOUR PROJECT NAME GOES HERE}}
yourname@domain > npm install
Note that you don’t provide any arguments to npm install this time. This means that npm will look inside the Node project’s configuration file, package.json, for a list of dependencies to install. This is the same way that you would download dependencies if one of your friends or coworkers gave you a link to a Node.js project to collaborate on. Specifying dependencies like this allows you not to include your dependencies on the project itself, and gives other developers an easy way to resolve these dependencies.
Cool… So what just happened?
Express generator sets up a very simple Express project, configured to perform common backend tasks like parsing requests, setting up routes and complex endpoints, and serving a public directory for client-side assets. Let’s take a look at the Express application’s entry point, app.js.
var express = require('express');
var path = require('path');
var favicon = require('serve-favicon');
var logger = require('morgan');
var cookieParser = require('cookie-parser');
var bodyParser = require('body-parser');
var routes = require('./routes/index');
var users = require('./routes/users');
var app = express();
// view engine setup
app.set('views', path.join(__dirname, 'views'));
app.set('view engine', 'ejs');
// uncomment after placing your favicon in /public
//app.use(favicon(__dirname + '/public/favicon.ico'));
app.use(logger('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(express.static(path.join(__dirname, 'public')));
app.use('/', routes);
app.use('/users', users);
// catch 404 and forward to error handler
app.use(function(req, res, next) {
var err = new Error('Not Found');
err.status = 404;
next(err);
});
// error handlers
// development error handler
// will print stacktrace
if (app.get('env') === 'development') {
app.use(function(err, req, res, next) {
res.status(err.status || 500);
res.render('error', {
message: err.message,
error: err
});
});
}
// production error handler
// no stacktraces leaked to user
app.use(function(err, req, res, next) {
res.status(err.status || 500);
res.render('error', {
message: err.message,
error: {}
});
});
module.exports = app;There’s a lot going on here, so let’s break it down by chunks. In the spirit of providing an overview of Node.js, let’s look at the require statements first.
var express = require('express');
var path = require('path');
var favicon = require('serve-favicon');
var logger = require('morgan');
var cookieParser = require('cookie-parser');
var bodyParser = require('body-parser');
var routes = require('./routes/index');
var users = require('./routes/users');
Require is an extraordinarily powerful tool for looking up libraries and external dependencies within a Node.js project. When you pass a string to require, it will try to look up a properly packaged CommonJS or native Node.js module, keep a reference to the loaded module, and return it to the caller. Then, if you try to load the library again, the cached reference will be returned rather than attempting to load a fresh instance. This lets you efficiently manage dependencies and script loading with almost no effort.
Taking a closer look at the require block, you will notice it is split into two halves, based on the type of require that is happening. The first group is loading pre-packaged Node modules, installed using npm. Require first checks your project’s node_modules folder for these files, and then checks the global npm cache. Express is a good example of a library that will be installed globally (we supplied the option -g to npm install earlier, meaning the library was installed at a system level), whereas the other modules were downloaded to the node_modules folder by NPM install, and will be loaded from there.
The second group of requires loads libraries local to the current project. This type of require uses normal relative file paths (. represents the current directory, .. is the parent), and will attempt to load javascript files relative to the current file’s parent directory. These statements load a pair of router files which will define a set of endpoints. We’ll take a closer look at these later.
Now let’s start serving
Now that we’re done resolving our dependencies, we can actually start initializing our Express project. Let’s take a look at the next couple of lines in app.js.
var app = express();
// view engine setup
app.set('views', path.join(__dirname, 'views'));
app.set('view engine', 'ejs');
// uncomment after placing your favicon in /public
//app.use(favicon(__dirname + '/public/favicon.ico'));
app.use(logger('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(express.static(path.join(__dirname, 'public')));
app.use('/', routes);
app.use('/users', users);
We do a couple of important set up tasks here. The obvious big one is creating our Express instance, but we also set up how we will be rendering our web pages and where to find them (in the views subdirectory), and begin to set up the actual logic of the Express application.
Express is built on the concept of middleware functions, small little functions that are each capable of processing or responding to a request, or sending the request to the next piece of middleware capable of handling your request. You can use middleware to set up generic handling for all endpoints, specific handling for endpoints matching some route, or specialized handling for an individual endpoint. This is done primarily via the use function.
use has a couple of nuances that allow you to set up complex server-wide handling easily. Looking at the use calls, you can see that some calls are passed a single argument, while others receive two. The single argument calls apply the selected piece of middleware to every request that reaches the server, while the two args version allows you to restrict the middleware to operating on a single route. We set up our pre-generated routes at the end of this call.
Next, we set up a couple of custom middleware functions.
// catch 404 and forward to error handler
app.use(function(req, res, next) {
var err = new Error('Not Found');
err.status = 404;
next(err);
});
// error handlers
// development error handler
// will print stacktrace
if (app.get('env') === 'development') {
app.use(function(err, req, res, next) {
res.status(err.status || 500);
res.render('error', {
message: err.message,
error: err
});
});
}
// production error handler
// no stacktraces leaked to user
app.use(function(err, req, res, next) {
res.status(err.status || 500);
res.render('error', {
message: err.message,
error: {}
});
});
Again, there’s a lot going on in this series of calls, so let’s break down what’s happening. Middleware is ultimately some function that will be executed, and these functions always receive the same three arguments, in the same order. The first argument is the request object. The request object is where you will look to get information about an individual request coming to your server. The request can be preproccessed by middleware, and since all pieces of middleware receive the same object reference for a given request, all other pieces of middleware will receive this information. That’s how the bodyParser library from earlier works. It allows JSON payloads to automatically be parsed and turned into a request body object, rather than plain text.
The second argument is the response object. The response object is used to do things like set HTTP status codes or response headers, report request progress back to the client, and actually send your response to the client when you are done handling the request.
The third argument is the next object. This is a reference to the next function in the middleware stack. You can call next to indicate that you are done attempting to handle this request, and that the next piece of middleware should make its attempt at handling. Middleware is run in order of execution, so the first thing that you app.use that matches the request’s path will be executed first, proceeding until either a response is sent or we run out of middleware to attempt to handle the request with.
There is a special case of middleware, namely error handlers. Error handlers must accept exactly four arguments (e.g. the function’s arity must be four). The last three arguments are the same as a normal piece of middleware, but an error object must be accepted as an additional first argument
Taking a look at the middleware added at the end of app.js, we have a single piece of standard middleware that will handle any routes that do not have middleware set up to respond, and will return a 404: not found message instead. The remaining middleware defines the development and production error handlers. As you can see these are four-argument functions, and are as such error handling middleware. Since the first piece of error middleware sends a response, in the dev environment we’ll return error messages with full stack traces, whereas in production we will merely notify the user that an error occurred.
Finally, on the last line of app.js, we export our Express application using the normal commonjs module.exports syntax.
So, about those route things…
Routes are a powerful tool that Express gives you to organize your code in a sensible fashion. Express wouldn’t make for a very good framework if we had to put everything in app.js, now would it? Let’s take a look at one of the generated route files that we required earlier.
var express = require('express');
var router = express.Router();
/* GET home page. */
router.get('/', function(req, res) {
res.render('index', { title: 'Express' });
});
module.exports = router;As you can see, the way a router works is very similar to the way to how your base Express application works. You initialize an instance of your router, register some number of pieces of middleware, and export it for your consumer (almost always your Express application). One notable difference is that instead of using use to register middleware, we are using a function named get. get is a convenience function for registering middleware that only executes for requests with the HTTP verb GET. Similarly, there is a function for PUT, POST, DELETE, and the other HTTP verbs as well. The second nuance of routers is that they inherit the base route of where they are used by the application. For the users route for instance, all endpoints will start with /users, and then have any additional route tokens appended to the end. Additionally, you can nest routes as deeply as you need to to achieve your desired code organization.
Cool, but how do I start it?
NPM comes to the rescue again. Express-generator set up the project’s package.json with a pre-defined start command, which will set up your application to run on a port defined by your shell’s environment variables, or the default port of 3000. Take a look at bin/www to see how this works, or just run npm start to get your application running! Note that npm start will take control of your current console and use it for logging, you may want to direct your output to a file by running something of the form npm start >> log.txt
So where do I go from here?
So far we’ve covered setting up your node instance and configuring it to serve a web service using Express, but there’s still a lot of learning to be done! In the coming weeks, we’ll go over how to take this basic Node.js HTTP server and configure it to serve templated web pages using EJS, we’ll go over how to configure a simple noSQL database for Node.js using MongoDB, and we’ll pull it all together to write a simple push notification server, controlled through a RESTful API.