JavaScript Promise

Nothing weights lighter than a promise

This maybe true regarding to human promises, however in the programming domain, promises are always kept. Following this optimistic note, today we'll be talking about JavaScript promises.

Event Handling Problem

Let's see what promises are good for and their basic capabilities starting with a problem they come to solve. Events are great for things of a repetitive nature like keydown, mousemove etc. With those events you don't really care about what have happened before you attached the listener. On contrary calling services and processing their response is a completely different kind of beast. Have a look at the following function, which reads a json file and returns it's content or an error in case of something goes wrong.

function readJSON(filename, callback) {
 fs.readFile(filename, 'utf8', function (err, res) {
     if (err) {
      return callback(err);
     }
     try {
       res = JSON.parse(res);
     } catch (ex) {
       return callback(ex);
     }
     callback(null, res);
 });
}

As you can see there're a lot of checks for errors inside the callback, which if forgotten or written in an incorrect order may cause it's creator quite a headache. This is where promises shine. JavaScript promises are not just about aggregating callbacks, but actually they are mostly about having a few of the biggest benefits of synchronous functions in async code! Namely, function composition of chainable async invocations and error bubbling; for example if at some point of the async chain of invocation an exception is produced, then the exception bypasses all further invocations until a catch clause can handle it (otherwise we have an uncaught exception that breaks our web app).

What is Promise?

A Promise is an object that is used as a placeholder for the eventual results of a deferred (and possibly asynchronous) computation. A promise can always be situated in one of three different states:

• pending - The initial state of a promise.
• fulfilled - The state of a promise representing a successful operation.
• rejected - The state of a promise representing a failed operation.

Once a promise is fulfilled or rejected, it can never change again. The promises ease significantly the understanding of the program flow and aid in avoiding common pitfalls like error handling. They provide a direct correspondence between synchronous and asynchronous functions. What does this mean? Well, there are two very important aspects of synchronous functions, such as returning values and throwing exceptions. Both of these are essentially about composition. The point of promises is to give us back functional composition and error bubbling in the async world. They do this by saying that your functions should return a promise, which can do one of two things:

• Become fulfilled by a value
• Become rejected with an exception

Cross Platform Support

Over the years developer community has sprung numerous implementations of Promises. The most notable are Q, When, WinJS and RSVP.js, however since our blog focuses on the latest developments in the JavaScript world, we'll be only covering newest Promise class introduced in EcmaScript 6. You can see the browsers' support for the feature here, and in case you wish of your program to work in other browsers, as usually you can use the polyfill.

EcmaScript 6 Promise

The Promise interface represents a proxy for a value not necessarily known when the promise is created. It allows you to associate handlers to an asynchronous action's eventual success or failure. This lets asynchronous methods return values like synchronous methods: instead of the final value, the asynchronous method returns a promise of having a value at some point in the future. So let's see our previous example using promises.

function readJSONPromise(filename) {
    return new Promise(function (resolve, reject) {
        fs.readFile(filename, 'utf8', function (err, res) {
            if (err) {
                reject(err);
            } else {
                try {
                    res = JSON.parse(res);
                } catch (ex) {
                    reject(ex);
                    return;
                }
                resolve(res);
            }
        });
    });
}

Oddly it seems very similar. So what do we gain? The true power reveals itself when we try to chain the calls.

readJSONPromise('./example.json').then(function onReadFile(res) {
    return res;
}).then(function onProcessFile(response) {
    console.log('response: ' + JSON.stringify(response));
}).catch(function onError(error) {
    console.error('error: ' + error);
});

Once you return the object, you can pass it to other function for further processing. It allows us to apply the concern separation design in an easy and clean way. You can look at the full code in Git repository.

Operation Timeout in MongoDB

Today I'd like to talk about a problem every MongoDB developer should be aware of - operation timeout. I have surely risen a lot of eyebrows and a few snide remarks, but let me reassure it's worth reading.

Connection vs Operation Timeout

So where do we start? The main problem with operation timeout in any database, not specifically to MongoDB, is the developer's confusion between connection timeout and operation timeout. So let's clear the air right away by clarifying the difference. Connection timeout is the maximal time you wait until you connect to the database. Whereas operational timeout is the maximal time you wait until a certain operation is performed, usually CRUD. This happens after you're already connected to the database.

Post MongoDB 2.6

If you've just started using MongoDB or had a luck to upgrade your existing instance to the newest version, that being 2.6 at the moment of writing, then you should know there is a build-in support for operation timeout by using $maxTimeMS operator in every request.

 
db.collection.find().maxTimeMS(100)

Akward? Surely, but it does the job pretty well.

Pre MongoDB 2.6

But what happens if you don't have the luxury of upgrading your database instance, either from IT or project constrains. In pre 2.6 world, things get ugly. Naturally we want our operations to be constrained within limited timeline, so that we could properly write error logs and take the effective measures. So how do we do this?

MongoDbManager

I've written a MongoDB wrapper library, which uses JavaScript setTimeout mechanism to tackle the issue. The full code can be found in GitHub. Let's look through the main ideas of the library in depth.

find = function find(obj, callback, logger) {
    var filter = obj.filter, name = obj.name, isOne = obj.isOne,
        isRetrieveId = obj.isRetrieveId, limit = obj.limit,
        projection = obj.projection || {};
    if (!isRetrieveId) {
        projection._id = 0;
    }
    connect(function (err1, db) {
        if (err1) {
            callback(err1);
            return;
        }
        var start = logger.start("get " + name), isSent = false,
            findCallback = function (err, items) {
                logger.end(start);
                if (isSent) {
                    return;
                }
                isSent = true;
                if (err) {
                    callback(err);
                } else {
                    callback(null, items);
                }
            };
        setTimeout(function findTimeoutHanlder() {
            if (isSent) {
                return;
            }
            isSent = true;
            callback(ERRORS.TIMEOUT);
        }, SETTINGS.TIMEOUT);
        if (isRetrieveId) {
            if (isOne) {
                db.collection(name).findOne(filter, projection,
                findCallback);
            } else {
                if (limit) {
                    db.collection(name).find(filter, projection)
                    .limit(limit).toArray(findCallback);
                } else {
                    db.collection(name).find(filter, projection).
                    toArray(findCallback);
                }
            }
        } else {
            if (isOne) {
                db.collection(name).findOne(filter, projection,
                findCallback);
            } else {
                if (limit) {
                    db.collection(name).find(filter, projection).
                    limit(limit).toArray(findCallback);
                } else {
                    db.collection(name).find(filter, projection).
                    toArray(findCallback);
                }
            }
        }
    }, logger);
}

A lot of code :( Let's take step by step or in our case line by line. Firstly we connect to the database by calling connect method. It checks whether there is an open connection and opens one in case there isn't. Then we create a timeout callback, findTimeoutHanlder, and queue it's invocation after SETTINGS.TIMEOUT. Right after this, we query the database with find method. Once the data is retrieved our timeout flag, isSent, is set to true, indicating the response was sent. Once the timeout callback is activated, it checks the value of the flag and in case it isn't set to true, error is returned.

Why is that? Activation of timeout callback means we reached a predefined timeout. If flag is still false, then we haven't still received the data from the database and we should quit. When the data is finally retrieved, we check the flag again. If it was set by timeout callback, then we don't need to do a thing, since the error was already returned.

This simple, yet powerful technique is used throughout the library wrapping other operations like update and insert as well. The code is fully documented and has a few examples, which should aid you with understanding the code within one hour.

If you have any questions or suggestions, please don't hesitate to comment below.

JavaScript Singleton Design Pattern

In the previous articles we discussed Factory, Builder and Prototype design pattern. Today it's time to draw a line under creational design patterns by talking about Singleton Pattern.

Even though it's the most well known design pattern among the developers, the thought of writing one in JavaScript, makes most developers tremble. Naturally there is no reason for that and in fact implementing it is not that big of a deal. But first, let's see how it looks in the following illustration:

Basically our singleton contains one instance of itself and returns only it. Client cannot create a new instance or get other instance then one proposed by singleton.

So how do we implement it? The same way, like in any other language - using static classes. To brush off the rust, please read Object Oriented JavaScript article.

var Singleton = (function () {
    var instance;
 
    function createInstance() {
        var object = new Object();
        return object;
    }
 
    return {
        getInstance: function () {
            if (!instance) {
                instance = createInstance();
            }
            return instance;
        }
    };
})();

var instance1 = Singleton.getInstance();
var instance2 = Singleton.getInstance();

console.log("Same instance? " + (instance1 === instance2));  
var instance3 = new Singleton();

In our example we get two instances and check if they are the same. They are! Later we try to create our own instance using new keyword, which of course fails.

Same instance? true
TypeError: object is not a function

Next time we'll talk about behavioral design patterns. Come prepared ;-)

JavaScript Continuous Integration with TravisCI

Last time we talked about automating JavaScript testing with Grunt.js, and even though we quite exhausted the topic, there is one thing left. The provided solution worked well for a solo developer or maybe a small team, however imagine you work with dozen developers, where everyone pushes one's commits constantly. Forcing all of them to follow a procedure of running automated script upon each commit, will be no trifle. Continuous integration comes to rescue. What it does is running predefined build scripts, in our case Grunt.js, on each predefined event - usually on each push.

TravisCI

As usual, we'll start a new topic with the easiest implementation to get you started with the technology. Once you master the basics, we'll continue with more advanced tools in the next article. Today we'll talk about TravisCI and create continuous integration for our last article code and only focus on needed changes. I've copied the code into new Git repository.

TravisCI integrates seamlessly with public and private GitHub repositories. Public ones are free of charge. To get you started, go to TravisCI site, connect with GitHub account and enable the toggle next your repository - that's it! Then we need to configure our repository to play together with the integration server.

Karma and TravisCI

TravisCI only supports Firefox based UI testing, so in order make things work, we need to align both karma.conf.js and karma.conf.require.js Karma configs using process.env.TRAVIS parameter, which notifies us whether we run the tests on Travis machine or not. We'll test our code on Chrome in development environment and on Firefox on integration one

browsers: process.env.TRAVIS ? ['Firefox'] : ['Chrome']

Since there is no actual screen to display the UI, Xvfb is used instead. However you'll need to tell your testing tool process about the display port, so it knows where to start Firefox. All this along with other configurations is stated in .travis.yml file. Before our testing scripts are run, we set the display to 99th configured screen and start xvfb process.

before_script:
  - export DISPLAY=:99.0
  - sh -e /etc/init.d/xvfb start

TravisCI configuration

Let's look into our .travis.yml configuration file line by line:

language: node_js
node_js:
  - 0.10

before_script:
  - export DISPLAY=:99.0
  - sh -e /etc/init.d/xvfb start
  - npm install
  - npm install -g bower
  - bower install

script:
  - grunt

TravisCI supports many languages, we of course interested in Node.js, which is configured in first line. Following the language declaration, we configure the versions of our distribution in lines 2 and 3. Later we define everything, that needs to be done prior to running the scripts.

Since each time we run the integration process on a blank machine, we should install all the packages listed in our package.json and bower.json files. To do so we first run the npm install command and after that both install bower globally and run the bower install directive. Lastly we specify the script command needed to be run, in our case simply grunt, as we want to run all the tasks defined in gruntfile.js file.

Package integration

Lastly, we need to add command name needed for running the tests in package.json.

"scripts": {
    "test": "grunt"
  }

Build status image

Once everything is configured, wouldn't it be cool to show the build status somewhere on team's dashboard or repository readme file. TravisCI provides a simple image, visualizing the status of last build. Just enter repository configuration page and click on the image to the right. Popup will be opened with image URL:

https://travis-ci.org/aie0/jsdeepdive-javascript-continuous-integration-with-
travisci.svg?branch=master

Putting it in readme is one step task, just copy-paste the following line, substituting TRAVISCI_STATUS_IMAGE_URL with status image and TRAVISCI_REPOSITORY_PAGE with TravisCI repository page.

[![Build Status](TRAVISCI_STATUS_IMAGE_URL)](TRAVISCI_REPOSITORY_PAGE)

In our case:

[![Build Status](https://travis-ci.org/aie0/jsdeepdive-javascript-continuous-
integration-with-travisci.svg?branch=master)](https://travis-ci.org/aie0/jsdeepdive-
javascript-continuous-integration-with-travisci)

Hope you enjoyed the article, cause next time we'll be talking about JenkinsCI, which can be configured with any repository.

Automate JavaScript Testing with Grunt.js

So far we've learned how to test your JavaScript code with Jasmine and running them against Node.js and browsers with Karma. We've also got familiar with modular design patterns in JavaScript. And yet, somehow it seems that we're still missing one last puzzle piece connecting all the others, it's called Grunt.js.

What is it?

According to it's site:

In one word: automation. The less work you have to do when performing repetitive tasks like minification, compilation, unit testing, linting, etc, the easier your job becomes. After you've configured it, a task runner can do most of that mundane work for you—and your team—with basically zero effort.

Zero or not, there is a bit of effort in making everything play together, but no worry - we'll figure it out. So what's our plan?

• Write classes, which are both usable in Node.js, Require.js and global environment.
• Write Jasmine specs to test our code in both Chrome and Firefox
• Write Karma and Node.js runners
• Write Grunt task to automate the testing

Writing universal JavaScript classes

In the end we'll type one command to test our code from every aspect. Feeling excited? Let's start! All the code can be found in GitHub, to where I copied some code from my project called Raceme.js, JavaScript clustering algorithms framework (some harmless PR :) First one is Vector class, which wraps the JavaScript array with minor functionality:

(function () {
    'use strict';

    var Vector = function Vector(v) {
        var vector = v;

        this.length = function length() {
            return vector.length;
        };

        this.toArray = function toArray() {
            return vector;
        };
    };

    if (typeof define === 'function' && define.amd) {
        // Publish as AMD module
        define(function() {return Vector;});
    } else if (typeof(module) !== 'undefined' && module.exports) {
        // Publish as node.js module
        module.exports = Vector;
    } else {
        // Publish as global (in browsers)
        var Raceme = window.Raceme = window.Raceme || {};
        Raceme.Common = Raceme.Common || {};
        Raceme.Common.Vector = Vector;
    }
}());

Notice the lower part of the code, where we define our class as AMD module using Require.js, CommonJS module for Node.js and global class for window environment. To spice things up, we'll add additional class, PlaneMapper, which will depend on our Vector class. It exposes one method, mapVector, mapping 2-dimensional coordinate point into vector. The problem with writing dependent universal classes is the loading process. As you remember, Require.js and Node.js use different loading methods - asynchronous versus synchronous. loadDependencies method unifies the approaches into one loading process. Pay attention to continuation of declaration logic in line 29; Once we have our PlaneMapper object defined, we finalize the declaration depending upon the method.

(function () {
    'use strict';

    var COMMONJS_TYPE = 2, GLOBAL_TYPE = 3;
    var loadDependencies = function loadDependencies(callback) {
        if (typeof define === 'function' && define.amd) {
            // define AMD module with dependencies
            define(['common/Vector'], callback); // cannot pass env type
        } else if (typeof(module) !== 'undefined' && module.exports) {
            // load CommonJS module
            callback(require('../common/Vector.js'), COMMONJS_TYPE);
        } else {
            // Publish as global (in browsers)
            callback(Raceme.Common.Vector, GLOBAL_TYPE);
        }
    };
    loadDependencies(function (Vector, env) {
        var PlaneMapper = function () {
            var mapVector = function mapVector(node) {
                return new Vector([node.x, node.y]);
            };

            return {
                mapVector: mapVector
            };
        };

        // finalize the declaration
        switch(env) {
            case COMMONJS_TYPE:
                module.exports = PlaneMapper();
                break;
            case GLOBAL_TYPE:
                var Raceme = window.Raceme = window.Raceme || {};
                Raceme.DataMappers = Raceme.DataMappers || {};
                Raceme.DataMappers.PlaneMapper = PlaneMapper();
                break;
            default:
                return PlaneMapper();
        }
    });
}());

Writing universal Jasmine specs

Code is written, time for testing. We'll create two Jasmine specs, each for one of the classes. As in before, we start with Vector class:

(function () {
    'use strict';
    describe('Mappers', function () {
        var loadDependencies = function loadDependencies(callback) {
            if (typeof define === 'function' && define.amd) {
                // load AMD module
                define(['common/Vector'], callback);
            } else if (typeof(module) !== 'undefined' && module.exports) {
                // load CommonJS module
                callback(require('../../src/common/Vector.js'));
            } else {
                // Publish as global (in browsers)
                callback(Raceme.Common.Vector);
            }
        };
        loadDependencies(function (Vector) {
            var vector;
            describe('Vector', function () {
                beforeEach(function() {
                    vector = new Vector([1, 2, 3]);
                });
                it('check length', function () {
                    expect(vector.length()).toEqual(3);
                });

                it('check toArray', function () {
                    expect(vector.toArray()).toEqual([1, 2, 3]);
                });
            });
        });
    });
})();

Nothing new here - we load the Vector class prior to declaring the spec using the same technique. Same with our mapper, besides loading two classes.

(function () {
    'use strict';
    describe('Mappers', function () {
        var loadDependencies = function loadDependencies(callback) {
            if (typeof define === 'function' && define.amd) {
                // load AMD module
                define(['common/Vector', 'dataMappers/PlaneMapper'], callback);
            } else if (typeof(module) !== 'undefined' && module.exports) {
                // load CommonJS module
                callback(require('../../src/common/Vector.js'), 
                    require('../../src/dataMappers/PlaneMapper.js'));
            } else {
                // Publish as global (in browsers)
                callback(Raceme.Common.Vector, Raceme.DataMappers.PlaneMapper);
            }
        };
        loadDependencies(function (Vector, PlaneMapper) {
            var vector;
            describe('PlaneMapper', function () {
                var mapper, node;
                beforeEach(function() {
                    mapper = PlaneMapper;
                    node = {
                        x: 5,
                        y: 10
                    };
                });
                it('check mapping', function () {
                    vector = mapper.mapVector(node);
                    expect(vector.toArray()).toEqual([5, 10]);
                });
            });
        });
    });
})();

Configuring Jasmine spec runners

Testing Node.js modules is easy - just run the jasmine-node command with path to the specs.

jasmine-node test/spec

Moving on to browser testing. We'll start with easier case using global declarations. First we create Karma configuration file, karma.conf.js. The main interest is in files and browsers sections, where we define our source and spec files in correct order and browsers we want to test.

...
files: [      
  'src/common/*.js',
  'src/dataMappers/*.js',
  'test/spec/*Spec.js'
],
...
browsers: ['Chrome', 'Firefox'],
...

Then invoking the tests using karma command.

karma start karma.conf.js

Lastly, let's test our Require.js modules. Since the modules will by loaded by Require.js instead of Karma, a new Karma configuration file is required - karma.conf.require.js. The first difference appears in frameworks section, where we tell Karma to use Require.js framework. This will require installing additional package called karma-requirejs.

...
frameworks: ['jasmine', 'requirejs'],
...
files: [
    {pattern: 'src/common/*.js', included: false},
    {pattern: 'src/dataMappers/*.js', included: false},
    {pattern: 'test/spec/*Spec.js', included: false},
    'test/test-require-main.js'
],
...

Additional difference comes in files section. Here we inform the test runner not to load our source and spec files. So why to list them at all? Listing the files enables us to use them later, during configuration of Require.js in test-require-main.js. Usually Require.js configuration appears in JavaScript file, mentioned in data-main attribute of script tag. However since we don't want to load HTML files, we configure our modules in test-require-main.js.

(function () {
    'use strict';
    var tests = [];
    for (var file in window.__karma__.files) {
        if (window.__karma__.files.hasOwnProperty(file)) {
            if (/Spec\.js$/.test(file)) {
                tests.push(file.replace(/^\/base\//,
                 'http://localhost:9876/base/'));
            }
        }
    }

    requirejs.config({
        // Karma serves files from '/base'
        baseUrl: 'http://localhost:9876/base/src/',

        // ask Require.js to load these files (all our tests)
        deps: tests,

        // start test run, once Require.js is done
        callback: window.__karma__.start
    });
}());

At first we pass through each file listed in the configuration by using window.__karma__.files list and initiate spec files list. While doing so, we adjust the domain of the specs modules to one used by Karma - localhost:9876. It will also be used as a baseUrl attribute in Require.js configuration. Then we integrate Require.js and Karma together by passing Karma's stating method, window.__karma__.start, as a callback in line 21. The heart of the fusing appears in line 18, where we configure to load our specs prior to calling the callback. Once specs are loaded, callback will be invoked starting the testing.

Writing Grunt tasks

As promised, it's time to integrate all parts using Grunt.js. For this to happen, we'll require four packages: grunt, grunt-cli and grunt-karma, grunt-jasmine-node. The first two for running the tasks and the rest are for calling Karma and Node.js runners. Make sure to install the packages locally into project's folder, otherwise it will not work. In fact all the packages should be installed locally, when you work with Grunt.js.

Installing them can be done easily using package.json and bower.json files. Once the files are in place just call appropriate install commands. It will download all the packages automatically into project's folder.

npm install
bower install

If you an eager environmentalist like me, who doesn't wish to store anything, but essential data on your repository, you may use .gitignore file, which tells Git to ignore specified paths.

node_modules/
bower_components/

Grunt tasks are defined using JavaScript code in gruntfile.js.

(function () {
    'use strict';
    module.exports = function(grunt) {
        grunt.initConfig({
            pkg: grunt.file.readJSON('package.json'),
            karma: {
                unit_global: {
                    configFile: 'karma.conf.js'
                },

                unit_requirejs: {
                    configFile: 'karma.conf.require.js'
                }
            },
            jasmine_node: {
                options: {
                    forceExit: true,
                    match: '.',
                    matchall: false,
                    extensions: 'js',
                    specNameMatcher: 'spec'
                },
                all: ['test/spec/']
            }
        });

        grunt.loadNpmTasks('grunt-karma');
        grunt.loadNpmTasks('grunt-jasmine-node');
        grunt.registerTask('default', ['jasmine_node', 
            'karma:unit_global', 'karma:unit_requirejs']);
    };
}());

Not very intimidating, isn't it? Basically what it does is configures our test tasks, loads the required packages and then runs the tasks. Now in details. At first it configures our Karma tasks by specifying two children in karma node: unit_global and unit_requirejs, each states it's configuration file name. Then it configures Node.js runner. Since it doesn't have any configuration file, all the settings are listed here. In the end, it runs the tasks in the order they appear in parameter array of registerTask method. Notice the usage of semicolon, when Karma tasks are specified. It tells Grunt to run specific tasks under karma node.

Tasks names can be changed, both jasmine_node and karma node's names cannot.

Aren't you eager to see the results?

grunt

Grunt will load and run the gruntfile.js file emitting the following result:

Running "jasmine_node:all" (jasmine_node) task
Common
    Vector
        check length
        check toArray
Mappers
    PlaneMapper
        check mapping
Finished in 0.014 seconds
3 tests, 3 assertions, 0 failures

Running "karma:unit_global" (karma) task
INFO [karma]: Karma v0.12.23 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [launcher]: Starting browser Firefox
INFO [Chrome 36.0.1985]: Connected on socket HrOcIkaJ5aqQG85SOqIS
with id 63263274
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.032 secs / 0.005 secs)
INFO [Firefox 31.0.0]: Connected on socket wrrkgK5_skzDJztmOqIT wi
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.032 secs / 0.005 secs
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.032 secs / 0.005 secs
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.032 secs / 0.005 secs
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.032 secs / 0.005 secs
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.032 secs / 0.005 secs)
Firefox 31.0.0: Executed 3 of 3 SUCCESS (0.026 secs / 0.002 secs)
TOTAL: 6 SUCCESS

Running "karma:unit_requirejs" (karma) task
INFO [karma]: Karma v0.12.23 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [launcher]: Starting browser Firefox
INFO [Chrome 36.0.1985]: Connected on socket PXxh9c5vacKQovhSOsI2
with id 36823086
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.004 secs / 0.002 secs)
INFO [Firefox 31.0.0]: Connected on socket Xu3qldD3wfmNskyOOsI3 wi
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.004 secs / 0.002 secs
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.004 secs / 0.002 secs
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.004 secs / 0.002 secs
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.004 secs / 0.002 secs
Chrome 36.0.1985: Executed 3 of 3 SUCCESS (0.004 secs / 0.002 secs)
Firefox 31.0.0: Executed 3 of 3 SUCCESS (0.005 secs / 0.002 secs)
TOTAL: 6 SUCCESS

Done, without errors.

Perfection! But it's only a tip of the iceberg. We'll be talking more about Grunt.js using conditional logic and reporting, so stay tuned ;)

D3 Data Visualization Library

Data visualization is the study of the visual representation of data, meaning "information that has been abstracted in some schematic form, including attributes or variables for the units of information". In the end everything we do, needs to be somehow presented to the users. Fortunately, we humans are intensely visual creatures. Few of us can detect patterns among rows of numbers, but even young children can interpret bar charts, extracting meaning from those numbers’ visual representations. For that reason, data visualization is a powerful exercise and is the fastest way to communicate it to others.

What is D3?

D3 is a JavaScript library, which helps in manipulating documents based on data. It uses HTML, SVG and CSS to create visualizations. With D3.js, the complete capabilities of new-age browsers can be used without being constrained to a framework. Fundamentally, D3 is an elegant piece of software that facilitates generation and manipulation of web documents with data. It does this by:

• Loading data into the browser’s memory
• Binding data to elements within the document, creating new elements as needed
• Transforming those elements by interpreting each element’s bound datum and setting its visual
 properties accordingly
• Transitioning elements between states in response to user input

Learning to use D3 is simply a process of learning the syntax used to tell it how you want it to load and bind data, and transform and transition elements.

Basics of D3

Naturally the first thing we need to do is to install it, which can be easily done using Bower. If you're not familiar with the tool, please read the article what is Bower is and why you need it:

bower install d3

I myself am made entirely of flaws, stitched together with good intentions

This Augusten Burroughs quote fits me well, when I try to explain someone a new technology. The reason behind this is trying to demonstrate live examples, instead of diving into theory, like many others do. Or maybe write a series of tutorials and gradually increasing the complexity of material. Today will be no different :) On D3 GitHub page, there is a whole gallery demonstrating it's capabilities. Even more examples can be found on Christophe Viau's site. We'll be picking on of them.

Force Directed Graph

I decided to choose Force Directed Graph example, cause it seemed cool enough to get your attention and the implementation wasn't too intimidating. The code can be found here - I've rewritten a bit the example to make it more readable and concise. Let's see what we're building first:

Pull and let go one of the nodes with the mouse and see what happens. Your mind should be blown away instantly, as it cannot comprehend the amount of coolness contained in a single example.

Once you look at the implementation, you'll be even more amazed how little code is needed to make things working and we shall see how the magic happens right away.

var width = 960,
    height = 500,
    svg = d3.select("body").append("svg")
      .attr("width", width)
      .attr("height", height),
    graph = miserables;

force = d3.layout.force()
      .charge(-120)
      .linkDistance(30)
      .size([width, height]);

force.nodes(graph.nodes)
    .links(graph.links)
    .start();

Firstly we create a svg element with specified dimensions - no news here, maybe only the d3.select method, which acts almost as you'd expect. That is selecting the first element, that matches the specified selector string, returning a single-element selection, even if selector matches several elements. If no elements in the current document match the specified selector, returns the empty selection.

Then we create our force layout, a flexible force-directed graph layout implementation using position Verlet integration. Force layout supports many behaviours, at this time we'll be focusing on the ones needed for our example. For broader description about the layout, please refer the it's API page.
In our snippet we override the default charge, linkDistance and size attributes. Size parameter is quite self explanatory. Now, what is charge? Charge is a force, that a node can exhibit, where it can either attract (positive values) or repel (negative values). In our case, repelling. The bigger the value, in it's absolute form, the sparser our graph will be. The last parameter, linkDistance, is the distance we desire between connected nodes. Most often this property is set to a constant value for an entire visualization, but D3 also lets us define it as a function. When we do that, we can set a different value for each link.

After configuring our layout, we initiate it with our nodes and links from the famous Les Misérables novel, formatted in the JSON format. start method starts the simulation; this method must be called when the layout is first created, after assigning the nodes and links.

var miserables = {
  "nodes":[
    {"name":"Myriel","group":1}
    ...    
  ],
  "links":[
    {"source":1,"target":0,"value":1}
    ...
  ]
}

Nodes can except any information, we might find useful in the visualization process. In our nodes we have the name of character and group to which one belongs. The group will be used for dyеing our nodes later. Links, on the other hand, must have source and target attributes, which serve for connecting the directed graph. The objects may have additional fields that you specify; this data can be used to compute link strength and distance on a per-link basis using an accessor function. In our case there is a value, which will be used later to calculate the stroke width. Moving forward:

var color = d3.scale.category20(),
  node = svg.selectAll(".node")
            .data(graph.nodes)
            .enter().append("circle")
            .attr("class", "node")
            .attr("r", 5)
            .style("fill", function(d) { return color(d.group); })
            .call(force.drag);

node.append("title").text(function(d) { return d.name; });

var link = svg.selectAll(".link")
    .data(graph.links)
    .enter()
    .append("line")
    .attr("class", "link")
    .style("stroke-width", function(d) {
      return Math.sqrt(d.value);
    });

We'll be using built-in category20 method, which constructs a new ordinal scale with a range of twenty categorical colors. There are several variations of this method with b and c suffix, returning different palette of colors. You can choose the one you like from library's wiki.

Next we select all nodes using selectAll method, which works similar to its single variant. Interesting fact you may notice, that we don't have any nodes yet. So what will be selected? The way D3 works is returning pseudo-array - a placeholders for our future elements. Once marked, nodes data is applied using data method. Then enter method actually does the job of entering selection: placeholder nodes for each data element for which no corresponding existing DOM element was found in the current selection. Note that the enter method merely returns a reference to the entering selection, and it is up to you to add the new nodes, which we do with append method, inserting the circle element on each node. Then we assign CSS class, radius and fill color. Notice the dynamic nature of fill color, which is retrieved according to the node's group using color alias to category20 method. In the end we invoke force.drag method, using call method, which is mere helper method, made for chaining the calls on the current selection. drag method binds a behavior to nodes to allow interactive dragging, either using the mouse or touch. Finally we add character's name to each node, by adding title element inside our circles.

Next we add the links to connect our nodes. Nothing interesting here, besides dynamically setting the stroke-width CSS attribute according to value attribute. Not sure what good it does, probably just for demonstration purposes. Running the example up until now, will generate all the elements positioned in the top left corner. The next section is what makes everything dance.

force.on("tick", function() {
  node.attr("cx", function(d) { return d.x; })
      .attr("cy", function(d) { return d.y; });

  link.attr("x1", function(d) { return d.source.x; })
      .attr("y1", function(d) { return d.source.y; })
      .attr("x2", function(d) { return d.target.x; })
      .attr("y2", function(d) { return d.target.y; });
});

From the moment we call start method of our force layout, tick events are dispatched for each tick of the simulation. The this section, we listen to the events to update the displayed positions of nodes and links. The event handler is executed at each iteration of the layout. When it does, the force layout calculations have been updated and would have set various properties in our nodes and linked objects, which we could use to position them within the SVG container. First let's reposition the nodes. As the force layout runs it updates the x and y properties, which define where the node should be centered. To move the node, we set the appropriate SVG attributes to their new values. Later we update our links by setting the appropriate start and end points. That's it!

The beauty of D3 library is how it empowers us with tools to visualize nearly for data in the way we like. Some say it is less efficient than it's competitors like Sigma.js and Processing.js, however performance issues, like in any other computer science domain, should be managed and here is a good post to get your started.

If you felt I was moving too fast or wanted to read more detailed tutorial about the library, have a peek at Dashing D3js site - it walks though the material quite thoroughly.

JavaScript Prototype Design Pattern

Let's continue our discussion about JavaScript Design Patterns. We've already talked about Factory and Builder pattern. Today I'll overview the Prototype pattern.

The Prototype pattern creates new objects by cloning one of a few stored prototypes. The Prototype pattern has two advantages: it speeds up the instantiation of very large, dynamically loaded classes (when copying objects is faster), and it keeps a record of identifiable parts of a large data structure that can be copied without knowing the subclass from which they were created. Have a look at the following illustration, depicting the pattern:

While there is a lot of information about cloning on the internet and even some suggest using it in the prototype design, the external approach is utterly incorrect. However since we are Object Oriented programmers, we would like to clone both public and private members. None of the external approaches will give you such result. On the other hand, if you will be willing to settle with public members cloning, might as well use parse/stringify methods combination of JSON class, which give the best results according to the cloning performance tests.

Implementation

We'll be basing our classes on JsDeepDive.Common.Manager example from Object Oriented JavaScript article, to show the effect on both public and private members:

var JsDeepDive = JsDeepDive || {};

function deepClone1(obj) {
  return JSON.parse(JSON.stringify(obj));
}

(function (key) {
 "use strict";
 JsDeepDive.PrototypedEntity = function (someParameter) {
  /* Start private parameters and functions of the class */
  var privates = {
   privateMember: undefined, 

   getPrivateMember: function getPrivateMember() {
    return this.privateMember;
   },   

   setPrivateMember: function setPrivateMember(value) {
    this.privateMember = value;
   },

   _constructor: function _constructor(someParameter) {
    this.privateMember = someParameter;
   }
  };
  /* End private parameters and functions of the class */

  this._ = function (aKey) {
   return key === aKey && privates;
  };
  privates._constructor(someParameter);        
 };

 JsDeepDive.PrototypedEntity.prototype = {
  getPrivateMember: function getPrivateMember() {
   return this._(key).getPrivateMember();
  },

  setPrivateMember: function setPrivateMember(test) {
   return this._(key).setPrivateMember(test);
  },
  publicMember: 1
 };
}({}));

var a = new JsDeepDive.PrototypedEntity(3);
a.setPrivateMember(2);
a.publicMember = 5;
console.log('a.privateMember: ' + a.getPrivateMember());
console.log('a.publicMember: ' + a.publicMember);
var b = deepClone1(a);
console.log('b.publicMember: ' + b.publicMember);
console.log('b.privateMember: ' + b.getPrivateMember());

Once you run the example, you'll encounter into error on line 53, since _ method is undefined, when called in line 36. Let's change things a bit. First we'll extend our _ method, so that we could update the privates property.

this._ = function (aKey, newPrivates) {   
 if (key !== aKey) {
  return;
 }
 if (newPrivates) {
  privates = newPrivates;
 } else {
  return privates;
 }
};

Next thing we do is to add a clone method, which will clone all public and privates using our new _ method:

clone: function clone() {
 var obj = {};
 for(var key in this) {
        obj[key] = this[key];
    }
 obj._(key, this._(key));
 return obj;
}

Now the full pattern:

var JsDeepDive = JsDeepDive || {};

function deepClone1(obj) {
  return JSON.parse(JSON.stringify(obj));
}

(function (key) {
 "use strict";
 JsDeepDive.PrototypedEntity = function (someParameter) {
  /* Start private parameters and functions of the class */
  var privates = {
   privateMember: undefined, 

   getPrivateMember: function getPrivateMember() {
    return this.privateMember;
   },   

   setPrivateMember: function setPrivateMember(value) {
    this.privateMember = value;
   },

   _constructor: function _constructor(someParameter) {
    this.privateMember = someParameter;
   }
  };
  /* End private parameters and functions of the class */

  this._ = function (aKey, newPrivates) {   
   if (key !== aKey) {
    return;
   }
   if (newPrivates) {
    privates = deepClone1(newPrivates);
   } else {
    return privates;
   }
  };
  privates._constructor(someParameter);        
 };

 JsDeepDive.PrototypedEntity.prototype = {
  getPrivateMember: function getPrivateMember() {
   return this._(key).getPrivateMember();
  },

  setPrivateMember: function setPrivateMember(test) {
   return this._(key).setPrivateMember(test);
  },
  publicMember: 1,
  clone: function clone() {
   var obj = {};
   for(var key in this) {
             obj[key] = this[key];
      }
   obj._(key, this._(key));
   return obj;
  }
 };
}({}));

var a = new JsDeepDive.PrototypedEntity(3);
a.setPrivateMember(2);
a.publicMember = 5;
console.log('a.privateMember: ' + a.getPrivateMember());
console.log('a.publicMember: ' + a.publicMember);
var b = a.clone();
console.log('b.privateMember: ' + b.getPrivateMember());
console.log('b.publicMember: ' + b.publicMember);

And the produced successful results:

a.privateMember: 2
a.publicMember: 5
b.privateMember: 2
b.publicMember: 5 

Hope you found it helpful and use this pattern in the future.