Recently I have been doing a lot with Node.js. It is a nice server technology that enables you to program on the server like you program on the client. Wouldn’t it be nice to create a java based business logic backend but a light weight client and server backend. Communication between node.js and java is done in two ways in the sample. We use http and json for querying the data and for providing new data. Next to that I have an event system that pushes events with new data to the clients using nowjs. The following image gives you an idea about the overall solution.

Overview solution

If you want to find out more about this sample read on.

Introducing the sample

The sample is based on the famous address book from the axonframework project. We reuse the axon sample and create a rest like interface that returns json to the clients. The source code is in the rest-ui project within the axonframework source tree under samples. This project also contains the listener code for the Contact and Address based events. The listeners listen for events and publish content from these events to the redis pub/sub mechanism. More on this in the later sections.

The other part of the sample is a node.js application. This application makes use of the rest api to obtain the data via http requests. Events about data changes are received through redis, using the pub/sub mechanism. The node.js application is available on Github. If you want to follow along, you can find the sources here:

https://github.com/jettro/axon-addressbook-nodejs
http://code.google.com/p/axonframework/source/browse/trunk#trunk%2Fsample%2Faddressbook%2Frest-ui

The following screen dump gives you an idea about the application.

Screen Shot 2011 07 25 at 19 06 57

In the screen you see the list of contacts. You can add a new contact, change the name of a contact and add addresses to contacts. On the right you see the messages pane. Here you receive messages about new contacts that have been created, changed or deleted.

Now let us have a look at some of the code. We start with the java backend and then continue with the node.js part.

The java part of the sample

Numerous blogs have been written about rest based clients using spring mvc. Therefore I do not want to lose to much time in that area. Most important spring configuration is within the web.xml and the dispatcher-config.xml. In the web.xml we configure that every request is going to the spring dispatcher. We do not have status resources, only json responses. Within spring we make use of annotation to configure the beans. To be able to respond with json we make us of Jackson to transform beans into json objects. All pretty straightforward spring. Check the config file for yourself if you want to see how it works.

Next step is the controller that handles all the requests.

Handling requests in the Cntroller

The Controller handles the following requests:

  • /contacts : GET, returns the list of all contacts
  • /contacts : POST, creates a new contact
  • /contacts : PUT, changes an existing contact
  • /contacts : DELETE, removes a contact
  • /contacts/{identifier} : GET, returns the details of a contact
  • /contacts/{identifier}/address : PUT, creates a new address or updates an existing one based on address type
  • /contacts/{identifier}/address : DELETE, removes the address of provided type

Let us have a look at one of the requests to show the flow. For example we take the request for details. The following code block shows the complete method.

@RequestMapping(value = "{identifier}", method = RequestMethod.GET)
public @ResponseBody Map<String, Object> details(@PathVariable String identifier) {
    List<AddressEntry> addressesForContact = repository.findAllAddressesForContact(identifier);
    String name;
    if (addressesForContact.size() > 0) {
        name = addressesForContact.get(0).getName();
    } else {
        name = repository.loadContactDetails(identifier).getName();
    }
    Map<String, Object> map = new HashMap<String, Object>();
    map.put("addresses", addressesForContact);
    map.put("identifier", identifier);
    map.put("name", name);
    return map;
}

The RequestMapping annotation tells spring which requests need to be mapped to this method. The controller already maps all /contacts to this controller. The {identifier} now maps everything with a url like /contacts/{identifier} to this method, as long as it is a GET request. This identifier is also used as a parameter for the method. With the @PathVariable we tell spring to take the value for this parameter from the url or Path that was requested. The final annotation in the method declaration is the @ResponseBody. This tells spring to use the return value as the response body, so not as the name of the view to use. We have already configured to use jackson to transform the response body into json. In the code of the controller you see we return a map with three objects in it. A list of addresses, an identifier of the contact and the name of the contact. The json coming out of this is:

{
"name":"Jettro",
"addresses":[
{
"name":"Jettro",
"identifier":"82b4fef5-3389-440d-8893-6831a64e200d",
"addressType":"PRIVATE",
"streetAndNumber":"Feyenoordlaan 010",
"zipCode":"3000AA",
"city":"Rotterdam"
}],
"identifier":"82b4fef5-3389-440d-8893-6831a64e200d"
}

That is what you need to know about the rest api that I have created. If you want to have a look at the controller, check here. The next step is about the listeners that listen for axon events related to the contacts and addresses and store information in redis.

Listening for events

We register listeners with axon for the contact related events as well as address related events. This is very easy with the annotations provided by the axonframework. The following code block shows how we register an event listener for the ContactCreatedEvent.

@EventHandler
public void handleContactCreatedEvent(ContactCreatedEvent event) {
    ContactEntry value = new ContactEntry();
    value.setName(event.getName());
    value.setIdentifier(event.getContactIdentifier());
    Message<ContactEntry> message = new Message<ContactEntry>("contact-created", value);
    publisher.publish(message);
}

Notice the @EventHandler annotation. This registers the listener with axon. As you can see we take the information from the provided event and put it in a Message object. The message object contains a type and the content. The type can be used at the receiving side to better understand how to handle the received event. The publisher is auto injected by spring. In my case it is a redis publisher.

Publishing to redis

A library to talk with redis is jedis. Jedis makes use of a pool which is configured in spring. Check the listener-context.xml to see how it works. Jedis does not have the most obvious api but it works. The following code block shows how a message is published to redis.

public void publish(Message<?> message) {
    StringWriter writer = new StringWriter();
    try {
        mapper.writeValue(writer, message);
    } catch (IOException e) {
        logger.warn("Problem while writing ContactEntry to a string writer", e);
        return;
    }

    Jedis jedis;
    try {
        jedis = jedisPool.getResource();
    } catch (JedisConnectionException e) {
        logger.debug("Could not obtain redis connection from the pool");
        return;
    }
    try {
        jedis.publish("nl.axonframework.examples.addressbook", writer.toString());
    } finally {
        jedisPool.returnResource(jedis);
    }
}

The code is not very complicated. The important part here is that we use a jackson provided ObjectMapper to map the provided message to a json representation. Then you can see the nice jedis api at work. As you can see we publish a message to a queue. Our node.js client will subscribe to this queue. That is have the redis publish/subscribe works. Not very hard is it?

That is it for the java side. We now know how to get data using the rest api and we know where to get information about events that take place. Let us move on to the node.js side of the application.

The Node.js part of the sample

I do not want to go in setting up a node.js project. Do not want to explain all the different libraries that I use. I have written other blog posts on node.js that I do not want to copy. The most important one is this:

http://blog.jteam.nl/2011/04/18/learning-node-js/

The node.js project uses 4 libraries:

  • express – doing the mvc part of the web application
  • jade – view templating
  • now – server push to the client (will explain this in a bit more detail)
  • redis – for redis interaction

Project layout

The public folder contains all the client related files like javascript, css and images. I will explain one file in here later on when discussing node.js. The test folder contains the test for the ContactRepository. Using this test helped me to verify that the integration between the java backend and the node.js application is working without having to start the application and go through the browser to check all requests. The views folder contains all the jade templates. Check mentioned blog post to find more information on jade. The app.’s initializes the application and maps all possible requests to methods in the ContactController. The controller knows where to get the data from and what page to render. The repository is responsible for obtaining data from the rest api on the java side.

Initializing the node.js application

As mentioned the app.js is the start of the application. Let us have a look at the different parts.

var ContactController = require('./ContactController');
var repository = require('./ContactRepository').createRepo('localhost',8080);
var contactController = new ContactController(repository);

This part initializes our components. We will have a look at the createRepo method later on. But notice that we initialize the controller with an instance of the repository.

app.get('/', contactController.listContacts);
app.get('/contact/new', contactController.newContactShowForm);
app.post('/contact/new', contactController.newContactPostForm);
app.get('/contact/:identifier',contactController.contact);
app.get('/contact/:identifier/edit',contactController.changeContactShowForm);
app.post('/contact/:identifier/edit',contactController.changeContactPostForm);
app.get('/contact/:identifier/delete',contactController.deleteContactShowForm);
app.post('/contact/:identifier/delete',contactController.deleteContactPostForm);
app.get('/contact/:identifier/address/new',contactController.newAddressShowForm);
app.post('/contact/:identifier/address/new',contactController.newAddressPostForm);
app.get('/contact/:identifier/address/:addressType/edit',contactController.changeAddressShowForm);
app.post('/contact/:identifier/address/:addressType/edit',contactController.changeAddressPostForm);
app.get('/contact/:identifier/address/:addressType/delete',contactController.deleteAddressShowForm);
app.post('/contact/:identifier/address/:addressType/delete',contactController.deleteAddressPostForm);

These are all the possible requests. The urls are easy to understand. As you can see, all urls are mapped to a method of the contactController

var Now = require('now');
var everyone = Now.initialize(app);

var redis = require("redis"),
    client = redis.createClient();

client.on("error", function (err) {
    console.log("Error %s", err);
});

client.on("message", function (channel, message) {
    everyone.now.receiveContact(message);
});
client.subscribe("nl.axonframework.examples.addressbook");

This is the harder bit. This has to do with receiving messages using the redis pub/sub mechanism and sending it to all connected clients using the nowjs library. I’ll discuss the now.js part later on.

Request handling

By using the repository object the controller becomes very easy. We only have to obtain the right data and render the right view. The most basic method is showing a list of contacts. The following code block shows the complete method.

ContactController.prototype.listContacts = function(req, res) {
    repository.listContacts(function(contacts) {
        res.render('index', {locals: {contacts: contacts, nowjs: true}});
    });
};

Notice that we use a callback that is provided to the repository listContact method. Check that we provide the list of contacts to the view, together with a second parameter called nowjs. Remember this, on this page we enable nowjs, meaning that we can receive messages when looking at this page. Other methods are similar, have a look at the method that handles submitting a form to create a new contact.

ContactController.prototype.newContactPostForm = function(req, res) {
    repository.newContact(req.body.new_name, function(code, message) {
        if (code == "ok") {
            res.redirect("/");
        } else {
            res.render('newcontact', {locals: {error:message}});
        }
    });
};

In line 2 we obtain the form parameter new_name from the request body. Than we create a new contact. The callback method expects a code indicating whether the submission of the new contact was ok. If not, we render the form again with the error message. This happens when we try to create a contact with a name that already exists. The following image shows the screen with an error message.

Screen showing the error message

Next up is the communication with a rest based application

The Repository as a client to a rest application

During the initialization of the application I already mentioned the createRepo method of the ContactRepository. The following code block shows the method. As you can see it initializes a repository with the default values for host and port.

ContactRepository.createRepo = function(host, port) {
    var repo = new ContactRepository();
    repo.host = host;
    repo.port = port;
    return repo;
};

Again most of the interaction with the backend is the same. Therefore I use the create contact again as an example. I have created tests for all the repository methods. Let us have a look at the test first.

function testNewContact() {
    logTestName("New contact");
    repository.newContact("My Test", function(code, message) {
        assert.equal("ok", code, "This should be no problem and an ok should be returned: " + code);
        // Check if the amount of contacts is increased
        repository.listContacts(function(contacts) {
            assert.equal(numContacts + 1, contacts.length, "Number of contacts is not right, create did not work: " + contacts.length);
            testAddAddress();
        });
    });
}

Notice that we check the return code and that the amount of contacts is increased. The following code block shows the implementation of the method of the repository.

ContactRepository.prototype.newContact = function(name, callback) {
    var opts = createHttpRequestOpts('/contacts', 'POST');

    var req = http.request(opts, function(res) {
        res.setEncoding('utf8');
        res.on('data', function(data) {
            if (res.statusCode != 200) {
                console.log(data);
                callback('error', 'Maybe the name is already taken?');
            } else {
                callback('ok', 'The new contact has been send')
            }
        });
    });

    var contact = {};
    contact.name = name;

    req.write(JSON.stringify(contact));
    req.end();
};

function createHttpRequestOpts(path, method) {
    return {
        host: host,
        port: port,
        path:path,
        headers:{'Accept':'application/json','Content-Type':'application/json'},
        method: method
    };
}

I use the nodejs provided http request. Within the help function the opts object is created with the host, port, path, method and the headers. The headers are important to request json. Of course we provide a callback to handle the http response. Everything not 200 is an error as a response code. Here we call the provided callback to the repository method with the ok or error code. At the end of the method we do the actual request creation with the json request.

The final bit is the server push using the now.js library.

Server push using now.js

Node.js comes with a socket.io library. This makes server push very easy. Still it can be a lot easier. That is when using now.js. Now.js enables you to call methods on the client that are actually on the server and vise versa. Let us have a look at the code for sending messages to the client when a redis event is received again.

var Now = require('now');
var everyone = Now.initialize(app);
client.on("message", function (channel, message) {
    console.log("Received message: %s", message);
    everyone.now.receiveContact(message);
});

The important part here is the initialization of now and the line with everyone.now.receiveContact. Now we will call the connected clients method receiveContact and provide the message. Of course we need some client code for this. This code is available in the local.js file in the public folder.

$(document).ready(function() {
    now.receiveContact = function(data) {
        var message = JSON.parse(data);
        var toWrite;
        switch(message.type) {
            case 'contact-removed':
                toWrite = "Contact removed";
                break;
            case 'contact-created':
                toWrite = "Contact with name " + message.content.name + " created";
                break;
            case 'contact-changed':
                toWrite = "Contact changed name to " + message.content.name;
                break;
            case 'address-created':
                toWrite = "Added address of type " + message.content.addressType + " to contact " + message.content.name;
                break;
            case 'address-removed':
                toWrite = "Removed address of type " + message.content.addressType + " to contact " + message.content.contact.name;
                break;
        }
        $("#messages").prepend("<div>" + toWrite + " </div>");
    };
});

As you can see, we construct the message based on the type of the message and prepend a new div within the #messages div. Really that is all there is to it.

Last remarks

That is it. Again a long blog item. Hope you got some new knowledge out of it. Feel free to post comments if you have questions or even better improvements.

Combining java and Node.js through redis pub/sub and a json remote interface
Tagged on:                 

6 thoughts on “Combining java and Node.js through redis pub/sub and a json remote interface

  • March 25, 2014 at 1:55 pm
    Permalink

    It’s not entirely clear to me what the point is of using Redis as opposed to just connecting with Spring. Is it the non-blocking nature of Node.js that requires a more event-driven infrastructure? If so, would this now be workable with Spring 4.0 or Spring Reactor as well?

    • March 27, 2014 at 8:24 am
      Permalink

      The idea is indeed to make it an event driven application. It is interesting to have a look at spring 4 or spring reactor. At the time I wrote this 3 years a go these technologies were not yet available. At the moment I do no really have the time to invest in this.

  • July 28, 2011 at 6:01 pm
    Permalink

    I count 8 different pieces for this. That’s a lot of moving parts. That’s what I like about java programmers; they’re not afraid of complexity!

    What are the advantages of this complex setup?

    • July 29, 2011 at 7:48 am
      Permalink

      That is actually a very good question. I think that in a lot of scenarios this is not the right setup. But in complex environments with a lot of different systems that need to interact with each other you often have to look for different solutions. It might also help to get a new technology like node.js adopted. The business logic is in java, and that nice website is in node.js. We have done a lot of projects in which a specialized company did the front-end/website and we programmed the backend business processes. They wanted php, we used java. In those scenarios this setup becomes interesting. The fact that it is completely decoupled makes it easy to create a complete new website without changing anything on the server.

  • July 28, 2011 at 4:39 pm
    Permalink

    Nice write-up!
    If you’re using Redis, you might be interested in Spring Data Redis: http://www.springsource.org/spring-data/redis
    Supports 3 different Redis clients (Jedis, JRedis and RJC) and has JSon integration with Jackson so you can serialize objects to a value or to a hash (if you want to).
    Additionally you could use the support classes so you can interact with Redis through a List or a Set interface (and the framework we’ll handle the rest).

    Cheers,
    Costin
    Spring Data Redis Lead

    • July 28, 2011 at 5:17 pm
      Permalink

      Thanks, I know the project of course, but this gives me a better reason to have a look at it.

Comments are closed.