Test fixtures are now part of the axon-test module. These fixtures allow you to [...]]]> Today, I released version 0.6 of the Axon Framework. 0.6 has many new features and is another step towards full production readiness. There is still some work to do, but first, let’s take a look at what has changed…

Test fixtures are now part of the axon-test module. These fixtures allow you to test your command handling logic based on events and commands. You define these tests in a given-when-then style: given these past events, when I execute this command, I expect these events to occur.

The CommandBus interface now allows for asynchronous command dispatching. Instead of using a return value, a Callback can be given to the command bus if you wish to be notified of the result of command execution.

The repositories make use of a UnitOfWork instance, which keeps track of changed aggregates and generated events. Instead of saving each aggregate separately, the UnitOfWork can be committed, causing changed aggregates to be stored, and events to be dispatched. With the new SimpleUnitOfWorkInterceptor or the SpringTransactionalInterceptor, it is no longer necessary to call repository.save(). In fact, you are discouraged from doing so.

Repositories are now capable of detecting conflicting modifications. When loading an aggregate, you may pass in the expected version of the aggregate. An exception is raised if that version does not match the actual version.

Event sourcing repositories allow for a more advanced way of detecting conflicts. Sometimes, not all concurrent modification are in conflict with each other. One user may very well change the shipping address of an order, while another user is adding items. Other modifications, however, are clearly conflicting, such as one user marking an order as shipped, and another one adding a product. For this purpose, you can register a ConflictResolver with the repository. This conflict resolver can, based on unseen and newly applied events, decide whether the changes made by the second user are accepted.

Axon now provides a mechanism to trigger snapshot creation based on the number of events needed to load to reconstitute an aggregate. When the number of events exceeds a given threshold, a snapshot is created and stored in the Event Store.

The SimpleCommandBus and SimpleEventBus now expose statistics over JMX. Although currently quite limited, the statistics will be improved in the coming versions.

All of these features help making applications with CQRS-based architectures even easier. But we’re not there yet. The coming months, we will try to make it even easier with features like Event versioning, support for aggregates with multiple entities, and whatever comes to mind in Axon’s growing community.

For more information about the Axon Framework, see www.axonframework.org.

In this post, I describe some CQRS event [...]]]> Command-Query Responsibility Segregation (CQRS) is slowly but steadily gaining ground as an architecture that helps developers to develop scalable, extensible and maintainable applications. Events play a major role in this architecture, and the way you design these events greatly influence the extensibility of your application.

In this post, I describe some CQRS event basics and design considerations that help keep your application extensible.

The full post is published on the JTeam blog. Continue to read here: http://blog.jteam.nl/2010/01/27/cqrs-designing-domain-events/

Command Query Responsibility Segregation (CQRS) is an architectural style that makes a clear distinction between commands, which tell an application to do something, and queries, which are requests for information from an application. This distinction comes from the fact that the requirements (and thus also the model) for the execution and validation of [...]]]>  

Command Query Responsibility Segregation (CQRS) is an architectural style that makes a clear distinction between commands, which tell an application to do something, and queries, which are requests for information from an application. This distinction comes from the fact that the requirements (and thus also the model) for the execution and validation of commands are significantly different than those for queries. Events play an important role in the synchronization of application state resulting from executed command.

Applying a CQRS style architecture involves the development of quite a lot of “plumbing” code: event dispatching, asynchronous event processing, transactions, etc. cqrs4j, an Apache 2 licensed open source framework, takes care of all the plumbing for you. Read on to find out how…

A brief introduction to CQRS

As I stated above, CQRS makes a distinction between the model that validates and executes commands and the model that is used for providing state information. This is quite significantly different than how most (web)app applications are build nowadays. In fact, the model used for commands (the box named “Domain” in the image below), does not expose any state, at all. You might wonder, but how do I know what to show in the front end? That comes directly from your data sources using a thin data layer. I’ll explain how the data gets there in a minute.

When a command comes in, it will load an aggregate from the repository and execute certain operations on it. As a result of these operations, the aggregate produces events, which are picked up for storage by the repository and for dispatching by the event bus. The event bus will dispatch each event to all (interested) event handlers. Some of these event handlers will perform actions on other (related) aggregates, some others will update the database tables they manage.

Having handlers update the data in the database means that your tables do not have to be normalized anymore. Instead, CQRS allows you to optimize your tables for the way you want to query them. This makes the data layer processing your queries really simple, and maintainable.

Furthermore, since all state changes are initiated by events, these events become a reliable source for an audit trail. By storing these events, you have an audit trail that you can use to replay the entire application history. Instead of just seeing “current application state” only, you can see all the changes that have led to the current state, providing valuable information trying to find bugs or deal with customer complaints.

The asynchronous and event driven nature of CQRS makes it extremely valuable for complex applications that have different views on the information in the application. Integration with third party systems that need to be notified of certain changes in your application –and vice versa–  is a lot easier if you use an event driven approach. And since event handling is done asynchronously, the application is more responsive and easier to scale.

But as you notice, there is quite a lot of processing and pushing around of events. When you have a customer breathing up your neck, you don’t want to be developing infrastructure code. You would want to be focusing on the business logic, which is quite complex by itself.

cqrs4j

cqrs4j is a Spring-oriented framework that provides the most important building blocks of a CQRS architecture. Creating an aggregate becomes really simple. You don’t have to worry about managing the storage and dispatching of (uncommitted) events. Event sourcing becomes as easy as wiring two beans in your application context.

Annotation support

cqrs4j comes with out-of-the-box annotation support. This makes is really easy to wire event handler methods. cqrs4j can automatically subscribe all your event handlers to the event bus and delegate all relevant events to the appropriate event handlers. Enabling annotation support is as easy as wiring a single bean in your application context.

Spring Integration support

Spring Integration is a framework that allows easy developments of a Messaging systems using the pipes-and-filters architecture. This fits nicely with most of the event dispatching process of CQRS. cqrs4j has support for Spring Integration, which allows you to publish all events as messages on a Spring Integration channel. Sending messages through JMS queues, via email, or through file system storage will only take a few lines of (XML) configuration.

Transaction support

If you update your database tables through incoming events, dealing with them one-by-one in a transaction can become time consuming and take up too much resources of your database. With cqrs4j, you can configure transactions just by setting the @Transactional annotation on your event handlers. You can also configure how many events should be handled in a single transaction.

cqrs4j project page

cqrs4j is an open source project, licensed under the Apache 2 license. Visit the project home page for the latest downloads, documentation and source code at code.google.com/p/cqrs4j.

We welcome your feedback

If you have any requests, remarks or other feedback, please let us know. You can report bugs and improvement requests on the issues page. You can also leave a message via the contact page or by leaving a comment.

Further reading

I have some more CQRS related blog articles coming up, but there are already quite a few around. Here is a few that helped me get started:

• Greg Young @ QCon: Unshackle your domain
• CQRS a la Greg Young
• CQRS – The Domain Events
• CQRS – Domain State
• DDDD – Why I love CQRS

Software Architects play an important role in setting up an [...]]]> Applications have to run in high-consequence environments. They have to serve hundreds of thousands of users 24 / 7. Our clients spend millions in hard- and software and highly depend on the revenue generated by these applications. Unnecessary outage of these application is fatal.

Software Architects play an important role in setting up an architecture that can cope with these high demands. At the JAOO, Michael Nygard‘ had a talk, “Failure comes in Flavors“, that gave very good insight in the risks and opportunities of today’s application. The talk was was divided into two sessions. The first session covered the bad news: the stability threats. He discussed several situations that pose a threat to the long and happy life of an application. The second session was a happier one. It covered the patterns that should be applied to the application architecture to prevent these threats.

In this post, I will elaborate on some of the stability threats and pick one specific pattern to resolve them: the circuit breaker.

Stability

Before we go on about the stability threats, we need a common understanding about what stability is.

Michael Nygard defines stability as “the consistent, long-term availability of features”, and can be divided into four different qualities:

• Severability; instead of crashing completely, only loose the functional area’s that are struck by failure
• Resilience; automatically recover from failures
• Recoverability; make sure only crashed components need to be restarted, not the entire application
• Tolerance; absorb shocks from other components instead of propagating them

Note that stability doesn’t mean that nothing will ever go wrong. It just states that when something does go wrong, only the features of the applications directly related to the struck component become unavailable, and only until the component comes up again.

Stability threats

The life of a web application developer is interesting. We build applications that are hosted in the most hostile environment imaginable: the internet. People can travel at light speed, nearly invisible and have all the power to bring your application down, willingly or accidentally. In fact, more application crash because of accidental abuse than of hackers.

This brings us to the first threat: the user. Unfortunately, the user pays for our client’s income (and ours too). We’ll have to deal with them.

Several aspects of the users behavior pose a threat for our application:

• Users don’t wait. If a page doesn’t show up quickly enough, he’ll push the link again, starting two threads to work for him, only wonderings the situation
• Users write about cool stuff and come to your site after reading that stuff. This may cause a sudden burst of users, possibly overloading your site
• Buyers will use all integration points, causing traffic in every little corner of your application
• Some users aren’t actually users, but bots, and may cause unused sessions to be created for them
• Users only visit your site when they’re awake, causing traffic surges at daytime, while your processors wait for orders at night.

A threat briefly mentioned above is the integration point. This is the point where your application meets another. An example of this is the database or an external payment provider.

Integration points are dangerous for a couple of reasons:

• External systems typically have to be reached over a network connection, making the integration dependent of ever more components
• Ever had the problem of a firewall dropping your packets? It’s a nice way of not being able to set up a connection without getting any feedback about why this happens.
• Some external systems just keep you waiting. If your payment provider is busy, they’ll be happy to keep you waiting. But will your user wait? How many threads will clog up and the integration point, waiting for a response?

I could go on for a long time, but these are the two threats that I want to cover in this post. If you want more, read Michael Nygard’s book “Release It”.

Stability patterns

Fortunately, there is a series of patterns that can be applied to increase an application stability. One of these patterns is the circuit breaker. You’re most likely familiar with the electrical circuit breaker. When you have an electricity leak or high power throughput, the circuit breaker will open and stop the electrical current. You’ll be in the dark, but at least your television, washing machine and music installation are saved from destruction.

Now, imagine your database has a hard time keeping up with your amount of requests. It will probably take longer to process each request, causing even more requests to pile up. In the end, your database server might break, taking your application with it. That is the time where we need a circuit breaker to stop the current from going to our database. I’m talking about digital current here, not electricity. Let’s keep that fans turning on the hardware!

A software circuit breaker should do a few things. Most important is that is should monitor each call performed on a specific backend. When a certain amount of calls fail or take too long, the circuit breaker opens, blocking the current. Each request coming in when the circuit breaker is opened will result in an immediate failure, throwing the last exception that was received from the last call. This has two positive effects: your thread returns more quickly and the load on the external system is relieved, allowing it to recover.

After a certain period of time, the circuit breaker will let a single request pass through to the backend. If that call fails, nothing changes. If the call succeeds, the circuit breaker will close, allowing the requests to pass through to the external system.

The number of use cases for a circuit breaker is most likely limited by our imagination. A nice one I could think of was to limit functionality when load becomes too high. Typically, this would be the functionality that is “nice to have”, but not essential to your application. To do this, your code would have to inspect the state of the circuit breaker when deciding which options are made available to your user. Do keep in mind though, that limiting functionality might cause your circuit breaker to sit and wait for requests which don’t come, never closing it. You’ll need another mechanism for detecting when it can be closed.

Implementing the circuit breaker

I’ve been playing around with a small circuit breaker implementation. You can find the code in our code repository. It’s really small, but seemed really powerful.

There are 2 main components in the application.

• The interceptor is responsible for intercepting calls to the external system and sending the requests and responses to the circuit breaker [see code]
• The circuit breaker itself decides if requests should be sent and what should happen when responses are received [see code]

To find out if a circuit breaker is actually useful, I’ve done a few load tests. I’ve ran these tests against the same application twice. Once with circuit breaker and once without. The results are promising.

When activated, the circuit breaker will cause failures to append a lot faster than when there is none. This means threads are released much faster, allowing the next unfortunate user to receive the error. Of course, it doesn’t help your user much, but at least you’ll be able to tell him something is wrong instead of giving him a timeout after 30 seconds.

Don’t believe me? Have a look at the screen shots of the meter tests below:

250 concurrent threads with a targeted throughput of 50 requests per second. The circuit breaker variant doesn’t have any failed calls, while the normal implementation has more than 40% failures.

250 concurrent threads with a targeted throughput of 100 requests per second. This is where the circuit breaker variant seems to have it’s first failures.

And finally, a 1000 thread burst, just to see what happens if you spam tomcat with 1000 concurrent, continuously refreshing users. It’s 80% failure against 32% for the circuit breaker.

Conclusion

In this post, I’ve covered two of the stability threats that can be prevented by a correct implementation of the circuit breaker pattern. Our svn code repository contains example code of an implementation of a circuit breaker. And finally, I’ve shown some unit test results to demonstrate the advantage of applying a circuit breaker to your architecture.