navigation

Reactive Java benchmark. Vert.x vs. Spring Framework 5

Reactive Java benchmark. Vert.x vs. Spring Framework 5

by
September 5, 2017

Reactive programming

If you are on a constant learning journey for developing flexible, loosely-coupled and scalable applications, then you definitely at least came across the term “reactive programming”. The concept of non-blocking, event-driven applications which scales with small number of threads is at the heart of this trending paradigm. Huge data processing in a manner of milliseconds with guaranteed high availability force the creation of a new model called back pressure. It is a mechanism which doesn’t allow the producer to overload the consumer – key ingredient in the reactive world.

Reactive Java

Spring Framework 5.0 RC3 has been released recently and as you are guessing this is the version of the popular framework which has adopted the reactive concept. On the other hand Vert.x has already gain lot of thumbs up in the Java community for its capabilities of developing reactive applications on the JVM. Both frameworks apply the architecture style and it’s up to you which one you will choose for your next project.

Purpose

The purpose of this blog is to benchmark the performance of the described frameworks.

Two identical projects (one with Vert.x and the other with Spring Framework 5.0) will handle multiple POST requests on a specified URL. The REST endpoints will accept JSON object and the system will map it to a corresponded Java object. The response from the server will be the newly created Java object but encoded to JSON.

The performance test will be executed with the help of JMeter.

NOTE: The frameworks specifics are not part of the blog’s content. If you want to dig deeper in developing reactive applications with Spring 5 or Vert.x you can use as a reference Twittergithub Apache JMeter™ or Vert.x Documentation.

Preparation

Clone the two projects from here or create them from scratch on your own.

JSON payload

According to the payload which we are going to send to our REST endpoints, we need to create 3 Java POJOs – Event, Team and Stadium. These classes will be common for the both projects.

Event.java

Team.java

Stadium.java

Both projects will have Launcher class which will initialize a Server capable of handling POST requests on a specified URL.

Launcher.java – Vert.x

Launcher.java – Spring 5

Server.java – Vert.x

Server.java – Spring 5

Our Spring5 project requires an additional class with name EventHandler. Its purpose will be to decode the JSON payload to an Event class and after that to encode it back to JSON during the creation of the response.

EventHandler.java

Start the applications one by one and fire up the JMeter test(Benchmark.jmx) from the github repo.

The performance test will simulate:

  1. 100 concurrent users
  2. 50 000 requests

The performance tests which I have executed have been performed on a machine with the following parameters:

OS : Fedora 27
CPU : Intel(R) Quad Core(TM) i5 @ 2.67GHz
Memory : 8GB DDR3

Results

  • Vert.x

Starting the test @ Mon Sep 04 15:04:50 EEST 2017 (1504526690631)
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary + 49725 in 00:00:09 = 5345.0/s Avg: 11 Min: 0 Max: 499 Err: 0 (0.00%) Active: 14 Started: 100 Finished: 86
summary + 275 in 00:00:00 = 4583.3/s Avg: 1 Min: 0 Max: 11 Err: 0 (0.00%) Active: 0 Started: 100 Finished: 100
summary = 50000 in 00:00:09 = 5339.6/s Avg: 11 Min: 0 Max: 499 Err: 0 (0.00%)
Tidying up … @ Mon Sep 04 15:05:00 EEST 2017 (1504526700061)
… end of run

  • Spring 5

Starting the test @ Mon Sep 04 15:08:27 EEST 2017 (1504526907389)
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary + 5298 in 00:00:03 = 2085.0/s Avg: 26 Min: 0 Max: 454 Err: 0 (0.00%) Active: 100 Started: 100 Finished: 0
summary + 44702 in 00:00:10 = 4517.2/s Avg: 15 Min: 0 Max: 535 Err: 0 (0.00%) Active: 0 Started: 100 Finished: 100
summary = 50000 in 00:00:12 = 4020.3/s Avg: 16 Min: 0 Max: 535 Err: 0 (0.00%)
Tidying up … @ Mon Sep 04 15:08:39 EEST 2017 (1504526919897)
… end of run

The results on my machine show that Vert.x can handle 50 000 requests 3 seconds faster than Spring 5.

NOTE: The Vert.x application is build on top of its latest stable version which is 3.4.2, while Spring 5 is in a release candidate phase.

Summary

Our company recently included Vert.x in its technology stack as a main framework for building reactive applications. The demo projects which we created during this blog compare not only the performance, but how a developer can implement identical functionality with different technologies. My personal opinion is that creating reactive applications with Vert.x is much more simpler and understandable than using Spring Framework 5.0.

Don’t get me wrong, I am huge Spring fan and my excitement of how the framework will grow is tremendous. I will be glad if you share your opinion or experience on the topic.

Boyko Dimitrov

Java Developer at Dreamix

More Posts - Website

Follow Me:
TwitterLinkedInGoogle Plus

Do you want more great blogs like this?

Subscribe for Dreamix Blog now!

  • Armen Arzumanyan

    My opinion the same

  • Mile Zajkovski

    It is obvious from the timestamps of the runs that your PC was tired from running the Vert.x and that’s why it run Spring 5 slower. :) Just kidding. I wanted to say thank you for bringing up this interesting technology and comparing it to our beloved Spring.

  • Vadym Mishchenko

    thanks, for article.
    would be interesting to see more variations: different(less,more) requests and different concurrent clients number.

    • Boyko Dimitrov

      Thank you, Vadym! I will try to find some spare time and execute different variations.

  • Igor Nikolaev

    How many cores each framework used in your test? I know that with Vert.X you would need to specify DeploymentOptions.setInstances in order to scale it, since a single verticle only runs on one core.

  • How long did you warm up the tests for? In my experience, you need about 30s warmup before the tests will reach their peak performance.

  • Erwin de Gier

    Two questions: 1) Can you explain the difference? Does Spring 5 use a similar approach as Vert.x with an event loop which is scaled over multiple CPU cores? 2) Why do you think Vert.x is easier than Spring 5. I am a huge Vert.x 5, but the Spring annotations approach to reactive leads to cleaner code.