Converting a jOOQ-Record into a vertx JsonObject

In vertx-jooq, there are a couple of ways to convert a jOOQ Record into a vertx JsonObject. First, checkout the straightforward way, which involves two steps: load the POJO using a generated DAO and then convert it with the toJson method.

It turns out however that on the way to a JsonObject, a lot of copying is going on. First the SELECT statement is mapped to a Record, then converted to a POJO and finally converted into a JsonObject. The code below is copied from DAOImpl class which is extended by every VertxDao:

If you’ve enabled the generation of interfaces in vertx-jooq (which is the default), then you could convert the Record directly into a JsonObject without converting to POJO first.

The major disadvantage of this solution is that you have to write the SELECT statement by yourself. On the other hand, you save the creation of the POJO, which is a plus. But what if we’re joining on another table so we cannot directly map into a generated Record? See this example:

Because the fetched Record is dynamic, we cannot simply call toJson on it. Instead we call the fetchOneMap method which “returns at most one resulting record as a name/value map.” Luckily, JsonObject has a constructor taking a Map<String,Object> which is exactly the same generic type returned (it even returns the same Map implementation that JsonObject uses under the hood: LinkedHashMap). It is noteworthy that this could even be done with pure jOOQ and vertx (leaving the execution aside). Some caveats though:

  1. No conversion happens. If you use a custom generator to generate the schema objects and have overwritten handleCustomTypeToJson / handleCustomTypeFromJson, you’re probably in bad shape. If one of the fields you’re fetching is one of those types, this conversion is bypassed.
  2. Because the constructor makes no checks and takes the map ‘as is’ without copying, you’re probably adding illegal JSON-types.
  3. The Map‘s key will have the name of the database column it is representing. This differs from the default-setting in vertx-jooq, which uses the name of the POJO’s member variable representing that column and thus would produce different JSON compared to the toJson method for the same Record/POJO. Starting with 2.3.5 however this can be changed so the keys are rendered with the same syntax.

Lessons learned

When simple CRUD is not enough, you have to fetch some joined Records and need to convert the results into a JsonObject, you should change the syntax how the JsonObject's keys are rendered in the POJO (3.). This is the first step in making both ways of generating JSON interoperable.

Things get trickier if you have to deal with conversion of custom JSON types. If your app/service is mostly reading data, you could handle conversion already on jOOQ-level so your converted type is already a valid JsonObject type. For example, if you’re using a SQL DateTime field “lastUpdated” on database level, just write a converter that formats the DateTime as a String. In turn, both the generated POJO and the fetchOneMap/fetchMaps methods will return a String for the “lastUpdated” entry and produce same results.

This can become a problem when those converted values need to be altered by the same app: a) it is more convenient and less error-prone to set a java.time.LocalDateTime object instead of dealing with Strings and b) some of the types may have special SQL-functions (e.g. dateAdd) which you cannot use any longer for that type.

Conclusion

Surprisingly there is no golden rule. First, I recommend to change the syntax of how JSON keys are rendered to be the same as the database column names (I will most likely change that behavior to the default in one of the next releases). This means you can use the fetchOneMap/fetchMaps methods most of the time to produce JSON. When dealing with custom types you should check how frequently those are updated by your app. If you’re facing a read-only app, write custom jOOQ converters for these types. If these types are altered quite often, you should stick to their actual type and handle the conversion into a proper JSON value on your own.

Advertisements

vertx-jooq 2.3.5 gives you control over the generated JSON

One of the goodies of vertx-jooq is that it allows you to automatically convert POJOs and jOOQ-Records from and into JSON-Objects. Until now however, the key of the generated JSON-field was fixed to the POJO’s member name that represents that column (which stays the default). Starting with version 2.3.5 this behavior can be changed in different ways:

  1. Subclass the VertxGenerator of your choice and overwrite getJsonKeyName-method.
  2. Subclass VertxGeneratorStrategy of your choice and overwrite getJsonKeyName-method.
  3. Set a different delegate used by the VertxGeneratorStrategy (see also #6) and change the way how the POJO’s member names are rendered.

Imagine you want the JSON keys to be exactly the same as their database column counterparts. Here is how you would do it in all three different methods:

Option 1:

Option 2:

Option 3:

Whatever way you prefer, you also need to alter your code generation configuration and set the correct Generator/GeneratorStrategy. Checkout the github page for a how-to or if you’re new to vertx-jooq.

Hello async my old friend

Let me present you vertx-jooq-async: the world’s first fully async and type-safe SQL code generation and execution tool for VertX™. Async? Wasn’t it asynchronous before? It was, but what the code did was wrapping JDBC-calls using Vertx’ executeBlocking-method, which just shifts the blocking code from one event loop to another (see also).

1r9aly

With this release however, things have changed. jOOQ is still used for generating code and creating type-safe queries, but code is executed utilizing vertx-mysql-postgresql-client. That library in turn is based on a non-blocking driver for MySQL and Postgres databases, so this time it is really non-blocking.

Although some stuff is yet missing – you should go and check out the github-page. Now.

vertx-jooq goes rx

After releasing a new version of your pet-project – what do you expect as a first reaction? Cheer? Love? Joy? No. People will ask for more. Welcome to the internet:

Clement Escoffier, vertx hero and avenger of unresolved github-issues however jumped in and added RX-Java support for vertx-jooq! Thanks to him, there is a new VertxDao that exposes various RX-like CRUD-methods:

This is cool! Also don’t forget to check the github-page for more details.

vertx-jooq 2.2.0 released

Today I’ve released version 2.2.0 of vertx-jooq with the following fixes/changes:

  • Fix: Codegeneration puts files in wrong folders #5
  • Fix: Move jooq-codegen dependency to vertx-jooq-generate #4
  • Enhancement: Create deleteAsync(Condition), fetchOneAsync(Condition) and fetchAsync(Condition)-methods #3

Especially #3 is a nice feature and increases the productivity when working with the auto-generated DAOs. Let’s have a look at the unit-test which tests the new methods:

You can add any org.jooq.Condition to fetch (row 6) or delete (row 8) records without having to write the boilerplate SQL on your own – just let vertx-jooq do the work for you.

Locking Gone Wrong

Sometimes you need to learn the hard way. Imagine the following method in your code that performs a synchronized operation like this:

The intention of the mayLock-method is to do some calculations (represented by the Thread.sleep method) guarded by the input parameter, e.g. some UserID. When you execute the code on a multicore CPU, the output should be something like:

17:14:39.905 [ForkJoinPool.commonPool-worker-2] INFO LockTest – entering method 127
17:14:39.905 [ForkJoinPool.commonPool-worker-1] INFO LockTest – entering method 127
17:14:39.913 [ForkJoinPool.commonPool-worker-2] INFO LockTest – entering lock ForkJoinPool.commonPool-worker-2
17:14:41.915 [ForkJoinPool.commonPool-worker-1] INFO LockTest – entering lock ForkJoinPool.commonPool-worker-1

Based on the timestamp, you can say, that the locking mechanic based on the argument works: about two seconds after the first Thread, the second Thread enters the synchronized section. Now changing the input parameter to 128 results into the following output:

17:20:33.849 [ForkJoinPool.commonPool-worker-1] INFO LockTest – entering method 128
17:20:33.849 [ForkJoinPool.commonPool-worker-2] INFO LockTest – entering method 128
17:20:33.858 [ForkJoinPool.commonPool-worker-1] INFO LockTest – entering lock ForkJoinPool.commonPool-worker-1
17:20:33.858 [ForkJoinPool.commonPool-worker-2] INFO LockTest – entering lock ForkJoinPool.commonPool-worker-2

That’s right. The method is executed right away without waiting for the lock. The reason behind this is Java’s autoboxing and the IntegerCache. Autoboxing is a convenient way of interoperating primitive int‘s and Integer-objects so you can write code like this:

When a primitive int is autoboxed into an Integer-object, the JVM first checks if the target value is within the range of the IntegerCache which is, by default between -127 and 127. That means the following code is valid:

Because 127 is within the bounds of the cache i refers to the same object as j. Let’s change to 128 now:

Because 128 is outside of the cache bounds, a new Integer object is created each time you do autoboxing (same goes for all values greater 127 and lower -127) and is therefore no good candidate for acquiring locks. Although you can increase the cache size as the Javadoc states, this solution does not scale very well.

/**
 * Cache to support the object identity semantics of autoboxing for values between
 * -128 and 127 (inclusive) as required by JLS.
 *
 * The cache is initialized on first usage.  The size of the cache
 * may be controlled by the {@code -XX:AutoBoxCacheMax=} option.
 * During VM initialization, java.lang.Integer.IntegerCache.high property
 * may be set and saved in the private system properties in the
 * sun.misc.VM class.
 */

In addition, you don’t know what parts in your code are locking on that Integer too, especially when you are working with third party libraries.

You might be tempted to switch to Strings, but something similar applies for Strings and the internal StringPool. Locking may work if the String is inside the StringPool but is not guaranteed to. So how to circumvent this?

Following the rule “first make it work – then make it fast“, we could synchronize the whole mayLock-method. Actually, this would work in many environments, but when you’re looking for performance locking on class-level is not an option. So my suggestion here is to create a Map<Integer,Object> which stores an Object you can synchronize on or a Map<Integer,java.util.concurrent.locks.Lock> if you need finer control over the locking process.

Since we’re using a Map with a hashing algorithm, Integer can safely be used as a key. The computeIfAbsent-method of the ConcurrentHashMap guarantees, that it returns the same object even if multiple Threads are accessing the method at the same time. When we run the test again, we see that the lock is correctly used.

However the Map needs to be maintained, e.g. we have to manually remove the key-value pairs once we’re done. Switching to a WeakHashMap that stores it’s keys with a weak reference (and is garbage collected when the key is no longer referenced) is not an option because of two reasons: a) this class is not threadsafe and b) Integers within the range of the IntegerCache will never be removed because they’re “hard” referenced.

As you can see, getting multithreading right is hard. Or simply put

99% of multithreaded Java code is wrong. The other 1% is written by Doug Lea and Brian Goetz.

Configuring A Staged Java Application On AWS ElasticBeanstalk – The Maven Way

In the previous post I’ve described a way how to configure a maven project that takes environment parameters from ElasticBeanstalk to boot the application. However there was one thing that bothered me: I’ve used a shell-script to assemble everything into a deployable zip-file. In this post I show you how to do it only with maven plugins (heavily inspired by this stackoverflow answer).

Here is the new pom:

As you can see, this time I’ve used the maven shade plugin to create a runnable jar. The assembly plugin is used here to pack everything into a zip-file. The configuration happens through an external descriptor in src/assembly/assembly.xml which looks like this:

Procfile and run.sh are located in the project root. To sum it up: each time you invoke mvn clean install or mvn package, maven will create a runnable jar, and then zip it alongside with the Procfile and run.sh. If you want to alter the name of the deployment artifact just change the finalName tag of the assembly plugin’s configuration.

The project is uploaded to Github.

Happy coding!