vertx-jooq-async 0.4 released!

vertx-jooq-async the (un)beloved child of vertx-jooq has been released in version 0.4! This version finally implements insertReturningPrimaryAsync in all VertxDAOs*.

In addition, the complex code in the DAOs default methods has been abstracted and moved to vertx-jooq-async-shared. That means less code duplication and fewer points of failure – a move that will very likely be done in vertx-jooq soon as well.

Finally the vertx-dependency version has been bumped to 3.5. (and so the support for RXJava2).

* Unfortunately, this only works for MySQL and numeric auto-increment keys. That is, because the implementation is based on the implementations of io.vertx.ext.asyncsql.impl.AsyncSQLConnectionImpl and only the MySQL variant returns the generated ID.

Advertisements

vertx-jooq 2.4 released

Note: Unfortunately, the dependencies to rxjava1 haven’t been removed completely in 2.4.0. RX users please checkout version 2.4.1.

I’m happy to announce the release of vertx-jooq 2.4. The main driver behind this release was the dependency upgrade to vertx 3.5 and thus the change of the rx-dependency in vertx-jooq-rx from rxjava to rxjava2. In addition, the JSON key names of the generated POJO are now the same as their database counterparts. This change has been announced here. Last but not least, a bug has been fixed when dealing with JsonObjectConverter or JsonArrayConverter.

Converting a jOOQ-Record into a vertx JsonObject

In vertx-jooq, there are a couple of ways to convert a jOOQ Record into a vertx JsonObject. First, checkout the straightforward way, which involves two steps: load the POJO using a generated DAO and then convert it with the toJson method.

It turns out however that on the way to a JsonObject, a lot of copying is going on. First the SELECT statement is mapped to a Record, then converted to a POJO and finally converted into a JsonObject. The code below is copied from DAOImpl class which is extended by every VertxDao:

If you’ve enabled the generation of interfaces in vertx-jooq (which is the default), then you could convert the Record directly into a JsonObject without converting to POJO first.

The major disadvantage of this solution is that you have to write the SELECT statement by yourself. On the other hand, you save the creation of the POJO, which is a plus. But what if we’re joining on another table so we cannot directly map into a generated Record? See this example:

Because the fetched Record is dynamic, we cannot simply call toJson on it. Instead we call the fetchOneMap method which “returns at most one resulting record as a name/value map.” Luckily, JsonObject has a constructor taking a Map<String,Object> which is exactly the same generic type returned (it even returns the same Map implementation that JsonObject uses under the hood: LinkedHashMap). It is noteworthy that this could even be done with pure jOOQ and vertx (leaving the execution aside). Some caveats though:

  1. No conversion happens. If you use a custom generator to generate the schema objects and have overwritten handleCustomTypeToJson / handleCustomTypeFromJson, you’re probably in bad shape. If one of the fields you’re fetching is one of those types, this conversion is bypassed.
  2. Because the constructor makes no checks and takes the map ‘as is’ without copying, you’re probably adding illegal JSON-types.
  3. The Map‘s key will have the name of the database column it is representing. This differs from the default-setting in vertx-jooq, which uses the name of the POJO’s member variable representing that column and thus would produce different JSON compared to the toJson method for the same Record/POJO. Starting with 2.3.5 however this can be changed so the keys are rendered with the same syntax.

Lessons learned

When simple CRUD is not enough, you have to fetch some joined Records and need to convert the results into a JsonObject, you should change the syntax how the JsonObject's keys are rendered in the POJO (3.). This is the first step in making both ways of generating JSON interoperable.

Things get trickier if you have to deal with conversion of custom JSON types. If your app/service is mostly reading data, you could handle conversion already on jOOQ-level so your converted type is already a valid JsonObject type. For example, if you’re using a SQL DateTime field “lastUpdated” on database level, just write a converter that formats the DateTime as a String. In turn, both the generated POJO and the fetchOneMap/fetchMaps methods will return a String for the “lastUpdated” entry and produce same results.

This can become a problem when those converted values need to be altered by the same app: a) it is more convenient and less error-prone to set a java.time.LocalDateTime object instead of dealing with Strings and b) some of the types may have special SQL-functions (e.g. dateAdd) which you cannot use any longer for that type.

Conclusion

Surprisingly there is no golden rule. First, I recommend to change the syntax of how JSON keys are rendered to be the same as the database column names (I will most likely change that behavior to the default in one of the next releases). This means you can use the fetchOneMap/fetchMaps methods most of the time to produce JSON. When dealing with custom types you should check how frequently those are updated by your app. If you’re facing a read-only app, write custom jOOQ converters for these types. If these types are altered quite often, you should stick to their actual type and handle the conversion into a proper JSON value on your own.

Hello async my old friend

Let me present you vertx-jooq-async: the world’s first fully async and type-safe SQL code generation and execution tool for VertX™. Async? Wasn’t it asynchronous before? It was, but what the code did was wrapping JDBC-calls using Vertx’ executeBlocking-method, which just shifts the blocking code from one event loop to another (see also).

1r9aly

With this release however, things have changed. jOOQ is still used for generating code and creating type-safe queries, but code is executed utilizing vertx-mysql-postgresql-client. That library in turn is based on a non-blocking driver for MySQL and Postgres databases, so this time it is really non-blocking.

Although some stuff is yet missing – you should go and check out the github-page. Now.

The Future Is Here

Getting asynchronous operations right is hard. And when you have to pass a result of one function to the next, it can only get worse. Since today, vertx-jooq’s API for instance only allowed asynchronous operations with a callback handler, which leads to code fragments like this:

What annoys me about this is that I have to define in each handler if it succeeded and what to do if an exception occurred on the database layer. Especially when you have to nest three or more operations this looks ugly. Of course this problem is not new and there exists an alternative approach which is using and composing Java 8 java.util.concurrent.CompletableFuture. By doing so, the same code becomes easier and more readable:

But using CompletableFuture within the Vertx world leads to a problem: Vertx has it’s own threading model to achieve the performance it actually has. On the other hand, some methods of CompletableFuture, e.g. CompletableFuture.supplyAsync(Supplier), run tasks on the common ForkJoinPool which would break the Vertx contract. Open-source-software to the rescue, there is a solution to this problem: VertxCompletableFuture. This special implementation guarantees that async operations run on a Vertx context unless you explicitly specify an Executor in one of the overloaded xyzAsync-methods*.

And here comes even better news: starting from version 2, vertx-jooq also supports this way of dealing with asynchronous database operations by utilizing VertxCompletableFuture. Checkout the new vertx-jooq-future module and the according code generator to create your CompletableFuture-based DAOs.

* There have been discussions in the Vertx developer group about a CompletableFuture based API, e.g. here and especially here. The current status is that they do not provide such API officially, mostly because VertxCompletableFuture breaks the contract of the supplyAsync-methods, since it runs within the Vertx context and not the ForkJoinPool. Also when you pass this CompletableFuture subclass to code that expects a regular CompletableFuture, it breaks the Liskov substitution principle and OOP (thanks for pointing that out in the comments Julien). My opinion is, that if you are using Vertx you are aware of the special threading model and can tolerate that behavior. But, of course, it’s up to you.

Vertx loves jOOQ

I’ve recently published a library that connects two frameworks I used a lot in the past: Vertx and jOOQ. Vertx is a “reactive tool-kit for the JVM” that enables you to write non-blocking code and has some nice features built in, like webserver/socketserver-implementation, a message-bus that can run on one or more instances in one network and many more goodies you should check out. On the other hand, if you are coding in Java and you like SQL, there is no way around jOOQ. In my opinion, jOOQ’s two killerfeatures are the possibility to write typesafe SQL and the awesome code-generator that generates POJOs, DAOs and Table-implementations based on your database schema (schema first!). However jOOQ is using JDBC under the hood which blocks the calling thread until the database operation completes.

That is where my library hooks in: it provides a code-generator that adds non-blocking CRUD-methods to all generated DAOs and converter methods that allow you to convert from Vertx’ JsonObject into jOOQ’s POJOs and vice versa.

For code examples, please refer to the github-page.