Month: May 2017

vertx-jooq 2.2.0 released

Today I’ve released version 2.2.0 of vertx-jooq with the following fixes/changes:

  • Fix: Codegeneration puts files in wrong folders #5
  • Fix: Move jooq-codegen dependency to vertx-jooq-generate #4
  • Enhancement: Create deleteAsync(Condition), fetchOneAsync(Condition) and fetchAsync(Condition)-methods #3

Especially #3 is a nice feature and increases the productivity when working with the auto-generated DAOs. Let’s have a look at the unit-test which tests the new methods:


@Test
public void asyncCRUDConditionShouldSucceed() throws InterruptedException {
CountDownLatch latch = new CountDownLatch(1);
CompletableFuture<Integer> insertFuture = dao.insertReturningPrimaryAsync(createSomething());
insertFuture.
thenCompose(v -> dao.fetchOneAsync(Tables.SOMETHING.SOMEID.eq(insertFuture.join()))).
thenAccept(Assert::assertNotNull).
thenCompose(v -> dao.deleteExecAsync(Tables.SOMETHING.SOMEID.eq(insertFuture.join()))).
whenComplete(failOrCountDown(latch));
await(latch);
}

view raw

ConditionInAction.java

hosted with ❤ by GitHub

You can add any org.jooq.Condition to fetch (row 6) or delete (row 8) records without having to write the boilerplate SQL on your own – just let vertx-jooq do the work for you.

Locking Gone Wrong

Sometimes you need to learn the hard way. Imagine the following method in your code that performs a synchronized operation like this:


public class LockTest {
private static final Logger logger = LoggerFactory.getLogger(LockTest.class);
public static void main(String[] args) throws InterruptedException {
ForkJoinPool.commonPool().execute(() -> { mayLock(127);});
ForkJoinPool.commonPool().execute(() -> { mayLock(127);});
ForkJoinPool.commonPool().shutdown();
ForkJoinPool.commonPool().awaitTermination(1, TimeUnit.MINUTES);
}
private static void mayLock(Integer arg) {
logger.info("entering method {} ",arg);
synchronized (arg){
logger.info("entering lock "+Thread.currentThread().getName());
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}

view raw

LockTest.java

hosted with ❤ by GitHub

The intention of the mayLock-method is to do some calculations (represented by the Thread.sleep method) guarded by the input parameter, e.g. some UserID. When you execute the code on a multicore CPU, the output should be something like:

17:14:39.905 [ForkJoinPool.commonPool-worker-2] INFO LockTest – entering method 127
17:14:39.905 [ForkJoinPool.commonPool-worker-1] INFO LockTest – entering method 127
17:14:39.913 [ForkJoinPool.commonPool-worker-2] INFO LockTest – entering lock ForkJoinPool.commonPool-worker-2
17:14:41.915 [ForkJoinPool.commonPool-worker-1] INFO LockTest – entering lock ForkJoinPool.commonPool-worker-1

Based on the timestamp, you can say, that the locking mechanic based on the argument works: about two seconds after the first Thread, the second Thread enters the synchronized section. Now changing the input parameter to 128 results into the following output:

17:20:33.849 [ForkJoinPool.commonPool-worker-1] INFO LockTest – entering method 128
17:20:33.849 [ForkJoinPool.commonPool-worker-2] INFO LockTest – entering method 128
17:20:33.858 [ForkJoinPool.commonPool-worker-1] INFO LockTest – entering lock ForkJoinPool.commonPool-worker-1
17:20:33.858 [ForkJoinPool.commonPool-worker-2] INFO LockTest – entering lock ForkJoinPool.commonPool-worker-2

That’s right. The method is executed right away without waiting for the lock. The reason behind this is Java’s autoboxing and the IntegerCache. Autoboxing is a convenient way of interoperating primitive int‘s and Integer-objects so you can write code like this:


Integer i = 23;
int j = i;

view raw

Autoboxing.java

hosted with ❤ by GitHub

When a primitive int is autoboxed into an Integer-object, the JVM first checks if the target value is within the range of the IntegerCache which is, by default between -127 and 127. That means the following code is valid:


Integer i = 127;
Integer j = 127;
boolean same = (i == j); //true
boolean equal = i.equals(j); //true

view raw

Intcache1.java

hosted with ❤ by GitHub

Because 127 is within the bounds of the cache i refers to the same object as j. Let’s change to 128 now:


Integer i = 128;
Integer j = 128;
boolean same = (i == j); //false
boolean equal = i.equals(j); //true

view raw

Intcache2.java

hosted with ❤ by GitHub

Because 128 is outside of the cache bounds, a new Integer object is created each time you do autoboxing (same goes for all values greater 127 and lower -127) and is therefore no good candidate for acquiring locks. Although you can increase the cache size as the Javadoc states, this solution does not scale very well.

/**
 * Cache to support the object identity semantics of autoboxing for values between
 * -128 and 127 (inclusive) as required by JLS.
 *
 * The cache is initialized on first usage.  The size of the cache
 * may be controlled by the {@code -XX:AutoBoxCacheMax=} option.
 * During VM initialization, java.lang.Integer.IntegerCache.high property
 * may be set and saved in the private system properties in the
 * sun.misc.VM class.
 */

In addition, you don’t know what parts in your code are locking on that Integer too, especially when you are working with third party libraries.

You might be tempted to switch to Strings, but something similar applies for Strings and the internal StringPool. Locking may work if the String is inside the StringPool but is not guaranteed to. So how to circumvent this?

Following the rule “first make it work – then make it fast“, we could synchronize the whole mayLock-method. Actually, this would work in many environments, but when you’re looking for performance locking on class-level is not an option. So my suggestion here is to create a Map<Integer,Object> which stores an Object you can synchronize on or a Map<Integer,java.util.concurrent.locks.Lock> if you need finer control over the locking process.


private static Map<Integer,Object> LOCKS = new ConcurrentHashMap<>();
private static void mayLock(Integer arg) {
logger.info("entering method {} ", arg);
synchronized (LOCKS.computeIfAbsent(arg,f->new Object())){
logger.info("entering lock "+Thread.currentThread().getName());
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

view raw

LockTest2.java

hosted with ❤ by GitHub

Since we’re using a Map with a hashing algorithm, Integer can safely be used as a key. The computeIfAbsent-method of the ConcurrentHashMap guarantees, that it returns the same object even if multiple Threads are accessing the method at the same time. When we run the test again, we see that the lock is correctly used.

However the Map needs to be maintained, e.g. we have to manually remove the key-value pairs once we’re done. Switching to a WeakHashMap that stores it’s keys with a weak reference (and is garbage collected when the key is no longer referenced) is not an option because of two reasons: a) this class is not threadsafe and b) Integers within the range of the IntegerCache will never be removed because they’re “hard” referenced.

As you can see, getting multithreading right is hard. Or simply put

99% of multithreaded Java code is wrong. The other 1% is written by Doug Lea and Brian Goetz.

Configuring A Staged Java Application On AWS ElasticBeanstalk – The Maven Way

In the previous post I’ve described a way how to configure a maven project that takes environment parameters from ElasticBeanstalk to boot the application. However there was one thing that bothered me: I’ve used a shell-script to assemble everything into a deployable zip-file. In this post I show you how to do it only with maven plugins (heavily inspired by this stackoverflow answer).

Here is the new pom:


<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0&quot;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"&gt;
<modelVersion>4.0.0</modelVersion>
<groupId>io.github.jklingsporn.eb</groupId>
<artifactId>example</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<finalName>app</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<extensions>true</extensions>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<!– use shade plugin to create runnable jar –>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>io.github.jklingsporn.eb.Main</mainClass>
</transformer>
</transformers>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
<!– use assembly plugin to create zip –>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.6</version>
<configuration>
<descriptors>
<descriptor>src/assembly/assembly.xml</descriptor>
</descriptors>
<appendAssemblyId>false</appendAssemblyId>
<finalName>app-${maven.build.timestamp}</finalName>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

view raw

pom.xml

hosted with ❤ by GitHub

As you can see, this time I’ve used the maven shade plugin to create a runnable jar. The assembly plugin is used here to pack everything into a zip-file. The configuration happens through an external descriptor in src/assembly/assembly.xml which looks like this:


<?xml version="1.0" encoding="utf-8"?>
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3&quot;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3 http://maven.apache.org/xsd/assembly-1.1.3.xsd"&gt;
<id>release</id>
<formats>
<format>zip</format>
</formats>
<fileSets>
<fileSet>
<directory>target</directory>
<includes>
<include>${build.finalName}.jar</include>
</includes>
<outputDirectory>/</outputDirectory>
</fileSet>
</fileSets>
<files>
<file>
<source>Procfile</source>
<fileMode>0700</fileMode>
</file>
<file>
<source>run.sh</source>
<fileMode>0700</fileMode>
</file>
</files>
</assembly>

view raw

assembly.xml

hosted with ❤ by GitHub

Procfile and run.sh are located in the project root. To sum it up: each time you invoke mvn clean install or mvn package, maven will create a runnable jar, and then zip it alongside with the Procfile and run.sh. If you want to alter the name of the deployment artifact just change the finalName tag of the assembly plugin’s configuration.

The project is uploaded to Github.

Happy coding!

Configuring A Staged Java Application On AWS ElasticBeanstalk

Update

I’ve just blogged about a maven only solution.

In this blog post I’ll describe how to deploy and setup a maven driven Java application on AWS ElasticBeanstalk with different staging profiles. To understand what’s going on, you should have knowledge about AWS and how to setup an ElasticBeanstalk environment.

I have setup this github-repository which contains a very basic maven project. Let’s have a look at it’s pom:


<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0&quot;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"&gt;
<modelVersion>4.0.0</modelVersion>
<groupId>io.github.jklingsporn.eb</groupId>
<artifactId>example</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<extensions>true</extensions>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.6</version>
<configuration>
<finalName>app</finalName>
<archive>
<manifest>
<mainClass>io.github.jklingsporn.eb.Main</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

view raw

pom.xml

hosted with ❤ by GitHub

No big surprises here: we set the compiler level to Java 8 and tell maven to build an executable fat-jar called app.jar during package-phase. The main itself just prints the passed arguments in the console as shown below:


public class Main {
public static void main(String[] args) {
System.out.println(String.format("Look at these wonderful arguments %s", Arrays.toString(args)));
}
}

view raw

Main.java

hosted with ❤ by GitHub

Executing mvn clean install should generate the class files and a runnable jar file in the target-folder. One could take this file and upload it to ElasticBeanstalk. We’re done.

Just kidding.

Two problems may arise:

  1. ElasticBeanstalk will reject the file the next time you upload it, because it already has a file called app.jar.
  2. The application wouldn’t print any arguments, because you haven’t passed any to it.

Solving the first problem is quite easy: you can just append the current timestamp (or project version) to the jar. Either manually, or by changing the finalName-tag to something like

app-${project.version}

or

app-${maven.build.timestamp}

However this doesn’t solve the second problem. To tackle this, one must understand how the application is started after it was deployed. On the EC2-instance, next to the uploaded jar-file there is a file called Procfile, which is used to start the jar. By default, it consists of a one-liner:

web: java -jar app.jar

The only thing you need to know regarding the first word web is:

The command that runs the main JAR in your application must be called web, and it must be the first command listed in your Procfile.

Good news is, that a user can upload it’s own Procfile which leads us to the question: where to put the arguments, especially if you have different arguments for each staging profile. At first sight, a good fit are maven-profiles, but that solution has two major drawbacks: secret informations like passwords should not be stored in your pom. And secondly, each time you change the arguments, you have to redeploy the whole application.

A much better approach is to store that kind of data on the instance itself using ElasticBeanstalk’s configuration. To do so, go to your environment, select the application you are using (or create a new one based on Java SE-Platform), click on Configuration, Software Configuration and create two new environment variables: JAVA_OPTS and JAVA_ARGS. Set the value of JAVA_OPTS to something like -Dfile.encoding=utf8 and JAVA_ARGS to my_staging_property. The values are now available on the instance – time to create our own Procfile. To do so, I’ve created a bash-script (Windows users look here) like this:


#!/bin/bash
set -e
now=$(date +"%Y-%m-%d_%H-%M-%S")
tag="./target/app-$now.zip"
mvn clean install
echo "web: ./run.sh" > target/Procfile
echo "exec java \$JAVA_OPTS -jar app.jar \$JAVA_ARGS" > target/run.sh
chmod +x target/run.sh
zip -j ${tag} target/Procfile target/run.sh target/app.jar

view raw

build.sh

hosted with ❤ by GitHub

  • Lines 3 and 4: I decided to append the timestamp to the build artifact to fix the redeploy issue.
  • Line 6, 7 and 8: Our Procfile just executes another shell script, called run.sh. That script uses the environment variables defined above to start the application.
  • Line 9: Zip everything into one file.

So instead of executing maven directly, run ./build.sh from the terminal. This will create a file with a name like app-2017-05-03_14-50-07.zip in the target/ – folder of your maven project which can then be deployed to any staging environment. Configuration is solely done by ElasticBeanstalk. E.g. set JAVA_OPTS to -Xmx256M on your dev-environment and -Xmx16G on production. Changing some parameters only requires to update your EB-configuration (e.g. tune some GC-settings during a load test) instead of redeploy everything.