In my time as a Java-programmer I never thought (thoroughly) about the meaning of InterruptedException, although one is getting in touch with it very early. To be honest in the last week I had to deal with that exception because it used to happen at work: whenever I restarted one of our server that exception pops up once in a while (this is one of the errors that never happens on your local (or remote) test-environment but on live systems under load but thats another story ;-)). The exception was thrown by a low-level database-connection-pooling-library called c3p0 which meant we lost some data. Luckily this isn’t a matter of life and death because I’m working in the gaming-industry, but the results aren’t that good also: data-loss, inconsistency and perhaps broken user-accounts.
To understand why that exception has been thrown I have to give you a small glimpse into our system-architecture. We have two layers, each represented by a ThreadPool. The network- and gamelogic-layer (layer 1) decodes messages and executes them and the database-layer (layer 2) creates, updates and deletes entities as requested by layer 1. Additionaly, for each player-representation there exists a database-manager-class which collects all requested entity changes by layer 1 and sends them every X seconds to the database. However, when a player logs out or there is a connection-loss on client-side we force this database-manager to commit the changes immediately (because he may re-log-in instantly and would might get stale data then). Look at the following code-snippet taken from that database-manager:
@Override
public boolean await(boolean instant, long maxTimeoutMs)
throws RamaDatabaseAccessException {
if (!isScheduled()) {
return true;
}
if (instant) {
cancelAndExecute();
return false;
} else {
/*
* Depending on the timeout wait for result or call cancelAndExecute().
*/
...
}
}
@Override
public Object call(){
/*
* Invoke some CRUD-operations depending on the scheduled transactions.
*/
...
}
When the player logs out, the await()-method is called having the instant-flag set which means the scheduled execution of the database-manager should be canceled and instantly executed(invocation of cancelAndExecute()). I will show you two possible solutions of how to implement this method and then discuss both of them.
Solution A:
private void cancelAndExecute() throws Exception{
if (this.future.cancel(false)) {
this.call();
}
}
Solution B:
private void cancelAndExecute() throws Exception{
if (this.future.cancel(false)) {
this.getExecutor().submit(this).get();
}
}
Both solutions have in common that they cancel the scheduled operation and then execute it immediately. Solution A executes the operation in the current Thread and the second solution delegates the operation to an ExecutorService. Obviously A breaks the architecture because the database-operation is executed by the gamelogic-layer. However this was the solution I preferred because only one Thread is involved in that case, the caller Thread. Solution B on the other hand blocks a Thread from the ExecutorService and the caller Thread (because of calling get()).
As I mentioned in the beginning, the InterruptedException was thrown while the caller Thread was waiting for a database connection and the server was shutting down. While the shutdown-process is running, our programm starts shutting down the network-layer to stop comunication with the game-clients. In that process the releaseExternalResources()-method from the external network-framework netty is called which itselfs invokes shutdownNow() on its associated ExecutorService. shutdownNow() then invokes Thread.interrupt() on all it’s worker Threads.
So how did the two solutions behave in that scenario? Solution A will throw an InterruptedException when it cannot acquire a database-connection instantly (which happens under load). The database-operation will be interrupted and not be performed. Solution B however will also throw an InterruptedException but with minor impact: B is waiting for another Thread that does the job whereas A does the job itself. The critical task itself will still be executed (because it’s executed by another Thread) and only the process of waiting for the completion of that task (calling get() on the Future in this case) will be interrupted.
When catching an InterruptedException you should restore the interrupted-state of the Thread as suggested by this site, so I finally came to the following solution:
Solution C:
private void cancelAndExecute() throws Exception{
if (this.future.cancel(false)) {
try {
this.getExecutor().submit(this).get();
} catch(InterruptedException i){
/*
* May be thrown when netty shuts down. In this case just return and
* don't wait for the result. (Pending tasks were finished later
* in the shutdown-hook).
*/
Thread.currentThread().interrupt();
logger.info("TransactionManager interrupted while waiting for result. Return without waiting.");
}
}
}
Thanks for reading!
jensonjava 