Useful eclipse templates

When working as a coder, you have to now your tools. And the tool you’re using most is the IDE of your choice. Most Java-programmers choose Eclipse as their development-platform, because it is free and powerful (I remember the days when JBuilder 2.0 was state-of-the-art – I don’t miss them). Facing your daily business, you’ll often find yourself write the same (boilerplate) code-snippets:

  • getters and setters
  • equals and hashcode
  • make a class a Singleton
  • create a local logger-instance
  • etc.

As a coder you should hesitate when writing them repetetive and ask yourself: is there a way to automatize it? I mean automatization is one of the main reason why computers were invented and the programs you’re writing are doing the same things repetetive (and fast). Some of those tasks can be automatized with build-in Eclipse-features (Source – menu), but others not. At least by default.

Eclipse has a cool feature, calling “Code Templates” which ist accessible via Window -> Preferences. “Code Templates” allow you to store code-snippets and substitute the code with a meaningful name. Let’s start with a simple example, e.g. having a shortcut to Java’s currenttime:

  1. Press the New… – Button.
  2. Enter a meaningful name like _currenttime (I prefer adding underscore to the name, to group all my templates).
  3. Add a description if you like.
  4. Write System.currentTimeMillis(); into Pattern-Textbox.
  5. Press OK.

When you’re writing code the templates are part of the auto-completion-feature which is accessible pressing CTRL+SPACE. In future, when you’d like to know the system’s current time, just write _cur and press CTRL + SPACE and then ENTER. Eclipse will replace that with the pattern you’ve written into the template. Here are some useful Code Templates:

Description: Creates the singleton boilerplate-code.

Pattern:


private static ${enclosing_type} instance = new ${enclosing_type}();

private ${enclosing_type}(){}

public static ${enclosing_type} getInstance(){
return instance;
}

Description: Creates a static SLF4J-Logger-instance in current class.

Pattern:

${:import(org.slf4j.Logger,org.slf4j.LoggerFactory)}
private static final Logger logger = LoggerFactory.getLogger(${enclosing_type}.class);

Description: Creates a try-finally block for locking-purposes.

Pattern:

try{
lock.lock();${cursor}
}finally{
lock.unlock();
}

Stackoverflow has many suggestions on this topic. What are your favorite templates?

Advertisements

Dealing with InterruptedException

In my time as a Java-programmer I never thought (thoroughly) about the meaning of InterruptedException, although one is getting in touch with it very early. To be honest in the last week I had to deal with that exception because it used to happen at work: whenever I restarted one of our server that exception pops up once in a while (this is one of the errors that never happens on your local (or remote) test-environment but on live systems under load but thats another story ;-)). The exception was thrown by a low-level database-connection-pooling-library called c3p0 which meant we lost some data. Luckily this isn’t a matter of life and death because I’m working in the gaming-industry, but the results aren’t that good also: data-loss, inconsistency and perhaps broken user-accounts.

To understand why that exception has been thrown I have to give you a small glimpse into our system-architecture. We have two layers, each represented by a ThreadPool. The network- and gamelogic-layer (layer 1) decodes messages and executes them and the database-layer (layer 2) creates, updates and deletes entities as requested by layer 1. Additionaly, for each player-representation there exists a database-manager-class which collects all requested entity changes by layer 1 and sends them every X seconds to the database. However, when a player logs out or there is a connection-loss on client-side we force this database-manager to commit the changes immediately (because he may re-log-in instantly and would might get stale data then). Look at the following code-snippet taken from that database-manager:

@Override
public boolean await(boolean instant, long maxTimeoutMs)
throws RamaDatabaseAccessException {
if (!isScheduled()) {
return true;
}
if (instant) {
cancelAndExecute();
return false;
} else {
/*
* Depending on the timeout wait for result or call cancelAndExecute().
*/
...
}
}

@Override
public Object call(){
/*
* Invoke some CRUD-operations depending on the scheduled transactions.
*/
...
}

When the player logs out, the await()-method is called having the instant-flag set which means the scheduled execution of the database-manager should be canceled and instantly executed(invocation of cancelAndExecute()). I will show you two possible solutions of how to implement this method and then discuss both of them.

Solution A:

private void cancelAndExecute() throws Exception{
if (this.future.cancel(false)) {
this.call();
}
}

Solution B:

private void cancelAndExecute() throws Exception{
if (this.future.cancel(false)) {
this.getExecutor().submit(this).get();
}
}

Both solutions have in common that they cancel the scheduled operation and then execute it immediately. Solution A executes the operation in the current Thread and the second solution delegates the operation to an ExecutorService. Obviously A breaks the architecture because the database-operation is executed by the gamelogic-layer. However this was the solution I preferred because only one Thread is involved in that case, the caller Thread. Solution B on the other hand blocks a Thread from the ExecutorService and the caller Thread (because of calling get()).

As I mentioned in the beginning, the InterruptedException was thrown while the caller Thread was waiting for a database connection and the server was shutting down. While the shutdown-process is running, our programm starts shutting down the network-layer to stop comunication with the game-clients. In that process the releaseExternalResources()-method from the external network-framework netty is called which itselfs invokes shutdownNow() on its associated ExecutorService. shutdownNow() then invokes Thread.interrupt() on all it’s worker Threads.

So how did the two solutions behave in that scenario? Solution A will throw an InterruptedException when it cannot acquire a database-connection instantly (which happens under load). The database-operation will be interrupted and not be performed. Solution B however will also throw an InterruptedException but with minor impact: B is waiting for another Thread that does the job whereas A does the job itself. The critical task itself will still be executed (because it’s executed by another Thread) and only the process of waiting for the completion of that task (calling get() on the Future in this case) will be interrupted.

When catching an InterruptedException you should restore the interrupted-state of the Thread as suggested by this site, so I finally came to the following solution:

Solution C:


private void cancelAndExecute() throws Exception{
if (this.future.cancel(false)) {
try {
this.getExecutor().submit(this).get();
} catch(InterruptedException i){
/*
* May be thrown when netty shuts down. In this case just return and
* don't wait for the result. (Pending tasks were finished later
* in the shutdown-hook).
*/
Thread.currentThread().interrupt();
logger.info("TransactionManager interrupted while waiting for result. Return without waiting.");
}
}
}

Thanks for reading!

Definition of done

Remember the following scenario: you have to develop feature X, which is the part of system Z. It works properly and your product owner gives his/her “ok” to it. But as time goes by there are new features that have to be implemented into Z, say feature Y. This feature depends on X and after you implemented it, you have to work on X again. Now you must explain your product manager and – with some bad luck – your product owner, why you have to work on a feature which was previously labeled as “done”. They might ask you why you have to touch a feature, which they thought you would never have to touch again.

The first thing that might come to their minds is to blame you, the programmer. You didn’t work clean and with too much coupling in your code. Honestly, this was the first thought I had while thinking about this context. However on second thought, I realized that systems got more and more complex. You almost never can ensure that a feature, once implemented, works until the end of time without being touched again (to be honest, you can strike the word “almost”). This shouldn’t sound like an excuse to lame programmers which haven’t learned how to decouple objects and systems. To be clear: it is still a bad sign for your codingskills when you have to change classes across the whole system when a new feature has to be implemented – but sometimes you have no other choice.

So what to answer on the question from above? There are two possibilities:
–    Never, ever commit to the phrase “this feature is done” without the appendix “at the current time and with the current scope”. PM and PO won’t like to hear that but anything else is just lying.
–    Option B is that your code really sucks and you made a mistake while implementing feature X. It is time to be honest: admit that you made a fault and get yourself a copy of a book about design-patterns and system-architecture.

Happy coding!

Why java.util.ScheduledThreadPoolExecutor violates the Open-Close-Principle

In my last post I described the concept of the NotifyingExecutorService. In short, it is a ThreadPoolExecutor (TPE)-extension that returns Future-objects one can listen to. I thought it was a good idea to adopt that pattern to java’s ScheduledThreadPoolExecutor (STPE) which allows the execution of delayed and periodic tasks. However, I failed hard.

The STPE extends the normal ThreadPoolExecutor and adds several schedule()-methods to execute tasks in the future and/or allow repetition of tasks with a fixed frequence. Whenever a schedule-method gets called it creates a new ScheduledFutureTask – object that holds the incoming task and some timing parameters. The method then calls decorateTask() and adds the resulting ScheduledFutureTask to the work queue:

public  ScheduledFuture schedule(Callable callable, long delay, TimeUnit unit) {
        if (callable == null || unit == null)
            throw new NullPointerException();
        if (delay < 0) delay = 0;
        long triggerTime = now() + unit.toNanos(delay);
        RunnableScheduledFuture t = decorateTask(callable,
            new ScheduledFutureTask(callable, triggerTime));
        delayedExecute(t);
        return t;
    }

protected  RunnableScheduledFuture decorateTask(
        Callable callable, RunnableScheduledFuture task) {
        return task;
    }

As you can see, decorateTask() returns the incoming task by default. Javadoc and I thought this was a good place to install the NotifyingFuture-concept:

  • Decorate the incoming task.
  • Add register-methods.
  • Notify listener when certain methods are called.

So I created a class that decorates the original task and overwrites the run()-method in the following manner:

@Override
    public void run() {
    	/*
    	 * execute task
    	 */
        this.decorate.run();
        /*
         * notify listener
         */
        notifyListener();
    }

Sadly, this approach works only for one-shot-executions. Periodic tasks won’t notify their listener and the reason lies in the nature of the decorator pattern that is used by the STPE. See the following snippet of the delegated ScheduledFutureTask (which is a private class inside the ScheduledThreadPoolExecutor):

/**
         * Runs a periodic task.
         */
        private void runPeriodic() {
            boolean ok = ScheduledFutureTask.super.runAndReset();
            boolean down = isShutdown();
            // Reschedule if not cancelled and not shutdown or policy allows
            if (ok && (!down ||
                       (getContinueExistingPeriodicTasksAfterShutdownPolicy() &&
                        !isStopped()))) {
                long p = period;
                if (p > 0)
                    time += p;
                else
                    time = now() - p;
                ScheduledThreadPoolExecutor.super.getQueue().add(this);
            }
            // This might have been the final executed delayed
            // task.  Wake up threads to check.
            else if (down)
                interruptIdleWorkers();
        }

        /**
         * Overrides FutureTask version so as to reset/requeue if periodic.
         */
        public void run() {
            if (isPeriodic())
                runPeriodic();
            else
                ScheduledFutureTask.super.run();
        }

When the task is periodic, runPeriodic() is called which is the root of all evil. Can you spot the pitfall? If not, look closer to line 16.
When the STPE is not yet shut down, the task adds itself to the queue. Remeber that we decorated that task, we didn’t subclass it. That means although the initial task contained our NotifyingFutureTask-implementation, whenever a task is periodic it get’s rid of it’s enclosing deocrator on the second execution.

Now tell me: how bad is that? Why has Sun chosen to use two different approaches (newTaskFor vs decorateTask) inside one class? Who knows the answer?

Notifying ExecutorService (2)

This is the second post about the NotifyingExecutorService. Don’t miss the first half!

The ThreadPoolExecutor in the java.util.concurrency package has several methods for submitting tasks. Each method wraps the incoming Runnable (or Callable) to a FutureTask containing that task and the future result by forwarding the submit()-methods  to the protected method newTaskFor(). The FutureTask itself has also some interesting methods to observe the state of the task (isDone(), iscancelled() for instance). Additionally the done()-method gets called when the task has been processed by the TPE – despite of whether the computation of the task failed or completed. By default, this method does nothing and is therefore intended to be overridden. This is a good entrypoint for the listener-modell described above:

1.    Create a subclass of FutureTask, override done() and add a method to register a listener to the task. Notify listener (if registered) when done() gets called.
2.    Create a subclass of ThreadPoolExecutor and return the subclassed FutureTask whenever newTaskFor() gets called.
3.    Return the subclassed FutureTask on every submit()-method instead of a plain Future to allow registration of listeners to a task.

That’s almost all. Everything is cool, isn’t it? Unfortunately no, because of two major problems that one has to be aware of:
1.    We have to ensure that the  listener gets notified although the task has been already executed. Remember that submitting tasks to the TPE is an asynchronous operation. The caller-thread adds the FutureTask to a queue and returns immediately. Meanwhile pool-threads are working on the queue. There is no guarantee what happens first: adding a listener to the task (by caller thread) or computing the result of the task (by pool-thread). Which leads us directly to the second problem:
2.    The thread that notifies the listeners is not the caller-thread it’s one of the pool-threads. That means that every computation that is made by the listener is done by a pool-thread. This is very bad, because a TPE is normally added to achieve decoupling. We would break the design when long running tasks are implied by the listener’s callback-method. Remember the game-example from above:

•    New Gameobject is created by gamelogic. (Thread A)
•    A “persist task” is added to our database-TPE which persists the object into the database (and assigns a new id to the object).  (Pool Thread B)
•    We register a callback-listener to the FutureTask returned by TPE. When notified, the listener sends the new Gameobject to the connected clients. (Thread A)
•    Database-TPE executes task and notifies listener. The messages are send to clients. (Pool Thread B)

The last operation should not be done by a pool-thread, because the pool should only execute database-tasks. The solution of the problem is to define a TPE that executes the callback-tasks when the FutureTask is done, simply by adding a TPE-parameter to the addListener()-method. Sadly this solution has a tiny disadvantage: passing a task to a TPE comes with some performance tradeoffs. When the listener-callbacks aren’t expensive you might consider executing the callback in the pool-thread. Google’s guava-library helps us in this case: use their MoreExecutors.sameThreadExecutor() which executes all submitted tasks in the caller thread.

Finallay, here are the classes:

public interface INotifyingExecutorService extends ExecutorService{

    @Override
    public <T> INotifyingFuture<T> submit(Callable<T> task);
    
    @Override
    public INotifyingFuture<Void> submit(Runnable task);
    
    @Override
    public <T> INotifyingFuture<T> submit(Runnable task, T result);
    
}

As you can see, this interface just extends the ExecutorService to enforce that an INotifyingFuture has to be returned by any INotifyingExecutorService.

public class NotifyingExecutorService extends ForwardingExecutorService implements INotifyingExecutorService{

    private final ExecutorService delegate;
    
    public NotifyingExecutorService(ExecutorService delegate) {
		super();
		this.delegate = delegate;
	}

	@Override
    public <T> INotifyingFuture<T> submit(Callable<T> task) {
    	if (task == null) throw new NullPointerException();
    	INotifyingFuture<T> ftask = newTaskFor(task);
        execute(ftask);
        return ftask;
    }
    
	@Override
    public INotifyingFuture<Void> submit(Runnable task) {
    	if (task == null) throw new NullPointerException();
        INotifyingFuture<Void> ftask = newTaskFor(task, null);
        execute(ftask);
        return ftask;
    }
    
	@Override
    public <T> INotifyingFuture<T> submit(Runnable task, T result) {
    	if (task == null) throw new NullPointerException();
    	INotifyingFuture<T> ftask = newTaskFor(task, result);
        execute(ftask);
        return ftask;
    }

    
	@Override
	protected ExecutorService delegate() {
		return this.delegate;
	}
    
	protected <T> INotifyingFuture<T> newTaskFor(Callable<T> callable) {
		return new NotifyingFuture<T>(callable);
	}
	
	protected <T extends Object> INotifyingFuture<T> newTaskFor(Runnable runnable, T value) {
		return new NotifyingFuture<T>(runnable, value);
	}
}

This class implements the new INotifyingExecutorService and extends guavas ForwardingExecutorService. This special ExecutorService implementation delegates all calls to any ExecutorService-instance that is passed into the constructor.

public interface INotifyingFuture<V> extends RunnableFuture<V>{

    /**
     * Sets this listener to a {@link INotifyingFuture}. When the future is done
     * or canceled the listener gets notified.<br> 
     * @param listener
     * @param the executor that executes the shiet.
     */
    public void setListener(IFutureListener<V> listener, ExecutorService executor);
    
    /**
     * Sets this listener to a {@link INotifyingFuture}. When the future is done
     * or canceled the listener gets notified.<br>
     * <b>Attention</b>: Be aware of the fact that everything that is done in that
     * listener is executed in same thread as the original task that this listener listens
     * to. Only use this method when you are sure that no long running task is performed
     * by the listener. When you want the listener's tasks to be performed asynchronous
     * use {@link #setListener(IFutureListener, ExecutorService)} instead.
     * @param listener
     */
    public void setListener(IFutureListener<V> listener);
}
public interface IFutureListener<V> {
	
	/**
	 * The task was computed successfully.
	 * @param result
	 */
	public void onSuccess(V result);
	
	/**
	 * called when future state is canceled.
	 */
	public void onCancel(RunnableFuture<V> cancelledFuture);
	
	/**
	 * Called when there was an error while executing 
	 * this future.
	 * @param e
	 * @param future the future that fails
	 */
	public void onError(Throwable e, Future<V> future);
}
public class NotifyingFuture<V> extends FutureTask<V> implements INotifyingFuture<V>{

    private static final ExecutorService DEFAULT_EXECUTOR = MoreExecutors.sameThreadExecutor();
    
	private IFutureListener<V> listener = null;
	private ExecutorService executor = null;
	private final AtomicBoolean executed = new AtomicBoolean();
	
	public NotifyingFuture(Callable<V> callable) {
		super(callable);
		setExecutor(DEFAULT_EXECUTOR);
	}

	public NotifyingFuture(Runnable runnable, V result) {
        super(runnable,result);
        setExecutor(DEFAULT_EXECUTOR);
    }

	
	@Override
	protected void done() {
	    if(listener==null){
	        return;
	    }
	    notifyListenerOnce();
	}

	/**
	 * Atomically executes the task only one time.
	 */
	protected void notifyListenerOnce(){
		if(!this.executed.getAndSet(true)){
			notifyListener();
		}
	}
	
	protected void notifyListener(){
		this.executor.submit(new TaskCompletionRunner<V>(delegateFuture(),this.listener));
	}
	
	/**
	 * @return the future that was processed.
	 */
	protected RunnableFuture<V> delegateFuture(){
		return this;
	}
	
	@Override
	public void setListener(IFutureListener<V> listener, ExecutorService executor) {
		setExecutor(executor);
		setListener(listener);
	}
	
	public void setExecutor(ExecutorService executor) {
        this.executor = executor;
    }

    @Override
    public void setListener(IFutureListener<V> listener) {
        this.listener = listener;
        /*
         * Probably the task was already executed. If so, call done() manually. 
         */
        runWhenDone();
    }
    
    protected void runWhenDone(){
    	if(isDone()){
        	notifyListenerOnce();
        }
    }
    
    private static class TaskCompletionRunner<V> implements Runnable{

        private final IFutureListener<V> listener;
        private final RunnableFuture<V> future;
        
        public TaskCompletionRunner(RunnableFuture<V> future, IFutureListener<V> listener) {
            this.future = future;
            this.listener = listener;
        }
        
        @Override
        public void run() {
            if(this.future.isCancelled()){
                this.listener.onCancel(this.future);
            }else{
                try {
                    this.listener.onSuccess(this.future.get());
                } catch (InterruptedException e) {
                    this.listener.onError(e, this.future);
                } catch (ExecutionException e) {
                    this.listener.onError(e, this.future);
                }
            }
        }
        
    }
}

Happy coding!

Notifying ExecutorService (1)

The java.util.concurrent.ExecutorService and it’s implementations are great to take advantage of today’s multicore systems. You can submit a task from Thread A and it will be queued until a (worker) Thread B from the ExecutorService takes it and calculates the result (this is also an example of how the Producer-Consumer-Pattern can be implemented, but that is not the reason of this post ;-). When a task is submitted by a thread, the thread returns immediately holding a java.util.concurrent.Future returned by the ExecutorService’s submit-methods. There is no blocking operation until someone calls get() on the returned Future (and the calculation of the Future has not yet been completed) to obtain the result of the computation. If however Thread A has a lot of work to do and the computation of the task takes a lot of time, this can be a problem.

Imagine the following example: you’re developing a game with the following participants: a game-client to visualize the game and read the user’s input, a server that validates and processes the client-data and a database that persists all relevant gamedata. Whenever a new gameobject (whatever that might be) is created by the gamelogic this object must be persisted into the database. To decouple gamelogic from database-operations you choose to submit all db-relevant tasks to an ExecutorService. Additionaly the communication between client and server is based on the id of the gameobjects which is assigned by the database.

  1. Client reacts on user’s input: Create new uber-soldier!
  2. Client sends request to server.
  3. Server (Thread A) validates request and creates a new task that is submitted to database-layer (Thread B).
  4. Thread A calls Future.get() to obtain the new uber-soldier-id. Thread A waits until Thread B is done (database assigned primary key).
  5. Server sends response to client with new uber-soldier-id.

Problems may arise when Thread A must handle messages from hundreds or thousands of clients. The longer the key-assignment by the database takes, the longer Thread A must wait. Sooner or later this leads into slow pings for every client. Players leave, your company is about to get broke. What to do?

One option is to assign the primary-keys not by database but by the game-server itself. However this gets complicated when several servers writing on several databases. Another option is a variant of the Observer-pattern. What if you could register Thread A as a listener on the future? Instead of blocking through a call on Future.get(), the listener would be notified when the key-assignment by the database is done. Great! Let me introduce the concept of NotifyingExecutorService.

While thinking about the problem, I found myself crawling through the internet and some libraries to make sure I won’t reinvent the wheel. Finally I stumbled upon the (awesome) guava-library (version 10+), and found their solution of the problem called ListeningExecutorService. That was almost all I was looking for: register to a (extended) Future and get notified when computation is done. Sadly the google-implementation only notifies the listener without passing the result to them. I did not know why they made this, but that is where I started coding.

To the second part.

Simple XML-deserialization using SimpleXML

Whenever you need to read-in some parameters to your java-application you have to chose the right format. In order to make the right descision, it is necessary to know what kind of data to store. Things are clear when you have key-value-data like this:
parameter1 = 1
parameter2 = 10
parameter3 = 25

To store such key-value-data you should use java.util.Properties. You can easily load such a file in three lines of code:

//Load
File f = new File(fileName);
Properties properties = new Properties();
properties.load(new FileReader(f));
//Retrieve data
String parameterName = "parameter1";
String property1 = properties.getProperty(parameterName);

However a key-value-store is not appropriate when the data to store

  • has it’s own format or
  • when a key can have more than one property or
  • when you want to map objects to your properties.

If one point is true for your data then you should have a look at XML-format. However, some years ago, it was pain to handle XML in Java (and I don’t speak of object-mapping here…). Gladly, things have changed and there exists the wonderful SimpleXML Framework in the Java-universe that helps you with both XML-serialization (write POJOs to XML) and XML-deserialization (read XML and map them to POJOs).

Read a XML-file

This example refers to the Strategy-Pattern-post I made earlier.

Imagine you want to read-in some calculations via XML. When you have a look at the Strategy-Pattern-Example you will find out, that a calculation consists of:

  • an operator
  • and two variables

Let’s define the class Calculation which stores those properties.

import org.simpleframework.xml.Attribute;
import org.simpleframework.xml.Root;

@Root(name="calculation")
public class Calculation {

	@Attribute
	private Operator operator;

	@Attribute
	private int valueA;

	@Attribute
	private int valueB;

}

The @Root-annotation says that this class needs can be serialized and mapped to an XML-element. The name-parameter defines the name of this element inside the XML-file. When no name is given, it defaults to the name of the class or field. The @Attribute-annotation specifies fields that are attribues (of an element) in terms of XML-specification.
Because one should have the possibility to calculate more than one calculation, we define the Calculations-class:

import org.simpleframework.xml.ElementList;
import org.simpleframework.xml.Root;

@Root(name="config")
public class Calculations {

	@ElementList(name="calculations")
	private List calculations;

}

This mapping-class is also pretty much straightforward. The @ElementList-annotation allows you to load multiple data-sets from the XML-file. You can also use @ElementArray for arrays of data or @ElementMap for java.util.Map-equivalent. (Note that you have to define the length of the array in your XML-file (length=”123″) as an attribute of the element that holds the annotation when using @ElementArray.)

Now let’s take a look to the XML-file with some example-data:

<config>
	<calculations>
		<calculation operator="ADD" valueA="1" valueB="2"/>
	</calculations>
</config>

We found out that mapping the XML to a POJO was easy. What about loading that file? As you will see, loading files with SimpleXML is also no rocket-science:

Serializer serializer = new Persister();
File file = new File("calculations.xml");
Calculations calculations = serializer.read(Calculations.class, file);

That’s all! No complex file-handling, no new lines of code (except of adding some annotations).
Watch out for my next posts that will show you how to: