How to solve JAR hell with a Parent Last Classloader

Java Classpath Hell or Jar Hell is a term used in the Java community when a wrong class is loaded from a wrong Jar into the runtime causing the system not to function properly. Say you have two classes with the same fully qualified class name in two Jars. When the two Jar files are loaded, only the first class will be loaded into the runtime. So if your program depends on the second class which is not loaded, the runtime will provide your program the first class, which is wrong. A common issue is that when you use two versions of the same library, it is most likely that the classes in these two jars have the same fully qualified class names. So in Java only one set of classes from one jar will be in the runtime and if your program depends on the second jar, i.e. the second version of the library, your program will not function as you expect.

Java distinguishes class files from their fully qualified class names. So unless you use some technology like OSGi or your own Classloader hack, you can't effectively use two versions of the same library in your Java program. In this post, Ill give you a piece of code I've written to solve Jar Hell.

In Java the classes are loaded by a hierarchy of ClassLoaders. There are three types of Classloaders in the Classloader hierarchy. 

• Bootstrap Classloader
• Extensions Classloader
• System Classloader

Bootstrap Classloader loads the core java classes like java.lang.* when the JVM starts up. Extensions Classloader loads jar files in the extension directory (JAVA_HOME/jre/lib/ext). These are the extensions of standard core java classes. System Classloader is the one that loads your classes. When Java instantiates an Object, it searches for the Class which is loaded into the Runtime. This search happens in a funny way. 

Say you are instantiating a java.lang.String object in your Main java class which is loaded by the System Classloader. The Class java.lang.String is looked for by calling the findClass(classname) method in that System Classloader. What the findClass(classname) method does is that it calls the findclass() method in its parent Classloader which is the Extensions Classloader. And the findclass() method in the Extensions Classloader delegates the call to the findclass() method of the Bootstrap Classloader which is its parent. If the parent has loaded the class already, the parent will return the Class and if the parent returns null, the child Classloader will try to load that class. This way Java makes sure that it loads the correct java.lang.String class and even if you have implemented your own String class with java.lang.String fully qualified class name, it wont be loaded since the Bootstrap Classloader will already have loaded that class.

If you instantiate an Object for your own Class say com.you.test.YourTestClass, even if the findClass() method of the System Classloader which loaded your Main class delegates it to its parents, the parent classloaders will not know about a class called com.you.test.YourTestClass, so it will come back to the System Classloader by returning null, and the object will be created from the class loaded from the System Classloader.

The problem with this approach is that if you have two classes with the same fully qualified class name say com.you.test.YourClassCausingJarHell in two Jars, the System Classloader will return the one it has already loaded, resulting in a Jar Hell. To solve this we can use a ParentLastClassLoader like follows to load the classes from the jar files you specify irrespective of the fact that the Classloaders have already loaded a class with that fully qualified class name. The ParentLastClassloader will first try to load the Classes from the given Jars and it will delegate to its parents only if it can't find the Classes hence named ParentLast. Take a look at the implementation and note that it puts the already loaded classes in a data structure to use when needed.

public  class ParentLastClassLoader extends ClassLoader {

    private String[] jarFiles; //Paths to the jar files
    private Hashtable classes = new Hashtable(); //used to cache already defined classes

    public ParentLastClassLoader(ClassLoader parent, String[] paths)
    {
        super(parent);
        this.jarFiles = paths;
    }

    @Override
    public Class findClass(String name) throws ClassNotFoundException
    {
        System.out.println("Trying to find");
        throw new ClassNotFoundException();
    }

    @Override
    protected synchronized Class loadClass(String className, boolean resolve) throws ClassNotFoundException
    {
        System.out.println("Trying to load");
        try
        {
            System.out.println("Loading class in Child : " + className);
            byte classByte[];
            Class result = null;

            //checks in cached classes
            result = (Class) classes.get(className);
            if (result != null) {
                return result;
            }

            for(String jarFile: jarFiles){
                try {
                    JarFile jar = new JarFile(jarFile);
                    JarEntry entry = jar.getJarEntry(className.replace(".","/") + ".class");
                    InputStream is = jar.getInputStream(entry);
                    ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
                    int nextValue = is.read();
                    while (-1 != nextValue) {
                        byteStream.write(nextValue);
                        nextValue = is.read();
                    }

                    classByte = byteStream.toByteArray();
                    result = defineClass(className, classByte, 0, classByte.length, null);
                    classes.put(className, result);
                } catch (Exception e) {
                    continue;
                }
            }

            result = (Class) classes.get(className);
            if (result != null) {
                return result;
            }
            else{
                throw new ClassNotFoundException("Not found "+ className);
            }
        }
        catch( ClassNotFoundException e ){

            System.out.println("Delegating to parent : " + className);
            // didn't find it, try the parent
            return super.loadClass(className, resolve);
        }
    }
}

When instantiating this ParentLastClassLoader, the current SystemClassLoader which loaded your Main class will be set as its parent. So you can write your Main class like follows

public  class MainClass {

 

    public static void main(String [] args) {

        //This will instantiate the object from the allready loaded class from the System Classloader

        com.you.test.YourClassCausingJarHell yourClass = new com.you.test.YourClassCausingJarHell();

  
        String[] pathsToJars = {"jar_path1", "jar_path2"};
        ClassLoader loader = new ParentLastClassLoader(Thread.currentThread().getContextClassLoader(),

                             pathsToJars);

        Class correctClass = loader.loadClass("com.you.test.YourClassCausingJarHell");         
        Method theMethod = correctClass.getMethod("theMethodYouWant");

        //This calls the right method from the right class. 

        theMethod.invoke(correctClass.getConstructor().newInstance()); 

    }
}

Reactor Pattern Explained - Part 3

This is the 3rd and final post of the topic Reactor Pattern Explained. If you haven't read the previous posts, please read them since they include a great deal of explanations.

Reacter Pattern Explained - Part 1
Reacter Pattern Explained - Part 2

In this post the usage of Thread pools in Handlers is explained. We will create an extended version of Handler class named HandlerWithThreadPool. Check this out.

public class HandlerWithThreadPool extends Handler {

    static ExecutorService pool = Executors.newFixedThreadPool(2);
    static final int PROCESSING = 2;

    public HandlerWithThreadPool(Selector sel, SocketChannel c) throws IOException {
        super(sel, c);
    }

    void read() throws IOException {
        int readCount = socketChannel.read(input);
        if (readCount > 0) {
            state = PROCESSING;
            pool.execute(new Processer(readCount));
        }
        //We are interested in writing back to the client soon after read processing is done.
        selectionKey.interestOps(SelectionKey.OP_WRITE);
    }

    //Start processing in a new Processer Thread and Hand off to the reactor thread.
    synchronized void processAndHandOff(int readCount) {
        readProcess(readCount);
        //Read processing done. Now the server is ready to send a message to the client.
        state = SENDING;
    }

    class Processer implements Runnable {
        int readCount;
        Processer(int readCount) {
            this.readCount =  readCount;
        }
        public void run() {
            processAndHandOff(readCount);
        }
    }
}

Notice that there is a new state PROCESSING introduced and that the read() method is over-ridden. Now when a Read Event is dispatched to this Handler, it will read the data but not change the state to SENDING. It will create a Processer which will process the message and run it in a different Thread in the Thread pool and set the Interested Operation to OP_WRITE. At this point even if the Channel is ready to be written to and the Hander is interested in writing, the Handler will not write since its still in PROCESSING state. See the run() method of the Handler, it will only write when its in SENDING state. Once the Processer is done with its read process, it will change the state to SENDING. Now the Handler can send data to the client.

Lets run the Reactor with the boolean isWithThreadPool set to true.

public static void main(String[] args) throws IOException{

    Reactor reactor  = new Reactor(9900, true);
    new Thread(reactor).start();
}

Notice that the size of the Thread pool is 2 which would limit the number of Handlers to run concurrently to 2. As we already know, when the events are handled concurrenly we can easyly improve the system performance.

Done :). Hope you find this useful. Have fun with coding.
Cheers.

Reactor Pattern Explained - Part 2

This is the continuation of my previous post Reactor Pattern Explained - Part 1 which gives a high-level understanding of the topic.

In this blog post I will explain the implementation of Reactor Pattern with a simple Client - Server system where the server will send Hello messages to each client when their names are told to the server. The server will listen to port 9900 and multiple clients will connect to the server to shout out their names. A thread pool will not be used here. First lets run the server in a single thread. Part 3 of this series will explain how a Thread pool is used.

First lets make the Client to connect to port 9900.

public class Client {
    String hostIp;
    int hostPort;

    public Client(String hostIp, int hostPort) {
        this.hostIp = hostIp;
        this.hostPort = hostPort;
    }

    public void runClient() throws IOException {
        Socket clientSocket = null;
        PrintWriter out = null;
        BufferedReader in = null;

        try {
            clientSocket = new Socket(hostIp, hostPort);
            out = new PrintWriter(clientSocket.getOutputStream(), true);
            in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
        } catch (UnknownHostException e) {
            System.err.println("Unknown host: " + hostIp);
            System.exit(1);
        } catch (IOException e) {
            System.err.println("Couldn't connect to: " + hostIp);
            System.exit(1);
        }

        BufferedReader stdIn = new BufferedReader(new InputStreamReader(System.in));
        String userInput;

        System.out.println("Client connected to host : " + hostIp + " port: " + hostPort);
        System.out.println("Type (\"Bye\" to quit)");
        System.out.println("Tell what your name is to the Server.....");

        while ((userInput = stdIn.readLine()) != null) {

            out.println(userInput);

            // Break when client says Bye.
            if (userInput.equalsIgnoreCase("Bye"))
                break;

            System.out.println("Server says: " + in.readLine());
        }

        out.close();
        in.close();
        stdIn.close();
        clientSocket.close();
    }

    public static void main(String[] args) throws IOException {

        Client client = new Client("127.0.0.1", 9900);
        client.runClient();
    }
}

Notice that the client doesn't use java.nio to create the Socket. It simply uses a java.net.Socket everybody knows about.

Now lets make the Reactor in the Server.

public class Reactor implements Runnable {

    final Selector selector;
    final ServerSocketChannel serverSocketChannel;
    final boolean isWithThreadPool;

    Reactor(int port, boolean isWithThreadPool) throws IOException {

        this.isWithThreadPool = isWithThreadPool;
        selector = Selector.open();
        serverSocketChannel = ServerSocketChannel.open();
        serverSocketChannel.socket().bind(new InetSocketAddress(port));
        serverSocketChannel.configureBlocking(false);
        SelectionKey selectionKey0 = serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
        selectionKey0.attach(new Acceptor());
    }


    public void run() {
        System.out.println("Server listening to port: " + serverSocketChannel.socket().getLocalPort());
        try {
            while (!Thread.interrupted()) {
                selector.select();
                Set selected = selector.selectedKeys();
                Iterator it = selected.iterator();
                while (it.hasNext()) {
                    dispatch((SelectionKey) (it.next()));
                }
                selected.clear();
            }
        } catch (IOException ex) {
            ex.printStackTrace();
        }
    }

    void dispatch(SelectionKey k) {
        Runnable r = (Runnable) (k.attachment());
        if (r != null) {
            r.run();
        } 
    }

    class Acceptor implements Runnable {
        public void run() {
            try {
                SocketChannel socketChannel = serverSocketChannel.accept();
                if (socketChannel != null) {
                    if (isWithThreadPool)
                        new HandlerWithThreadPool(selector, socketChannel);
                    else
                        new Handler(selector, socketChannel);
                }
                System.out.println("Connection Accepted by Reactor");
            } catch (IOException ex) {
                ex.printStackTrace();
            }
        }
    }
}

The Reactor is a Runnable. See the while loop in the run() method. It will call selector.select() to get the SelectionKeys which have pending IO events. When the SelectionKeys are selected, they will be dispatched one by one. See the dispatch() method. The SelectionKey will have an attatchment which is also a Runnable. This attatchement will either be an Acceptor or a Handler.
Notice how the Acceptor inner class in the Reactor accepts connections to make SocketChannels. When a SocketChannel is created a new Handler will be created as well. (HandlerWithThreadPool will be discussed in the next section)

public class Handler implements Runnable {

    final SocketChannel socketChannel;
    final SelectionKey selectionKey;
    ByteBuffer input = ByteBuffer.allocate(1024);
    static final int READING = 0, SENDING = 1;
    int state = READING;
    String clientName = "";

    Handler(Selector selector, SocketChannel c) throws IOException {
        socketChannel = c;
        c.configureBlocking(false);
        selectionKey = socketChannel.register(selector, 0);
        selectionKey.attach(this);
        selectionKey.interestOps(SelectionKey.OP_READ);
        selector.wakeup();
    }


    public void run() {
        try {
            if (state == READING) {
                read();
            } else if (state == SENDING) {
                send();
            }
        } catch (IOException ex) {
            ex.printStackTrace();
        }
    }

    void read() throws IOException {
        int readCount = socketChannel.read(input);
        if (readCount > 0) {
            readProcess(readCount);
        }
        state = SENDING;
        // Interested in writing
        selectionKey.interestOps(SelectionKey.OP_WRITE);
    }

    /**
     * Processing of the read message. This only prints the message to stdOut.
     *
     * @param readCount
     */
    synchronized void readProcess(int readCount) {
        StringBuilder sb = new StringBuilder();
        input.flip();
        byte[] subStringBytes = new byte[readCount];
        byte[] array = input.array();
        System.arraycopy(array, 0, subStringBytes, 0, readCount);
        // Assuming ASCII (bad assumption but simplifies the example)
        sb.append(new String(subStringBytes));
        input.clear();
        clientName = sb.toString().trim();
    }

    void send() throws IOException {
        System.out.println("Saying hello to " + clientName);
        ByteBuffer output = ByteBuffer.wrap(("Hello " + clientName + "\n").getBytes());
        socketChannel.write(output);
        selectionKey.interestOps(SelectionKey.OP_READ);
        state = READING;
    }
}

A Handler has 2 states, READING and SENDING. Both cant be handled at the same time because a Channel supports only one operation at one time. Since its the client who speaks first, a server Handler will start with the READING state. Notice how this Handler is attatched to the SelectionKey and how the Interested Operation is set to OP_READ. This means that the Selector should only select this SelectionKey when a Read Event occurs. Once the read process is done, the Handler will change its state to SENDING and will change the Interested Operation to OP_WRITE. Now the Selector will select this SelectionKey only when it gets a Write Event from the Channel when its ready to be written with data. When a Write Event is dispatched to this Handler, it will write the Hello message to the output buffer since now the state is SENDING. Once sending is done, it will change back to READING state with Interested Operation changed to OP_READ again. It should be obvious that since both Handler and Acceptor are Runnables, the dispatch() method of the Reactor can execute the run() method of any attatchment it gets from a selected SelectionKey.

Here is the main method. We will run it without a Thread pool for the moment.

public static void main(String[] args) throws IOException{

    Reactor reactor  = new Reactor(9900, false);
    new Thread(reactor).start();
}

To see how this works first run the server. Then run several clients and see how they get connected to the server. When each client writes a name to standard in of the client, the sever will respond to the client with a Hello message. Notice that the server runs in a single Thread but responds to any number of clients which connect to the server.

Read the next section Reacter Pattern Explained - Part 3 to see how to use a Thread pool to run Handlers.