Register your own URLStreamHandler in a Spring web application (Tomcat)

Question

I am trying to register a custom URLStreamHandler to handle requests for Amazon S3 URLs in a generic way. The handler implementation is very similar to the S3-URLStreamHandler (github) . I did not put the Handler class in the package sun.net.www.protocol.s3 , but I used the custom package com.github.dpr.protocol.s3 . To make Java this package, I provided the system property -Djava.protocol.handler.pkgs="com.github.dpr.protocol" , following the documentation for the URL class . However, if I try to handle the s3 URL, for example s3://my.bucket/some-awesome-file.txt , I get a MalformedURLException :

 Caused by: java.net.MalformedURLException: unknown protocol: s3 at java.net.URL.<init>(URL.java:600) at java.net.URL.<init>(URL.java:490) at java.net.URL.<init>(URL.java:439) at java.net.URI.toURL(URI.java:1089) ... 

My application is a Spring-based web application that currently runs on tomcat but should not be cluttered with any knowledge about the container of the main application.

I already debugged the corresponding code and found that my URLStreamHandler could not be initialized, because the class loader used to load the class does not know it. This is the corresponding code from java.net.URL (jdk 1.8.0_92):

 1174: try { 1175: cls = Class.forName(clsName); 1176: } catch (ClassNotFoundException e) { 1177: ClassLoader cl = ClassLoader.getSystemClassLoader(); 1178: if (cl != null) { 1179: cls = cl.loadClass(clsName); 1180: } 1181: } 

The class loader class java.net.URL (used by Class.forName ) is null , indicating the bootloader class loader, and the system class loader does not know my class. If I set a breakpoint and try to load the handler class using the current thread class loader, it works fine. This is my class, apparently exists and is in the class path of the application, but Java uses the “wrong” class loaders to find the handler.

I know this question in SOF , but I should not register the custom URLStreamHandlerFactory , since tomcat registers its own factory ( org.apache.naming.resources.DirContextURLStreamHandlerFactory ) when the application starts, and there should be only one factory registered for one JVM. Tomcat DirContextURLStreamHandlerFactory allows you to add custom factories that can be used to process custom protocols, but I don’t want to add a Tomcat dependency to the application code, because the application must work in different containers.

Is there a way to register a handler for custom URLs in container independent mode?

---

UPDATE 2017-01-25:
I gave different options @Nicolas Filotto suggested trying:

Option 1 - Setting a Custom URLStreamHandlerFactory Using Reflection
This option works as expected. But using reflection, it introduces a strong dependence on the internal actions of the java.net.URL class. Fortunately, Oracle is not too keen on fixing usability issues in the Java base classes - in fact, this related bug report has been open for almost 14 years (Sun / Oracle works great) - and it can be easily tested.

Option 2 - Put the JAR handler in {JAVA_HOME} / jre / lib / ext
This option works. But just adding a handler banner as an extension of the system will not do the trick - of course. You will also need to add all the handler dependencies. Since these classes are visible to all applications using this Java installation, this can lead to undesirable effects due to different versions of the same library in the class path.

Option 3 - Put the JAR handler in {CATALINA_HOME} / lib
This does not work. According to the Tomcat Class Load Documentation, resources placed in this directory will be loaded using the Tomcat Common classloader. This class loader will not be used by java.net.URL to find the protocol handler.

Given these options, I will stay with the reflection option. All options are not very pleasant, but at least the first one can be easily tested and does not require any deployment logic. However, I adapted the code a bit to use the same lock object for synchronization, as java.net.URL does:

 public static void registerFactory() throws Exception { final Field factoryField = URL.class.getDeclaredField("factory"); factoryField.setAccessible(true); final Field lockField = URL.class.getDeclaredField("streamHandlerLock"); lockField.setAccessible(true); // use same lock as in java.net.URL.setURLStreamHandlerFactory synchronized (lockField.get(null)) { final URLStreamHandlerFactory urlStreamHandlerFactory = (URLStreamHandlerFactory) factoryField.get(null); // Reset the value to prevent Error due to a factory already defined factoryField.set(null, null); URL.setURLStreamHandlerFactory(new AmazonS3UrlStreamHandlerFactory(urlStreamHandlerFactory)); } } 

Answer 1

You can:

1. Use a decorator

One way to set a custom URLStreamHandlerFactory might be to use a decorator like URLStreamHandlerFactory to wrap a URLStreamHandlerFactory that can already be defined (in this case tomcat). The tricky part is that you need to use reflection (which is pretty hacks) to get and reset the current factory potentially specific.

Here is the pseudo code of your decorator:

 public class S3URLStreamHandlerFactory implements URLStreamHandlerFactory { // The wrapped URLStreamHandlerFactory instance private final Optional<URLStreamHandlerFactory> delegate; /** * Used in case there is no existing URLStreamHandlerFactory defined */ public S3URLStreamHandlerFactory() { this(null); } /** * Used in case there is an existing URLStreamHandlerFactory defined */ public S3URLStreamHandlerFactory(final URLStreamHandlerFactory delegate) { this.delegate = Optional.ofNullable(delegate); } @Override public URLStreamHandler createURLStreamHandler(final String protocol) { if ("s3".equals(protocol)) { return // my S3 URLStreamHandler; } // It is not the s3 protocol so we delegate it to the wrapped // URLStreamHandlerFactory return delegate.map(factory -> factory.createURLStreamHandler(protocol)) .orElse(null); } } 

Here is the code to determine it:

 // Retrieve the field "factory" of the class URL that store the // URLStreamHandlerFactory used Field factoryField = URL.class.getDeclaredField("factory"); // It is a package protected field so we need to make it accessible factoryField.setAccessible(true); // Get the current value URLStreamHandlerFactory urlStreamHandlerFactory = (URLStreamHandlerFactory) factoryField.get(null); if (urlStreamHandlerFactory == null) { // No factory has been defined so far so we set the custom one URL.setURLStreamHandlerFactory(new S3URLStreamHandlerFactory()); } else { // Retrieve the field "streamHandlerLock" of the class URL that // is the lock used to synchronize access to the protocol handlers Field lockField = URL.class.getDeclaredField("streamHandlerLock"); // It is a private field so we need to make it accessible lockField.setAccessible(true); // Use the same lock to reset the factory synchronized (lockField.get(null)) { // Reset the value to prevent Error due to a factory already defined factoryField.set(null, null); // Set our custom factory and wrap the current one into it URL.setURLStreamHandlerFactory( new S3URLStreamHandlerFactory(urlStreamHandlerFactory) ); } } 

NB: Starting with Java 9, you need to add --add-opens java.base/java.net=myModuleName to your launch command so that you can deeply reflect the java.net package that includes the URL class from your myModuleName module otherwise call setAccessible(true) will raise a RuntimeException .

---

2. Expand it as an extension

Another way to avoid this ClassLoder problem might be to move your custom URLStreamHandler to a dedicated jar and deploy it to ${JAVA-HOME}/jre/lib/ext as an installed extension (with all its dependencies) so that it is available in Extension ClassLoader , so your class will be defined in ClassLoader , high enough in the hierarchy to be visible.