I get the following warning when my Akka 1.2-based application tries to work with more than 500 or so grid nodes:
Jan 05, 2012 1:36:43 PM org.jboss.netty.util.internal.SharedResourceMisuseDetector WARNING: You are creating too many HashedWheelTimer instances. HashedWheelTimer is a shared resource that must be reused across the application, so that only a few instances are created.
I am going to delve into Akka code to find out where HashedWheelTimer is used, and what should I do to avoid too many of those created. While I am doing this, I thought I would post a warning here if one of the experts can give me a pointer or two that could narrow and speed up my search, please?
Using Akka 1.3-RC6, I get the following exception:
Jan 12, 2012 1:24:38 PM org.jboss.netty.util.HashedWheelTimer WARNING: An exception was thrown by TimerTask. org.jboss.netty.channel.ChannelException: Failed to open a socket. at org.jboss.netty.channel.socket.nio.NioClientSocketChannel.newSocket(NioClientSocketChannel.java:49) at org.jboss.netty.channel.socket.nio.NioClientSocketChannel.<init>(NioClientSocketChannel.java:83) at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.newChannel(NioClientSocketChannelFactory.java:139) at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.newChannel(NioClientSocketChannelFactory.java:86) at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:218) at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188) at akka.remote.netty.ActiveRemoteClient.connect(NettyRemoteSupport.scala:470) at akka.remote.netty.ActiveRemoteClientHandler$$anonfun$channelClosed$1$$anon$4.run(NettyRemoteSupport.scala:599) at org.jboss.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:519) at org.jboss.netty.util.HashedWheelTimer$Worker.notifyExpiredTimeouts(HashedWheelTimer.java:440) at org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:379) at java.lang.Thread.run(Thread.java:722) Caused by: java.net.SocketException: Too many open files at sun.nio.ch.Net.socket0(Native Method) at sun.nio.ch.Net.socket(Net.java:323) at sun.nio.ch.Net.socket(Net.java:316) at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:101) at sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:60) at java.nio.channels.SocketChannel.open(SocketChannel.java:142) at org.jboss.netty.channel.socket.nio.NioClientSocketChannel.newSocket(NioClientSocketChannel.java:47) ... 11 more
source share