glassfish:Resource temporarily unavailable in tsStartJavaThread

Today the test server (using glassfish3.1.1) suddenly reported the following exception while deploying the application. -bash: fork: retry: Resource permanently unavailable. If you have no further questions, then you have no further questions.

[ERROR] [ERROR][thread ] Could not start thread Keep-Alive-Timer. Resource temporarily unavailable
[ERROR] Exception in thread "Main Thread" java.lang.OutOfMemoryError: Resource temporarily unavailable in tsStartJavaThread (lifecycle.c:1096).
[ERROR] Java heap 3G reserved, 64M committed
[ERROR] Paged memory=18014398505034060K/18389412K.
[ERROR] Your Java heap size might be set too high.
[ERROR] Try to reduce the Java heap size using -Xmx:<size> (e.g. "
[ERROR]         at java.lang.Thread.start0(Native Method)
[ERROR]         at java.lang.Thread.start(Thread.java:597)
[ERROR]         at sun.net.www.http.KeepAliveCache$1.run(KeepAliveCache.java:89)
[ERROR]         at sun.net.www.http.KeepAliveCache.put(KeepAliveCache.java:75)
[ERROR]         at sun.net.www.http.HttpClient.putInKeepAliveCache(HttpClient.java:370)
[ERROR]         at sun.net.www.http.HttpClient.finished(HttpClient.java:358)
[ERROR]         at sun.net.www.http.ChunkedInputStream.closeUnderlying(ChunkedInputStream.java:196)
[ERROR]         at sun.net.www.http.ChunkedInputStream.processRaw(ChunkedInputStream.java:428)
[ERROR]         at sun.net.www.http.ChunkedInputStream.readAheadNonBlocking(ChunkedInputStream.java:493)
[ERROR]         at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:584)
[ERROR]         at sun.net.www.http.ChunkedInputStream.available(ChunkedInputStream.java:698)
[ERROR]         at java.io.FilterInputStream.available(FilterInputStream.java:142)
[ERROR]         at java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:196)
[ERROR]         at com.sun.enterprise.util.io.FileUtils.copy(FileUtils.java:915)
[ERROR]         at com.sun.enterprise.admin.remote.RemoteResponseManager.<init>(RemoteResponseManager.java:79)
[ERROR]         at com.sun.enterprise.admin.remote.RemoteAdminCommand.handleResponse(RemoteAdminCommand.java:1056)
[ERROR]         at com.sun.enterprise.admin.remote.RemoteAdminCommand.handleResponse(RemoteAdminCommand.java:1045)
[ERROR]         at com.sun.enterprise.admin.remote.RemoteAdminCommand.access$600(RemoteAdminCommand.java:101)
[ERROR]         at com.sun.enterprise.admin.remote.RemoteAdminCommand$1$1.handleReport(RemoteAdminCommand.java:526)
[ERROR]         at org.glassfish.admin.payload.PayloadFilesManager.processReport(PayloadFilesManager.java:473)
[ERROR]         at org.glassfish.admin.payload.PayloadFilesManager.access$900(PayloadFilesManager.java:93)
[ERROR]         at org.glassfish.admin.payload.PayloadFilesManager$DataRequestType$4.processPart(PayloadFilesManager.java:776)
[ERROR]         at org.glassfish.admin.payload.PayloadFilesManager.processPartsExtended(PayloadFilesManager.java:624)
[ERROR]         at org.glassfish.admin.payload.PayloadFilesManager.processParts(PayloadFilesManager.java:643)
[ERROR]         at com.sun.enterprise.admin.remote.RemoteAdminCommand$1.useConnection(RemoteAdminCommand.java:531)
[ERROR]         at com.sun.enterprise.admin.remote.RemoteAdminCommand.doHttpCommand(RemoteAdminCommand.java:700)
[ERROR]         at com.sun.enterprise.admin.remote.RemoteAdminCommand.doHttpCommand(RemoteAdminCommand.java:543)
[ERROR]         at com.sun.enterprise.admin.remote.RemoteAdminCommand.executeRemoteCommand(RemoteAdminCommand.java:482)
[ERROR]         at com.sun.enterprise.admin.remote.RemoteAdminCommand.executeCommand(RemoteAdminCommand.java:419)
[ERROR]         at com.sun.enterprise.admin.cli.remote.RemoteCommand.executeCommand(RemoteCommand.java:335)
[ERROR]         at com.sun.enterprise.admin.cli.CLICommand.execute(CLICommand.java:264)
[ERROR]         at com.sun.enterprise.admin.cli.AsadminMain.executeCommand(AsadminMain.java:306)
[ERROR]         at com.sun.enterprise.admin.cli.AsadminMain.main(AsadminMain.java:238)
[ERROR] [ERROR][thread ] Could not start thread Thread-0. Resource temporarily unavailable
[ERROR] Deployment of /home/javadev/codes/*****************.war failed.
[ERROR] For more detail on what might be causing the problem try running maven with the --debug option 
[ERROR] or setting the maven-glassfish-plugin "echo" property to "true".

We first analyzed the abnormal, see “Paged memory = 18014398505034060 k/18389412 k.” immediately startled, thought who is boring and the change of configuration information. Looking at the configuration parameters for the GlassFish startup JVM in the GlassFish console, I found nothing unusual and nothing changed compared to the other configurations. The same configuration worked well on the other test servers, initially eliminating GlassFish and JVM parameter configuration issues.
= “-bash: fork: retry: Resource permanently” (check the number of processes :

 [javadev@snaqi-test3 ~]$ ps -ef|wc -l
 1025

But in the/etc/security/limits the conf javadev user configuration

    javadev             soft    nproc   1024
    javadev             hard    nproc   16384
    javadev             soft    nofile  2048
    javadev             hard    nofile  65536

Obviously the number of user processes has reached the limit of the limit. Change line 1 from 1024 to 2048. Problem solved. In addition, the third line 2048 can be enlarged to prevent the “too many file litres” of the program.

Read More: