gruchalski.com

For the ICS application we are using ColdFusion 9. For quite a while we had a major problem. Our CF server kept crashing at least once a day without any warning. One minute it was running just fine, 30 seconds later all memory was consumed (3GB assigned for the JVM). Server was totally becoming unresponsive and the only way to bring it back to life was restarting CF service.

FusionReactor was no help for us, nor memory snapshots. It looked like we were blind, couldn’t find the problem. The code was optimised as much as possible, we went through every query, created indexes on the databases, profiled it, var’d all variables in the CFCs and so on – nothing was helping. And then finally, Saturday morning that was, a breakthrough: there’s clearly a problem with garbage collection (how obvious was that…), how can we tune it? Turns out that JVM supports 3 different collectors: throughput (-XX:+UseParallelGC), concurrent (-XX:+UseConcMarkSweepGC) and incremental (-XX:+UseTrainGC) – here’s the details “Tuning Garbage Collection with the 5.0 Java[tm] Virtual Machine”. My first thought was that we may have problems because we’re running in a VM, each of those VMs has 4 virtual CPUs assigned. According to the JVM docs we should have 4 collector threads by default. Somehow, at some point they were all failing to collect. So I changed the collector to concurrent and since then, about 5 weeks ago we had no crashes at all.

This is what I’m using for JVM settings:

-server -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:+UseConcMarkSweepGC -Xbatch -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.libPath={application.home}/../lib -XX:+HeapDumpOnOutOfMemoryError -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20

This entry was posted on Thursday, March 24th, 2011 at 5:30 pm and is filed under ColdFusion. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.