Solving a Memory Leak Caused by the Default Tolven Configuration

For me, one of the major attractive features of the Tolven platform is its scalability.  In theory, Tolven can scale to handle anything from a personal health record for an individual or family to a large hospital, or even a network of large hospitals.  In practice, however, successfully scaling up can be a little tricky.  There’s some configuration to be done, and finding the magic combination is like finding a needle in a haystack.

As the number of users using our system grew, one major issue we discovered was a slow memory leak caused by the default EJB pooling configuration.  Out of the box, the JBoss application server comes configured to give each thread its own pool of Stateless Session Beans.  There are some benefits to this approach, such as a thread not having to contend with other threads for resources.  However, due to the way Tolven creates threads when processing documents, the number of EJBs in memory was growing boundlessly and crashing our system.  We resorted to periodic restarts of the application to prevent crashes until we could resolve the issue.

By switching to an EJB pool with a maximum size, we were able to prevent this unbounded growth.  Of course, with a maximum size pool, we had to consider the maximum value.  Setting it too low would create a bottleneck where threads were waiting for limited resources in the pool to become available.  Setting it too high could cause the same  problem we’re trying to avoid: running out of memory.   Fortunately, the default size of the pool we used has proven to be more than sufficient for Tolven, without even coming close to maxing out our available memory.

As usage continues to grow, we will continue to evaluate this configuration, as well as others, to ensure we can scale application performance to handle the load.

For instructions on how to configure the Stateless Session Bean pool, click here to visit the Roberts-Hoffman  forum.