It seems that there is a memory leak somewhere. You should set "-XX:+HeapDumpOnOutOfMemoryError" You can specify also the directory with "-XX:HeapDumpPath=..." but the target directory must be writeable by the Openfire process.
The alternative is to use jmap to create heap dumps - jmap must be run as the same user Openfire is running. jmap and jhat are described for example here: http://petermodzelewski.blogspot.de/2013/06/short-jhat-tutorial-diagnosing.html