Memory leak causing OOM error in idle server

Description

Zanata is leaking memory. Running in Docker, I found it hit the upper limit overnight, while it was idle.
The only actions that were performed were:

  • Start server

  • Click sign in

  • Enter username and password

  • Click Sign in

I can see using top -p (pid) that the process gradually uses more % memory.

Potentially caused by the introduction of the transaction reaper.

Environment

None

Activity

Show:
Sean Flanigan
August 31, 2016, 8:09 AM

I tried attaching IntelliJ's debugger to an idle Zanata server, and according to the Memory View, over a period of several minutes the process apparently leaked a variety of objects:

  • native arrays

  • com.arjuna.ats.internal.jta.recovery.arjunacore.RecoveryXids

  • java.lang.* including:

    • java.lang.Thread*

    • java.lang.ref.Finalizer

    • java.lang.ref.SoftReference

  • org.apache.activemq.artemis.utils.LinkedListImpl$Iterator

  • org.apache.activemq.artemis.utils.LinkedListImpl[]

  • org.apache.activemq.artemis.utils.PriorityLinkedListImpl$PriorityLinkedListIterator

  • sun.nio.*

Given the absence of org.zanata.* classes, and the presence of Thread, Finalizer and various Artemis classes, I think this could be https://issues.jboss.org/browse/JBEAP-2947, which was fixed in EAP 7.0 GA and (I think) WildFly 10.1.

This leak doesn't seem to occur with WildFly 10.1, at least in a quick test.

Note: I think this problem may affect Zanata 3.9 on WildFly 10.0, but not on EAP 6 (which uses HornetQ, not ActiveMQ/Artemis). Zanata 3.8 (WildFly 9) should be unaffected.

I think we should consider changing the recommended WildFly version for Zanata 3.9 to 10.1.

Ready for Release

Assignee

Sean Flanigan

Reporter

Damian Jansen

Labels

Tested Version/s

None

Components

Sprint

None

Fix versions

Affects versions

Priority

Highest
Configure