1. Home
  2. Article Catalog
  3. Event-driven financial calculations with Thomson Reuters Eikon for Excel and Visual Basic for Applications

Out Of Memory Issues

Out-of-Memory Issues in RFA Java Applications

Erwin Casas
Investment & Advisory Market Development Manager for Middle East & Africa, Refinitiv Investment & Advisory Market Development Manager for Middle East & Africa, Refinitiv

About this article

This article is intended for Java software users who are developing RFA Java Consumer applications. It is going to explain how to verify, if the application is out of memory, due to slow processing data events/messages and the ways to solve the problem.

Overview

An application can make a data request by creating an Interest Specification which contains the data request information (e.g., the item name, the model type, the service name) and send it to RFA. It is then placed into RFA’s queue and RFA will send it to the server. Next, the server will process the request and send the data back to RFA i.e. a Refresh followed by Updates until the application cancels the request. A new event (data) is passed to the application and is placed in the application’s event queue. The application can retrieve an event from the event queue by calling dispatch() method periodically or on a notification as the figure shown below:

  • rgjfkf
  • ffdfdb
  • bfdbfdbdb

 

The processEvent(..) method which is implemented by the application is then called to process the event. Finally, the application code handles data in the event as required e.g. displaying in a GUI, storing the data in a database, processing for algorithmic trading and so on. An Out of memory issue can occur if processEvent(..) spends too much time processing each event, resulting in a gradual increase in pending events (in the queue) and thereby consuming a huge amount of memory in the queue.

Problem

The users can experience out of memory when the application does not process the incoming events fast enough; leading to an ever increasing number of events which have not yet been processed by the application pending in the event queue. As a result, the memory usage grows until the application runs out of memory. The users can verify if the out of memory issue is due to a lot of pending events in the event queue by observing a heap dump file when the problem occurs. Java VisualVM(jvisualvm.exe) which ships with JDK can be used to take a heap dump. When browsing the heap dump file, it will show OMMItemEventMsg instances and their internal instances i.e. RwfTypedBufferRwfMsgParsed as the top classes using up the most memory as per the example below:

In the event queue instance (EventQueueImpl), you will see a lot of pending events shown in _count of EventQueueImpl instance which could reach a million (or higher) as in the example shown below:

 

If you take a heap dump periodically, you will see that _count of EventQueueImpl increases as per the number of pending events in the event queue.

Solution

To solve the problem, we have to change the implementation of the application so that it can process the events in a more timely manner and thereby keep the number of pending events in the event queue low. The lower the number of events in the event queue, the lower the memory usage will be. The suggested solutions are:

  1. Minimize the time spent Processing incoming data in the callback method processEvent(..). Avoid making database and/or other blocking calls in the method. The application could use an additional thread to process the received event separately. Therefore, minimum time is spent in processEvent(..) and other events in the event queue are picked up to be processed in a more timely manner. The example source code:
    	
            

public void processEvent(Event event)

       {

           if (event.getType() == Event.OMM_ITEM_EVENT)

           {

               OMMItemEvent ie = (OMMItemEvent)event;

               OMMMsg respMsg = ie.getMsg(); 

               //WorkerThread implements Runnable to process data in run()

               //it accepts OMMMsg(data) and OMMPool used to acquire/release OMMMsg

               Runnable worker = new WorkerThread(respMsg,_mainApp.getPool()); 

               // _executor is an ExecutorService instance which mananges a pool of threads

               //.execute(..) calls WorkerThread.run() to process data by a separated thread 

               _executor.execute(worker);      

           }

       } 

       class WorkerThread implements Runnable 

       {  

           private OMMMsg datamsg;

           private OMMPool pool;

           public WorkerThread(OMMMsg msg,OMMPool p)

           {

               pool = p;

               datamsg = pool.acquireMsg();

               //copy(keep) data which will be processed later

               datamsg.initFrom(msg);

           }  

           public void run() {  

               //process data which is in OMMMsg instance(datamsg) accord to business logic

               ...      

               //release OMMMsg which data has been processed already  

               pool.releaseMsg(datamsg);

           }

}

Popular Topics