Skip to main content


         This documentation site is for previous versions. Visit our new documentation site for current releases.      
 

Running out of memory when using NLP

Updated on July 5, 2022

If you have high memory consumption or exceed memory usage when using text analytics in Pega Platform, use the following best practices to improve your system performance.

Cause

The system might run out of memory because of issues related to the following factors:

  • Text size.
  • Heap size of the machine on which you run your application.
  • Inefficient regular expressions in Ruta scripts.

To ensure continuous text analysis and avoid out of memory issues, follow these recommendations for the natural language processing (NLP) engine:

Solution

  • Check whether the analyzed text is of the recommended size for optimal processing.
    A large text in NLP terminology has the following attributes:
    • Without sentiment analysis enabled:
      • 25,000 characters (max)
      • 5,000 words (max)
    • With sentiment analysis enabled:
      • 5,000 characters (max)
      • 1,000 words (max)
    Note: Text coming through chat channels generally does not exceed these limits, so this problem might only occur in email channels. For email channels, you can set the maximum text length to be analyzed by adding the condition @length(.pyInboundEmail.pyBody) > 25000 to the pySkipAnalysis When rule. This prevents the system from analyzing emails that exceed the maximum length.
  • Check whether the heap size is within the recommended range of 16-24 GB.
  • Optimize your Ruta scripts to avoid storing rule element matches and rule matches and remove unnecessary annotations:
    1. Mark the entity types as UNMARK.
    2. Minimize the use of other annotations when you do not need them, for example, BREAK and SPACE.

Have a question? Get answers now.

Visit the Support Center to ask questions, engage in discussions, share ideas, and help others.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best.

Pega.com is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice
Contact us