Note: This blog post is from 2020. Some content may be outdated--though not necessarily. Same with links and subsequent comments from myself or others. Corrections are welcome, in the comments. And I may revise the content as necessary.

Perhaps you're getting errors referring to "metaspace" or "OutOfMemoryError: Metaspace", whether in your web sites, error logs, or even the CF Admin, and you wonder "what to do". Or you may be getting odd occurrences of blank pages, and if you look in your coldfusion-error.log you are finding such metaspace errors.

TLDR; In all these cases, the solution is simple (and may seem contrarian to some ears): REMOVE the maxmetaspace element from your JVM arguments. Indeed, I would go so far as to say everyone should simply remove it, even BEFORE you may get errors.

In the post that follows, I will explain how to remove it, including how you need to be VERY careful when doing that. You may also wonder why I recommend removing it, versus raising it. I cover that, as well as a bug report I filed with Adobe related this tis (which was fixed as of CF2021), below.

I also created an abbreviated version of this post, on the Adobe CF portal, if that may interest some readers.

How to know if you are having the problem

I mentioned how the problem may exhibit itself (as pages coming back blank or failing), but of course the real proof will be that you will see errors in your logs. You may find in your coldfusion-out.log or application.log, perhaps looking like this:

or more specifically, in your coldfusion-error.log, you may see lines like this:

That's the absolute proof that this is your problem (you're running out of metaspace), especially if you find them happening around the same time you experienced odd behavior from CF (or your app server. Those on other app servers should look to the logs for this info.)

Do beware that the CF logs are set to rotate when they fill, so even if you may not find these errors in the specific logs I named, look at their archived versions, which would have a -1 and so on, reflecting when they filled and were archived.

The solution

And if the CF settings for maxmetaspacesize are indeed the default (192m), or even if you have tried to "raise" it, I'm proposing here that for most people, you just need to REMOVE that argument and stop "chasing that rabbit".

More below. First, some precautions.

Be careful about modifying JVM arguments

First and foremost, whether you're experienced in modifying the JVM arguments underlying CF (or other Java app servers), you do need to be VERY cautious about making this modification. If you make certain easy mistakes, the configuration will be broken and CF (or your other app server) will not be able to start (heck, it may not even be able to STOP, as the same JVM args are used in the process that STOPS CF as starts it).

Before proceeding, you should backup the file that holds the arguments (before making any changes), and then do be very careful about editing it, as I detail below.

Where to find the JVM args in CF

In the case of ColdFusion at least, this and other JVM args will be found in the CF Admin, in its "Java and JVM" page, and then its "JVM arguments" field.

Changing that page ends up changing the underlying jvm.config file, where you can also find the java arguments appearing on its java.args line (you may see slightly different args which appear on that line than in the "JVM arguments" field of the Admin).

If you have only one CF instance, it's in your cfusion/bin folder, while if you have multiple instances, it will be found in a sibling folder to cfusion with the name of your instance, and then in its bin folder.

Backing up the JVM config file before changing things

Whether you will be modifying the Admin or that file, you should make a backup of this underlying jvm.config file.

You can call the backup jvm.config.bak, for instance. (Note that if you change the CF admin JVM page, it will automatically make its own backup as jvm.bak, and you may be well-served to name your backup something else, as I have proposed.)

How to remove the argument, carefully

Finally, whether changing the args in the CF Admin or in jvm.config, you will find the argument looking like this:

You may also find an -XX:MetaspaceSize argument, which represents the "initial" size for the metaspace. CF does not set one by default, but you or someone else may have set one. I would argue also to remove that.

When removing either, you want to be sure to remove the opening dash before that XX: prefix. Be sure also to leave a space between the args currently found before and after this one.

If you make either mistake (leave the dash, or fail to leave a space between args), CF (your Java app) may not start or stop. Notice in the screenshot, where I point to the leading dash, and how it can appear on the end of line to the argument, so could easily be missed and "left" by mistake.

Restart CF, after making the change

After removing the argument, be sure to restart CF. (And don't make the change unless you are prepared to restart CF right away, lest you be gone when CF is restarted unexpectedly, and perhaps won't start because of a mistake you made.)

If somehow CF won't start, be prepared to restore the file you changed, and try again. If you had opened an editor to make the change to the file, and it's still open, look closely at what your changed (and if your editor supports an undo feature, see what happens if you undo your last change).

With this maxmetaspace argument removed, you should no longer ever get any outofmemory errors about the metaspace.

You don't need to change anything about the "failing" pages

And to be clear, there's nothing about whatever pages GOT the error that needs to change. They did not CAUSE the error: they were merely victims of it. Any CF restart would "resolve" the problem, at least temporarily... until CF ran out of metaspace again--in which case some OTHER page would likely fail. Thus the need to solve the REAL problem, by removing this artificially low maxmetaspace setting.

Still more, including the related bug report I have just filed

Update: As of the release of CF2021, Adobe has in fact removed the maxmetaspacesize setting (assuming you do not set it or import it from another instance.) So I have stricken out the remaining paragraphs in this section.

Again, I have filed a bug report today asking Adobe to STOP setting this maxmetaspacesize value for us. If you agree, please add your vote here: https://tracker.adobe.com/#/view/CF-4207269.

Note that it offers some additional perspective on how it came to be that Adobe does set it, and why I think they should not (and why I recommend removing it vs raising the argument's value).

This problem has plagued CF shows for a few years now. Let's stop the madness!

And again, as the above demonstrates, I think everyone should remove the argument. It's simply no longer needed, in the way that the older maxpermsize was. (I have added this sentence and a similar one at the top, prompted by a helpful comment from "Wouter". And it also led me to think to add these following two additional sections.)

Then why does Java even offer the argument?

Fair question. It's there if you ever wanted to keep the user of the metaspace from growing WILDLY large. Again, metaspace is taken from available memory on the box (not from within the heap), so it's less of a concern for most to bother limiting it. But maybe you may want to say, "ok, feel free to grow large. I won't set an arbitrarily low limit--like CF's default 192m setting--but don't go using more than 1gb". In that case, feel free to set the maxmetaspace to 1g (and you can say 1024 or 1g. Java will figure it out).

But I'm saying that I've never seen anyone NEED to. But that leads to another question...

How to watch the use of metaspace within CF/the JVM

Another reasonable question. If you have CF2018 or above (Standard or Enterprise), you can view it within the PMT (Performance Monitoring Toolset), if you set that up. One of the things it can track is the use of memory in various JVM memory spaces, including the Metaspace. See the "jvm" page, and the "non-heap" graph there, which breaks down the heap spaces that are NOT in the heap. An article including screenshots is here.

And the same can be viewed with FusionReactor, which tracks memory spaces via the Resources menu on the left, then "Memory spaces", and then you can select a desired memory space in the top right corner of that graph, as shown here. FR also tracks such memory space usage in its logs (written every 5 seconds and kept for 30 days), viewable in the Metrics>Archive Metrics page, and the "Memory section of logs it shows. (To be clear, this also allows you to look at values BEFORE a restart.)

Finally, of course, various jvm tools (command line or gui's, or GC logging) can show JVM memory spaces, including the metaspace.

Indeed, if you were to watch the metaspace and see it rising substantially and over time, you would want to really address WHY it is rising (rather than merely "limit" it)--as well as understanding then why and when it is that SOME but not other pages experience the error. I will say that classically, unexpected metaspace growth in the metaspace (as well as formerly in the permgen space) has been due to excessive class-loading, which can have any of many explanations. But that's getting well beyond the scope of this post, of how to readily solve the maxmetaspace problem itself. Perhaps I'll do another on that broader, background topic.

Conclusion

In the meantime, I do hope that my suggestion at the top (for YOU to remove the offending argument if you get these errors) will help many folks, and I do realize that this post may lead to debate and questions, so fire away.

And as always, if you may prefer help either implementing this change (or somehow further assessing it), or recovering from problems if you try it yourself, I am available for online remote consulting, with satisfaction guaranteed. More at carehart.org/consulting.

For more content like this from Charlie Arehart:

• Signup to get his blog posts by email:
• Follow his blog RSS feed
• View the rest of his blog posts
• View his blog posts on the Adobe CF portal

Need more help with problems?

• If you may prefer direct help, rather than digging around here/elsewhere or via comments, he can help via his online consulting services
• See that page for more on how he can help a) over the web, safely and securely, b) usually very quickly, c) teaching you along the way, and d) with satisfaction guaranteed