How to solve common problems with applying ColdFusion updates
Note: This blog post is from 2016. Some content may be outdated--though not necessarily. Same with links and subsequent comments from myself or others. Corrections are welcome, in the comments. And I may revise the content as necessary.Has this happened to you: you want to apply some update to your current version of CF--and it fails. Ugh. While the ColdFusion admin has a simple update UI mechanism, what can you do if the update fails to apply? Indeed, how can you know if it DID fail? how can you recover? what might you do to prevent a failure? The answers may be simple on the surface, but not obvious to most.
And you'll likely be in panic mode. Many find after applying a ColdFusion update that either CF won't start at all, or they can't access the ColdFusion Admin, or some part of CF or their app doesn't work. Or perhaps the problem may not become clear for hours or even days.
Typically the issue is that there was an error during the update process which CF attempts, and that might be rather easily confirmed and resolved. In this post, I share several tips and observations related to all this, based on my years of providing remote CF troubleshooting support.
It wasn't hard to search through the log file for the 1 warning that I had showing up: ag -i "Status: [^S]" hf-11-00012.log
Just look for a "Status:" message that doesn't start for "S" (as in SUCCESSFUL)
The Warning was trying to copy or move the entire cf root folder - I can't see why the installer would want to do this? Does the updater back up the entire cf directory as it updates?
Move Folder: Destination: /opt/coldfusion11/cfusion
Status: WARNING
Additional Notes: WARNING - There was a problem copying /opt/coldfusion11/cfusion
But as I said in the blog post, the most common reason that it can't do THAT (or whatever it wants to do, especially modifying files) is that the CF instance (and processes or services related to CF) did not come down when the update told them to. (Or if you stopped CF yourself, and tried to apply the update manually, you did not bring down all the processes/services related to CF.)
So as I discuss in the post, you have 2 choices: try the automated update again, or try a manual update.
And if you are going to try the automated update process, I recommend you first restart CF (in case doing that would make it restart faster when the updater would do it), and then before trying the automated installer again, go ahead and stop the related CF processes/services yourself. Then the updater need not bother (but of course you can't stop CF itself, as the automated updater runs from the Admin!)
Or again, finally, you can just go ahead and stop CF and all the related processes/services, and do the CF installer manually, from the command line, as I discuss above.
For anyone reading what I'm writing here and feeling a bit lost, please reread the post above.
And Peter, if with that explanation (and your own rereading of the post) you are somehow still doubting what I say (that CF is NOT trying to "backup the entire cf directory"), please do write back.
I have a question regarding manually installing the update. Would I use the java.exe file that is located within the Coldfusion directory, or would it be the java.exe file that is located within the JRE version that Coldfusion is currently running on? For example, if Coldfusion 11 is running on Java 8, should I be using that java.exe to execute the .jar file?
The reason I ask is because I tried to run the updater via CFADMIN, and it failed. Looking through the update logs, I see ALOT of the following error:
Jump To: Next Validate_CF_Directory -- Prev: NO JUMP
Status: ERROR
Additional Notes: ERROR - java.lang.StackOverflowError
I also got a few:
com.adobe.ia.customrule.DocRootValidate
Status: ERROR
Additional Notes: ERROR - Unexpected Fatal Error in Class: com.adobe.ia.customrule.DocRootValidate.install()
Thanks!
As you may have noticed, I left the details of the discussion of running Java to that Adobe blog post I pointed to, and perhaps you saw that it has you point to the Java that is implemented inside of CF (in CF's JRE folder). (The blog post has an error where slashes are missing, but most should be able to see past that.)
One problem with their proposing you do that is that technically I would argue you should be using the same version of Java that CF is itself using. And as you note, you could have changed CF to use a DIFFERENT java version than the one in the CF JRE folder (as set in the CF Admin Java & JVM page, mirrored as the java.home in the jvm.config file).
So I would propose that you should point to THAT one (wherever it is) as the path for the java command. (And as their blog post notes, do be sure to run that command line "as admin" if on Windows.)
As for your errors, those are indeed odd. To be clear, I would expect if you ran the update from the CF Admin that CF WOULD itself be using the JVM that it points to (in the CF Admin "java & jvm" page).
So I would be curious if you would somehow NOT experience those same errors if you ran it yourself.
Definitely an odd one. Perhaps there's more to your errors than meets the eye. For instance, perhaps there are errors in the log that precede those you refer to, and they could point to some other problem. Or as I hint at above, if you were to restart CF and THEN try the update from the Admin, might that work without error? WOrth a shot before trying it from the cmd line.
Anyway, will love to hear how things sort out for you. And of course if you may prefer direct help, I can offer that. See the consulting page for my approach, rates, satisfaction guarantee, and more. Either way, I hope you'll update folks here on how things go.
The log file starts off with a bunch of "custom action" that are all successful. When it gets to "install action", that is where the errors start. Nothing very clear from the messages.
I tried installing from the CF Admin twice (CF11 Update 14). When the first attempt failed, I rebooted the server to start with a clean version, and it failed again. What is odd, is that I have Coldfusion installed on my E:/ drive, and Java on my C:/. I noticed that a new "Coldfusion11" folder was added to my C:/ drive when I ran the install, and that is where all the update information has been stored. I have never had this happen. The only change I have made recently was updating the Java to version 8. Perhaps something is wonky between CF and the fact that the Java is on another drive.
I will try the manual update next week and will get back on how that went.
- So had you run the command line "as admin", as I mentioned? You don't confirm it (which is ok), but since you say that a cf11 folder was unexpectedly created on the C drive, I thought I'd ask. I've seen that happen in the past but never pursued when/why it had.
- Since that is referring to your running the update with the Java on C, what happens if you try it with the Java within CF (on your E drive)? Your first message referred only to the errors as being from when you ran the update via the CF Admin. I'll be curious to hear what happens if you try the update using the java within CF. I suppose that's what you'll be trying next week?
- If that does work, it could be interesting then if you may install Java onto your E drive (along with CF) and if you run the update again using THAT. (You could technically uninstall the update, but you don't need to. The second running of the update would just basically "do it again" with no harm for being a duplicate, at least in my experience.)
We'll see how things go for you. :-)
https://forums.adobe...
And see the comments today and yesterday, where I identified some issues regarding certain file processing in the update.
If you look in your ColdFusion11\cfusion\db\slserver54\bin\, is there any file of the pattern msvcr*.dll? If not, then it's looking like a workaround (until Adobe perhaps issues a new update) may be to copy the file of that pattern from the update back folder ( \ColdFusion11\cfusion\hf-updates\hf-11-00015\backup\db\sqlserver54\bin) to that folder above. Then try to start the CF ODBC services.
Let me know if that works for you.
Running the manual installer worked for me. These were the instructions that I received from Adobe.
Please stop the ColdFusion services.
1) Download the hotfix from: https://cfdownload.a...
2) Install JDK 7u80 or JDK 8u121 or higher. Change the JVM path in jvm.config file to point it to Java(For example: java.home= C:\Program Files\Java\jre1.8.0_121)
3) Copy the above hotfix to the java folder- C:\Program files\Java\jre1.8.0_121\bin
3) Launch the command prompt as Run as administrator & navigate to this location C:\Program files\Java\jre1.8.0_121\bin
4) Use command "java -jar hotfix_014.jar" & follow the on screen instructions. Restart the CF service and check.
Glad it got you going.
The windows application error stated "Faulting module path: MSVCR100.dll" so I searched the entire server and no file by that name existed. I found the file on a development machine (in Windows\system32) and copied it over to the windows system 32 folder on the other server, and … presto! CF ODBC Server started right up.
Hope this helps someone! I've received a lot of good advice from Charlie and the others here.
I would hope Adobe will fix it, and/or we find out what may be unique to cause some to experience it and not others.
"Another gotcha to beware, when CF is started by the manual installer" covered my last two update issues.
The manual updater for my installation runs in a CMD window, and when it is done, it restarts CF as a process as the same user I launched the updater with. In my case, the administrator!
The CF add-on service starts up, but the CF Application service does not--and if I try, it complains.
As you pointed out--I just looked for the Coldfusion process in the task manager. I stopped the task. I then manually could restart the CF Application service.
And I hope someday Adobe can fix that.
https://coldfusion.a...
https://coldfusion.a...
Hope it may help readers, here as well as there.
https://stackoverflo...
Here or on SO, I'll take whatever I can get! :) Thanks.
While I appreciate your concern that it seems disrespectful for them to just let them pile up there and "not remove them for you", I can see why they'd think that risky to do for you. The files in there are useful for a variety of things, and even the update folder for "a couple of updates ago" may prove valuable, as I will elaborate.
And I realize you're saying that "all updates have been applied successfully", but even so I'll share a few reasons why that may not be as clear as it seems.
First, I have helped many people who found out days or weeks after applying an update that there WAS in fact an error in the update. They didn't know, because CF came up, and they could run their app and the CF Admin, but down the road something showed up as failing, and we traced it to an error in the update. If they had simply deleted the whole hf-updates (or even that for a past "successful update"), we could not have looked at the log (the install log in the folder for each update) to know that this a missed failure in the update was the real problem.
Second, as you may know there was a recent problem in the Sept updates where the update proved over time to have problems. It wasn't immediately obvious, as it affected only certain situations. If one had deleted the update folder for the previous "successful" update/s, they would not have been able to do the uninstall back to that. (The uninstall uses that.)
So that at least argues for keeping the LAST update folder in place.
If you have more than one past update, and you decide after some period of time (I'd think months) that updates prior to that are no longer needed, you could delete those folders. (Or again perhaps better, move them to some archive location. You could move them back in if you needed to do an uninstall.)
As for the .jar file for each update, it's used to run that update (which one may choose to do after an uninstall), and it grows and grows with each update, as they are cumulative. But that can be downloaded from Adobe, so it's perhaps the safest thing of all to "remove". (The updates.xml file gets updates also when an update is downloaded.)
There is also a .properties file for each update applied, but it's tiny.
Still, if you wanted to, you could consider all three for each update as a group: the update folder, the .jar file, and .properties file. And once you really no longer "need" it, you could remove them as a group (or I'd argue, move them, to be safe).
(One might propose that you could compress those 3 as a group, but I found that the space savings on compression was only about a 20% reduction. This was the same if compressing just the update folder, and/or also the .jar file.)
So there you have it: again I'd say don't remove the entire hf-updates folder (as you proposed), but you could consider carefully which of the other items within it can be removed, and I'd recommend you do it as a group (per update).
And all that said, I could see value in Adobe at least adding an interface offering folks the ability to delete or move the group of related files/folders for a given update beyond the last one. You could make a feature request for that at tracker.adobe.com, pointing to your comment here, so interested folks can see the subsequent discussion. The link to your comment is https://www.carehart... (I added this paragraph as an update to my comment just after initially posting it, as I forgot I had meant to mention it.)
I'd look forward to your thoughts, or those of anyone else. This would make a good blog post, which I hope to turn this into.
From what you wrote, it sounds like those files/directories can be safely deleted or relocated without breaking anything currently running in production (including reboot survival), with the recommendation being to relocate those contents in the event that a rollback is required.
Indeed, if you had asked originally whether removing the contents would affect currently running production/restarts, I would have said no (though to be clear I would have still offered the rest also, for the sake of others).
So to your bottom line conclusion, I do concur.
And by all means, the full exploration you ventured is as much appreciated as the "short version" as I know it will help others!! =)
Thanks for your ongoing attention to detail and accuracy and for your continued contributions to the CF community. It feels like an ever-shrinking island out here sometimes.
As for the CF community seeming a shrinking island, I realize it can feel that way--especially listening to what some say.
FWIW, there are many different places where you will find vibrant CF communities, from the CFML Slack group, to the Facebook CF Programmers group, to the CF Portal (coldfusion.adobe.com), to the CF forums (community.adobe.com), to of course the community surrounding Ortus products (which I realize has a mix of CF and Lucee folks), to name just a few. More at my category of such help resources, http://www.cf411.com...
To stretch the analogy, while some may well get the sense from their perspective that CF is an island, I'd say it's more like an archipelago (a group of islands), like Hawaii or the Florida Keys, each with a different vibe. Definitely bigger and more varied than a visit to any one island would suggest. :-)
But I will say that we're getting off-topic (my doing here), and also such discussion can bring out the vigor in some who would be otherwise silent. :-) So let's leave it at that, and wish everyone happy holidays.
JAVA_HOME env var?
I have a question regarding your instructions vs. Adobe.
I am trying to update my server with to the latest version.
https://helpx.adobe....
The instruction to do it manually:
1. Download the hotfix installer from the link. (To where?)
2) Download the packages zip file from this link and extract its contents to a location accessible to all ColdFusion server instances.
3) Update "packagesurl" in cfusion/lib/neo_updates.xml of cfusion and all its child instances to point to <InstallerRepositoryUnzippedPath>/bundles/bundlesdependency.json present inside the downloaded folder.
In the neo_updates.xml file these are the items it's mentioning:
<packagesurl>https://www.adobe.co...</packagesurl>
<defaultpackagesurl>https://www.adobe.co...</defaultpackagesurl>
Should these actually be touched?
Also, I couldn't find a reference to the neo file on your page. Does that mean it's not necessary to change?
Any and all info is appreciated.
Thanks
Still, I did tweak a couple of things in the text (including the code shown for the command to use "java -jar" to apply the update using that new location).
At some point I should just do a new version, tightening things up a bit more where I can, and elaborating on anything else that's new since this 2016 post. Hope it's still helpful.