[Looking for Charlie's main web site?]

Don't miss helpful "new" CF feature to identify code causing "implicit" scope searches

Don't miss that Adobe had added a useful feature (a "patch", made available in Apr 2024) to help in identifying any CFML code you may have which refers "implicitly" to scopes that would no longer searched (for any variables without a scope prefix), which is the new default behavior for CF2021, CF2023 and beyond as of the March 2024 updates (updates 13 and 7, respectively).

TLDR; (more on each of these points, in the rest of this post)

  • By following the simple couple of steps, CF will start logging (to a new unscoped.log) whenever code is run that would access an unscoped variable when that would cause CF to implicitly search through scopes (external to the request) which it would no longer search if "searchimplicitscopes" was false. (To be clear, the new logging only works if searchimplicitscopes is true, otherwise such searching would fail if searchimplicitscopes is false, as is the new default as of the March 2024 updates)
  • The "patch" is a jar which you must manually obtain and put into place--it is NOT included with the March 2024 CF update, or any others. The steps are very simple, discussed below or in an Adobe technote that was released in the weeks after the March updates, with the title: View unscoped variables in a log file
  • Note that this patch is also NOT included in the June 2024 CF updates, CF2021 update 14 and CF2023 update 8
  • Further, beware that if you DO apply any update to CF after applying this patch, that update will REMOVE this "patch" (and any jars in the lib/updates folder which is referred to in the technote). Therefore, you would need to put the jar BACK in manually after any such CF update, for it to continue doing its logging
  • Finally, FWIW, note that you can even leverage this patch in the CF two updates PRIOR to the March 2024 updates which introduced the change in the default for searchimplicitscopes, so updates 5/6 and 11/12, respectively. That means someone could also use this patch to test BEFORE moving to either the March 2024 updates or later
Again, more on each of these points below. But for some, the news and the link to the technote (and my couple of tips above) may be all they feel they need to hear. For others, I think more perspective may help, so read on for that.

If you're just hearing about this now...

First, don't feel bad if this may be your first hearing of this "patch". You could easily have missed news of it. Sadly, there was no mention of it added to the CF update technotes for that March updates, CF2021 update 13 and CF2023 update 7 (nor is it mentioned in the subsequent updates in June 2024).

I've been meaning to blog about this new logging capability since it was released in April. This is that post. If it's any consolation, I've learned about the extra bit I'm sharing here either in my own exploration of the "feature" since then or from my helping folks implement it as part of my consulting.

Background

Hopefully by now (if you use CF2021 or CF2023) you have heard how the CF updates for them in March 2024 (updates 7 and 13, respectively) changed the default behavior of searchimplicitscopes from true to false. I have a blog post on that March update with more on the searchimplicitscopes change, if you need more info.

As I explain there (and so do the technotes and Adobe forum announcement about the update in March 2024), one can control the behavior of this feature either via a jvm argument or an application-level setting (which can override the JVM argument). Again, see the technote or my blog post for the details. Or, of course, you could specify the scopes where needed (though that's not even always easily done).

One last things before we move on to the "patch": note that the upshot of this change in behavior is NOT that you "must scope all your variables". Many have misconstrued this as the crux of the change. Again, it's ONLY where an unscoped variable would be resolved by searching through scopes OUTSIDE of the request: cgi, url, form, cookie, file, and client. If instead it could be resolved by searching within scopes within the request (such as variables, local, query, and arguments), you don't NEED to scope the variable.

Setting up the "patch" to log unscoped variables searched for in "implicit" scopes

Again, as elaborated on in the technote about the patch, there are basically 3 steps:

  1. download the patch (a jar) from Adobe. Here are the links to the patch for CF2021, and the patch for CF2023
    • (You'd be forgiven for not finding the patch download link on the technote. It's buried within text that simply refers to the CF versions. You've have to notice the URL goes to a jar. Also, of minor interest: note that the jar file name is unlike any in the past, having the numbers 8888888 and 8888889, respectively, whereas usually the number would equate to a bug report that the update was fixing.)
  2. place the file in the cfusion/lib/updates folder
    • Note finally that it's critical that you do NOT remove the chf*.jar file in the lib/updates. That file was put there by whatever CF update you applied. Removing that would be entirely inappropriate
    • If you are running more than the single default "cfusion" instance of CF, note that you would need to put the jar into the lib/updates folder under that instance (and restart that instance) in order to see the log (and the logs for an instance are stored by default in a logs folder under that instance)
  3. restart the cfusion instance

Once you restart the instance, then (and only then):

  1. if you make a request that tried to refer to an unscoped variable,
  2. and it is resolved by searching in one of those scopes outside the request,
  3. while you have set searchimplicitscopes=true either via the JVM argument or application setting (or while being on the update from Oct 2023, as discussed further below, where the default was still true)

That will cause CF to add a log line in an unscoped.log, which is created if it doesn't exist and stored in the logs folder for the CF instance (cfusion/logs, or [instancename]/logs).

(As an aside, note first that the lines written to this log--like with many CF logs--is also written to the coldfusion-out.log. To be clear, all CF logs are limited in size and number, as configured on the CF Admin "logging settings" page.)

What to expect from the logs

Assuming all the above is true, and a log line is written to the unscoped.log file, it might might look like this:

"Information","http-nio-8523-exec-5","07/17/24","13:59:02","test4","C:\somefolder\somefile.cfm:varname,scope:UrlScope"

Note that it includes: a) the date and time of the request, b) the application name, if any, c) the full file path holding the code that was executed, d) the NAME of the unscoped variable that attempted to access one of the scopes outside the request, and e) the scope in which the variable was found.

Why the new logging capability is helpful but not perfect

As you can see, this is a very helpful feature. With it, you can find if and when requests are executing code that accesses unscoped variables which are resolved in one of the scopes that searchimplicitscopes=false would reject. You don't NEED to just "set it to false and hope you thought to find all the potentially affected code".

That said, note that it does only find such code if it is EXECUTED: this is not a tool that scans all the code within the affected file. As such, it will NOT report "all occurrences of code in a file that would fail".

More than that, this logging feature of course won't help find ALL potentially troublesome code in all the files in your app. For that, consider the feature added by Pete Freitag to his commercial Fixinator tool. Even then, such static scanning is itself very challenging, since again it's not simply "any unscoped variables" that will fail--but only those which at runtime would be resolved by CF not finding the variable in one of the local scopes.

Finally, I'd argue that the logging feature added by this Adobe "patch" could be still MORE helpful: note that the log line does not track the NUMBER of the line that triggered the log line. That can be quite frustrating, as the file may well have MANY references to the variable. Of course, Adobe could argue "don't you want to just find and change all of them?", but in fact there are some situations where that may not be the right solution such that finding the line in error is most important.

There are also occasions where the name listed in the log line might not even be found within the code, such as if there is some means by which the variable name is itself being pulled from a variable (maybe from within a query column, or dynamically from within one of the incoming scopes).

Still, for most people the new capability should make it easier to find code at risk of failing if searchimplicitscopes=false.

And for many, that might seem (finally) "all one needs to know", but as always it's not. :-)

How the "patch" is NOT included in the updates after March 2024

I said it in the tldr; at the opening but it bears repeating: even though this "patch" capability was added in April, following the March updates, do NOT make the mistake of presuming that (like other special hotfixes/patches) it would be "rolled into the next update". I can confirm it was NOT included in the next update, which came out in June (CF2023 update 8 and CF2021 update 14).

We should not expect it will be included in any of the future updates, so you WILL need to add this patch even beyond the March 2024 updates. More than that, see the next section.

Why you will need to apply the "patch" again after each update beyond March 2024

Perhaps more important: note that if you DO apply any future CF updates, you will have to apply this "patch" again. The CF update process has (since CF10) always removed ALL files from the cfusion/updates folder, which has been the place where other such special hotfixes/patches have been placed.

The new update would place this "patch" jar file into the "backup" folder for the update. As such, you could find it there, for example:

D:\ColdFusion2023\cfusion\hf-updates\hf-2023-00008-330668\backup\lib\updates\hf202300-8888889.jar

Simply copy the file out of there, placing it back in the cfusion/lib/updates folder (and/or [instance]/lib/updates), and restart the instance.

Again, this is a process we need to do with ANY such special hotfix for a given update, when the next update doesn't incorporate that fix. And FWIW, this is something I discussed in my 2019 blog post on implementing such special CF hotfixes, which I'd offered on the Adobe CF portal rather than my blog here.

How you can leverage this "patch" even for the two updates BEFORE those from March 2024

Before we conclude, I want to share one more important point that may benefit some.

If you look closely at the technote for this "patch", you will see that Adobe indicates that in fact the patch works not ONLY on updates since March 2024, but also the update before that. It states that: "The patch applies to ColdFusion (2023 release) Update 6 and higher, and ColdFusion (2021 release) Update 12 and higher."

Note that these two updates were released in fact in Oct 2023 (as can be seen, for instance, in the technote for CF2023 update 6.) More than that, I tested it even worked with the update before THAT (so updates 5 and 11, respectively).

What's interesting about all this is that we didn't hear about this until April, and only AFTER the update in March. That suggests Adobe was already working on this matter back then, but just didn't "pull the trigger" on implementing the change in the default value of searchimplicitscopes until that March update.

Even so, I would argue that if the news of this patch had been made available back then, we'd have several months to better "prepare" for this change before it came out. Oh well, it's water under the bridge. I bring it up now for the sake of any folks who MAY well be on updates 5/6 of CF2023 or 11/12 of 2021, who can at least proceed to apply this patch to begin their testing regarding this change regarding searchimplicitscopes.

All that said, don't try using this patch with still-earlier updates: I've confirmed that would lead to an error in CF2023 update 4:

java.lang.NoSuchMethodError: 'org.apache.logging.log4j.Logger coldfusion.log.Logger.getWrappedLogger()'
and this error in CF2021 update 10:
java.lang.NoSuchMethodError: coldfusion.log.Logger.getWrappedLogger()Lorg/apache/logging/log4j/Logger;

Other related topics I may blog about

Finally, I've hinted here at a couple of topics that could benefit from further elaboration, but they're tangents not related to this "patch". I may do a post on these topics (in which case I'd come back here and change these to be links to those other posts).

  • Why this implicit scope searching can be a problem, leading to vulnerabilities or unexpected behavior in your apps
  • More examples, including when you do NOT need to "scope all your variables"
  • Challenges, where it's not even so simple as "scoping your variables"
  • Why you may find that setting searchimplicitscopes=true in application.cfc may not take effect (you're setting it in a method, but it should be set OUTSIDE any method)
  • and any other matters that may arise from questions/comments folks may offer on this or my previous post

I hope that this post better helps you consider and/or implement the "patch" to obtain helpful logging about the searchimplicitscopes matter. Again, I discuss the change in my blog post on the update in March, as well as in comments on the Adobe forum thread from then (which that blog post points to).

But as for this patch, I felt that this post might be helpful given ongoing confusion I find among folks--as well as the fact that this whole matter of the searchimplicitscopes change will trouble people with versions of CF long into the future (including those who will move to new versions and trip over all this then). As always, I just want to help.

And while I certainly welcome questions, comments, or feedback below, do note that if you might like more direct, remote screenshare-based consulting, I offer such help as discussed on my consulting page.

For more content like this from Charlie Arehart: Need more help with problems?
  • If you may prefer direct help, rather than digging around here/elsewhere or via comments, he can help via his online consulting services
  • See that page for more on how he can help a) over the web, safely and securely, b) usually very quickly, c) teaching you along the way, and d) with satisfaction guaranteed
Comments
Copyright ©2024 Charlie Arehart
Carehart Logo
BlogCFC was created by Raymond Camden. This blog is running version 5.005.
(Want to validate the html in this page?)

Managed Hosting Services provided by
Managed Dedicated Hosting