Select Page

Considering A Stale-While-Revalidate Pattern To Caching In ColdFusion

Cyberdime
Published: June 21, 2022

In a recent episode of Syntax.fm, Scott and Wes discussed HTTP caching headers. From their discussion, I learned that there is am experimental value called, stale-while-revalidate. And, while this post isn’t about HTTP caching, their discussion got me thinking about different ways in which I might manage a server-side cache of data that needs to be kept in-sync with a remote data source. I wanted to do some noodling of what a “stale while revalidate” workflow might look like in ColdFusion.

The concept, “stale while revalidate”, usually means that your request to read a value is served immediately from a cache. Then, a background non-blocking request is spawned to fetch the latest value from the “source of truth”. This allows subsequent requests for the value to receive the most up-to-date version. This allows all requests for the value to be extremely fast since they are all fulfilled using locally-cached data.

the Lucee CFML doc:

Note, the type “synchronized” is no longer supported and will be ignored; all struct/scopes are “thread safe” since version 4.1.

Array are, apparently, also synchronized by default; though, I’ve definitely hit an Array iteration deadlock in earlier versions of Lucee CFML. And, fun fact: I just learned while writing this post that arrayNew() allows you to pass in an argument for creating non-synchronized arrays.

That said, data-structure access isn’t the only potential reason for synchronization; but, getting into all the reasons you might want to lock access is beyond the scope of this post.

run parallel iterations over Arrays.

That said, the ergonomics of runAsync() feel a bit lighter; though, I don’t actually know if there’s much of a technical difference between the task runner for this function and the task runner for the CFThread tag. In this exploration, I chose runAsync() specifically because it returns a value (a Future) that I can store. This makes it easier to know if I’ve already requested a background refresh.

ASIDE: The CFThread tag stores a reference to the spawned thread in the cfthread scope. As such, I could have implemented this experiment with a CFThread tag; but, it’s just easier to use the returned value from runAsync().

To keep things simple, my cached value is going to be a Boolean. And, our background cache revalidation will simply flip the Boolean value. I’m going to scope the revalidation to the current ColdFusion Request. Meaning, only the first read of the value in a given ColdFusion Request will spawn the background check – subsequent reads of the value in the same request will just read from the cache and will trigger no other actions.

Here’s my demo ColdFusion component, Flag.cfc. It only exposes a single method – getValue() – which returns the currently-cached value and then spawns the background cache revalidation:

component
	output = false
	hint = "I provide access to a flag value which is refreshed asynchronously."
	{
	/**
	* I initialize the flag value.
	*/
	public void function init() {
		variables.currentValue = false;
	}
	// ---
	// PUBLIC METHODS.
	// ---
	/**
	* I get the CURRENT flag value, which may be STALE. Accessing the flag value MAY
	* trigger a BACKGROUND FETCH to re-cache the latest value.
	*/
	public boolean function getValue() {
		revalidateInBackground();
		// NOTE: For the sake of this demo, we're going to consider this value to be a
		// thread-safe value. As such, we're not going to apply any locking around its
		// access despite it being cached in-memory and shared across requests.
		return( currentValue );
	}
	// ---
	// PRIVATE METHODS.
	// ---
	/**
	* I spawn an asynchronous thread to revalidate the flag in the background.
	*/
	private void function revalidateInBackground() {
		var futureKey = "$$flagAsyncFuture";
		// Only run the async validation once per request (an optimization).
		// --
		// NOTE ON LOCKING: Normally, when I only want to do something once, I would add a
		// double-check lock around it. However, in this case, since the scope of the
		// FETCH contention is a single request, I'm not going to worry about it. More
		// than likely, there will be no race-condition (again, this is scoped to the
		// request); and, even if there is contention in this case, the worst-case
		// scenario is that we run the asynchronous check more than once, which is fine.
		if ( request.keyExists( futureKey ) ) {
			return;
		}
		try {
			// CAUTION: We do not want to use the `.error()` method on the runAsync()
			// result because doing so will turn it from an ASYNCHRONOUS call into a
			// BLOCKING / SYNCHRONOUS call. Instead, we're going to use a try/catch block
			// inside the closure so that we can retain the asynchronicity while still
			// catching errors internally.
			request[ futureKey ] = runAsync(
				() => {
					try {
						systemOutput( "Revalidating in runAsync() closure.", true );
						// !! Simulating some sort of background flag change. !!
						variables.currentValue = ! variables.currentValue;
					} catch ( any error ) {
						systemOutput( "ERROR REVALIDATING FLAG!!!!" );
					}
				}
			);
		// Catch any errors when spawning the thread that powers the runAsync() function.
		// --
		// CAUTION: I AM NOT SURE that this is strictly necessary. I don't know how the
		// thread-pool exhaustion will manifest in the calling context. That said, I
		// usually wrap my thread-spawning code in a try/catch block since I know that the
		// CFThread tag will sometimes throw an error when no thread can be spawned.
		} catch ( any error ) {
			// Swallow thread-pool exhaustion errors for now...
			systemOutput( "ERROR: runAsync() could not obtain thread.", true, true );
		}
	}
}

As you can see, in my revalidateInBackground() method, I’m checking to see if the Future has already been stored in the request scope; and, if so, I short-circuit the method. This should ensure that – in vast majority of cases – only a single background refresh will be performed in a given ColdFusion request.

The error handling here is a bit of blind-spot for me. I know that I can’t add the .error() handler since that will turn the runAsync() workflow into a blocking request, which is exactly what we don’t want. I’ve attempted to cover my bases by wrapping both the thread-spawning code and the internal operator code in a try/catch.

That said, assuming that I’ve cached this ColdFusion component in the application scope, let’s look at the console logging when we make several calls to the getValue() method within a single request:

<cfscript>
	systemOutput( "+ + + + + + + + + + + + + + + ", true );
	systemOutput( "Starting Demo Request.", true );
	systemOutput( "Checking flag...", true );
	systemOutput( "--> Flag: [ #application.flag.getValue()# ]", true );
	systemOutput( "Checking flag...", true );
	systemOutput( "--> Flag: [ #application.flag.getValue()# ]", true );
	systemOutput( "Checking flag...", true );
	systemOutput( "--> Flag: [ #application.flag.getValue()# ]", true );
	systemOutput( "Ending request.", true );
	systemOutput( "- - - - - - - - - - - - - - - ", true );
</cfscript>

Because we are caching the runAsync() Future in the request scope, only one of the preceding 3 calls should spawn a background cache revalidation. And, in fact, when we run this ColdFusion code, we get the following logs:

ColdFusion logs showing that the background cache revalidation only runs once per request.

As you can see, for each ColdFusion request, we only see one log from within the runAsync() callback, despite the fact that we’re making 3 calls to the getValue() method in each request. We can also see the the value of the flag is being flipped on each request, but not between getValue() calls. This is because it’s being served from the local cache for the duration of the request (mostly).

CAUTION: Even though it is working out this way in my demo, the runAsync() doesn’t necessarily run at the end of the current request. It just doesn’t block. If other aspects of your request block (such as a database query), there’s a distinct possibility that your runAsync() callback will be invoked before the end of the parent request. As such, there’s a possibility that the state of the cached value will be “revalidated” within the bounds of a single request.

Again, to be clear, I’ve never done this in a production scenario before. I’m just starting to noodle on what this would look like in a ColdFusion application. It’s a bit challenging to think through the details without a real-world use-case. But, sometimes you gotta start with the building blocks before you can assemble the larger concepts.

Check out the license.

Source: www.bennadel.com