Select Page

Trying To Get The Most Trustworthy IP Address For A User In ColdFusion

Cyberdime
Published: October 13, 2022

On a recent Penetration Test (PenTest), one of our systems was flagged for not properly validating the X-Forwarded-For HTTP header, which is a recording of the various IP addresses along the network path being made by an inbound request. To be honest, I’ve never really thought deeply about IP addresses from a security standpoint before; but, having this show up on a PenTest sent me down a bit of a rabbit hole. And, I thought it might be worth talking a bit about why IP addresses pertain to security in ColdFusion.

When I first started programming in ColdFusion, everything lived on a single server. And, in those days, using the cgi.remote_addr to access the user’s IP address was simple and safe – this CGI value was not a value that could be easily spoofed (as I understood it). As such, using the cgi.remote_addr for things like audit logging and rate limiting in ColdFusion was straightforward.

But, when we moved our ColdFusion server behind a load-balancer for the first time, things started breaking. This is because the load-balancer sat in between the user and the ColdFusion server and changed the value being stored in cgi.remote_addr: it was no longer the user’s IP address, it was the load-balancer’s IP address.

You can imagine that this immediately tripped all rate-limiting circuit breakers since all inbound request traffic appeared to be coming from just a handful of (internal) IP addresses.

To fix that issue, we started using the X-Forwarded-For HTTP header. This HTTP header is a comma-delimited list of IP addresses, starting with the user’s IP address and followed by each subsequent reverse proxy’s IP address.

And, I haven’t given IP addresses too much thought since then. Until this Penetration Test. And when I started to read-up on using and validating X-Forwarded-For headers, I came across this amazingly in-depth article on “real” IP addresses by Adam Pritchard (March 2022). In that article, Pritchard states that the X-Forwarded-For header value should be considered a user-provided value and therefore cannot be trusted.

The cautions in his article have since been included in the Mozilla Developer Network (MDN) docs on X-Forwarded-For:

This header, by design, exposes privacy-sensitive information, such as the IP address of the client. Therefore the user’s privacy must be kept in mind when deploying this header.

The X-Forwarded-For header is untrustworthy when no trusted reverse proxy (e.g., a load balancer) is between the client and server. If the client and all proxies are benign and well-behaved, then the list of IP addresses in the header has the meaning described in the Directives section. But if there’s a risk the client or any proxy is malicious or misconfigured, then it’s possible any part (or the entirety) of the header may have been spoofed (and may not be a list or contain IP addresses at all).

If any trusted reverse proxies are between the client and server, the final X-Forwarded-For IP addresses (one for each trusted proxy) are trustworthy, as they were added by trusted proxies. (That’s true as long as the server is only accessible through those proxies and not also directly).

Any security-related use of X-Forwarded-For (such as for rate limiting or IP-based access control) must only use IP addresses added by a trusted proxy. Using untrustworthy values can result in rate-limiter avoidance, access-control bypass, memory exhaustion, or other negative security or availability consequences.

Conversely, leftmost (untrusted) values must only be used where there will be no negative impact from the possibility of using spoofed values.

For someone like me who doesn’t spend time thinking about how networks and proxies work, this language is a bit hard to parse. But, my understanding is that the X-Forwarded-For header is inherently untrustworthy; and, that we should only trust the value being provided by the first trusted proxy in the network hops.

Thankfully, Pritchard goes on in his article to talk about the different types of proxies that are commonly in use; and, how we – as ColdFusion developers – can lean on the various technical choices that said proxies are making in order write more secure code.

At work, our entire system sits behind Cloudflare. And, as Pritchard states in his article, Cloudflare injects an HTTP header value on inbound requests that we can consider “trusted”:

Let’s start with some good news.

Cloudflare adds the CF-Connecting-IP header to all requests that pass through it; it adds True-Client-IP as a synonym for Enterprise users who require backwards compatibility. The value for these headers is a single IP address. The fullest description of these headers that I could find makes it sound like they are just using the leftmost XFF (X-Forwarded-For) IP, but the example was sufficiently incomplete that I tried it out myself. Happily, it looks like they’re actually using the rightmost-ish.

After reading this, I went to look up the Cloudflare docs on HTTP headers, and found the same concept:

CF-Connecting-IP provides the client IP address connecting to Cloudflare to the origin web server. This header will only be sent on the traffic from Cloudflare’s edge to your origin web server.

What I believe this all means is that the CF-Connecting-IP HTTP header value is essentially the cgi.remote_addr as seen from the CDN’s perspective. Which means, it should be the non-spoofable IP address of whatever machine is connecting to the CDN.

Taking this information, I updated our ColdFusion (Lucee CFML) logic to give priority to the CF-Connecting-IP HTTP header if it is present. And, since we have to deal with Staging and Local development environments that don’t sit behind Cloudflare, falling back to older means of access then user’s IP address:

NOTE: In the following code, I include the use of the Java Commons IP Math library to actually try and parse the inbound IP address. I included that for funzies using Lucee’s ability to load JAR files on the fly; but, in production, I just use the “light-weight validation”.

<cfscript>
	echo( "Hello from IP: #getRequestIpAddress()#" );
	// ------------------------------------------------------------------------------- //
	// ------------------------------------------------------------------------------- //
	/**
	* I get the most appropriate IP address for the current, incoming HTTP request.
	*/
	public string function getRequestIpAddress() {
		// In Production, we are sitting behind the Cloudflare CDN. Cloudflare injects the
		// header, "CF-Connecting-IP", which is (according to their docs) "the client IP
		// address connecting to Cloudflare". Since Cloudflare is a "trusted proxy", this
		// value represents the most trustworthy IP address in the request chain. If this
		// header is available, it should take precedence.
		var ipAddress = getHeaderValueSafely( "CF-Connecting-IP" );
		// In non-production environments, we are NOT behind the Cloudflare CDN; but, we
		// are behind several proxies (load balancers, nginx, etc). If the Cloudflare
		// header isn't available, fallback to the "X-Forwarded-For" proxy-injected value.
		// --
		// CAUTION: This might be a USER-PROVIDED VALUE and should be consumed with much
		// caution. A malicious actor might provide this header in a crafted request; and,
		// most proxies are designed to just accept-and-augment any existing header.
		if ( ! ipAddress.len() ) {
			ipAddress = getHeaderValueSafely( "X-Forwarded-For" )
				.listFirst()
				.trim()
			;
		}
		// If no header-based IP address is available, fallback to using the remote IP
		// address.
		// --
		// CAUTION: Depending on your server configuration, this value may be an automatic
		// reflection of one of the header-based IP values. For example, Tomcat can be
		// configured to use the "X-Forwarded-For" header to populate CGI.remote_addr. As
		// such, this value may also be considered a USER-PROVIDED value.
		if ( ! ipAddress.len() ) {
			ipAddress = cgi.remote_addr;
		}
		// Since we may be dealing with a user-provided value, we need to apply some
		// validation, making sure the provided IP address looks like an IP address. If
		// any of the validation fails, we'll return this fallback IP.
		var fallbackIp = "0.0.0.0";
		// LIGHT-WEIGHT VALIDATION: The longest valid IPv6 address should be no longer
		// than 45-characters (from what I've read).
		if ( ipAddress.len() > 45 ) {
			return( fallbackIp );
		}
		// LIGHT-WEIGHT VALIDATION: Make sure that the IP-address (IPv4 or IPv6) contains
		// only the expected characters. This doesn't validate formatting - only that all
		// of the characters are in the expected ASCII range.
		if ( ipAddress.reFindNoCase( "[^0-9a-f.:]" ) ) {
			return( fallbackIp );
		}
		// HEAVY-WEIGHT VALIDATION: We could parse the IP address and make sure that it is
		// semantically valid. However, since the IP address is already not to be trusted,
		// I am not sure that the overhead of parsing it holds much value-add.
		if ( ! isValidIpFormat( ipAddress ) ) {
			return( fallbackIp );
		}
		return( ipAddress.lcase() );
	}

	/**
	* I get the HTTP header value with the given name; or, an empty string if none exists.
	*/
	public string function getHeaderValueSafely( required string name ) {
		var headers = getHttpRequestData( false ).headers;
		if ( ! headers.keyExists( name ) ) {
			return( "" );
		}
		var value = headers[ name ];
		// While the vast majority of headers are simple string values, the specification
		// allows for multiple headers (with the same name) to be provided in an HTTP
		// request. ColdFusion combines those like-named headers as an index-based (ie,
		// Array-Like) Struct. This UDF makes an arbitrary decision to collapse the header
		// value into a list in such cases. Workflows that are expecting complex headers
		// in the request need to use a non-generic solution.
		if ( ! isSimpleValue( value ) ) {
			value = headerValueStructToList( value );
		}
		return( value.trim() );
	}

	/**
	* I convert the given complex HTTP header value to a collapsed, comma-delimited list.
	*/
	public string function headerValueStructToList( required struct value ) {
		var items = [];
		var size = value.size();
		for ( var i = 1 ; i <= size ; i++ ) {
			items.append( value[ i ] );
		}
		return( items.toList( ", " ) );
	}

	/**
	* I use the Commons IP Math library to parse the given IP address (thereby validating
	* that it is in the proper format).
	* 
	* Commons IP Math: https://github.com/jgonian/commons-ip-math
	*/
	public boolean function isValidIpFormat( required string input ) {
		var jarPaths = [ "./commons-ip-math-1.32.jar" ];
		var IpClass = input.find( ":" )
			? createObject( "java", "com.github.jgonian.ipmath.Ipv6", jarPaths )
			: createObject( "java", "com.github.jgonian.ipmath.Ipv4", jarPaths )
		;
		// There is no validation method on the IP classes. As such, we simply have to try
		// and parse the input value and see if an error is thrown.
		try {
			IpClass.parse( input );
			return( true );
		} catch ( any error ) {
			return( false );
		}
	}
</cfscript>

As you can see in the getRequestIpAddress() method, I am giving precedence to the Cloudflare header first; then falling back to the other IP address values if the Cloudflare one doesn’t exist.

In Prichard’s article he warns against having generic fallbacks:

A default list of places to look for the client IP makes no sense

Where you should be looking for the “real” client IP is very specific to your network architecture and use case. A default configuration encourages blind, naive use and will result in incorrect and potentially dangerous behavior more often than not.

If you’re using Cloudflare you want CF-Connecting-IP. If you’re using ngx_http_realip_module, you want X-Real-IP. If you’re behind AWS ALB you want the rightmost-ish X-Forwarded-For IP. If you’re directly connected to the internet, you want RemoteAddr (or equivalent). And so on.

There’s never a time when you’re okay with just falling back across a big list of header values that have nothing to do with your network architecture. That’s going to bite you.

But, my fallbacks aren’t generic. They are specifically looking for and giving highest precedence to the CDN that we use at work.

To test this, I tried making a local HTTP request to this above ColdFusion script using various combinations of header:

<cfscript>
	// Try with both HTTP headers.
	http
		result = "apiResponse"
		method = "get"
		url = "http://127.0.0.1:57833/x-forward-for/target.cfm"
		{
		httpParam
			type = "header"
			name = "CF-Connecting-IP"
			value = "101.101.101.101"
		;
		httpParam
			type = "header"
			name = "X-Forwarded-For"
			value = "202.202.202.202"
		;
	}
	echo( apiResponse.fileContent & "<br />" );
	// ------------------------------------------------------------------------------- //
	// ------------------------------------------------------------------------------- //
	// Try with one fallback header.
	http
		result = "apiResponse"
		method = "get"
		url = "http://127.0.0.1:57833/x-forward-for/target.cfm"
		{
		httpParam
			type = "header"
			name = "X-Forwarded-For"
			value = "202.202.202.202"
		;
	}
	echo( apiResponse.fileContent & "<br />" );
	// ------------------------------------------------------------------------------- //
	// ------------------------------------------------------------------------------- //
	// Try with no fallback headers.
	http
		result = "apiResponse"
		method = "get"
		url = "http://127.0.0.1:57833/x-forward-for/target.cfm"
	;
	echo( apiResponse.fileContent & "<br />" );
</cfscript>

And, when we run this ColdFusion code, we get the following output:

Three IP addresses being pulled from CF-Connecting-IP, X-Forwarded-For, and CGI.remote_addr, respectively, in Lucee CFML

As you can see, when the CF-Connecting-IP is present, it takes precedence and is used as the “trusted” IP address for the inbound request. But, when that header is no present (which it won’t be in non-production environments), we fallback to the X-Forwarded-For header and the cgi.remote_addr value.

In the world of web development, security is a never-ending journey. And, now that I’m using Cloudflare’s CF-Connecting-IP as the source of truth for inbound IP addresses in production, I feel like I took a baby-step forward in securing our ColdFusion application.

Lucee Dev Forum post on Docker. Apparently, in Tomcat, you can configure the server to use the X-Forwarded-For header to drive the cgi.remote_addr value:

<Valve
	className="org.apache.catalina.valves.RemoteIpValve"
	remoteIpHeader="X-Forwarded-For"
	requestAttributesEnabled="true"
/>

Now, I don’t know anything about low-level server stuff. So, I don’t even know where this “Valve” is. But, given everything that I’ve read recently about IP addresses and security, this setting seems … not great. Meaning, it’s essentially taking the user-provided / untrusted X-Forwarded-For HTTP header and jamming it in what I have historically thought of as a trusted value, cgi.remote_addr. That’s gotta be a bad idea, right?

I’ll be discussing this setting with our security team.

Want to use code from this post?
Check out the license.

Source: www.bennadel.com