What's with HTTP_X_FORWARDED_HOST?

I’ve seen some requests to boxesandarrows.com with HTTP_X_FORWARDED_HOST=myweb2.search.yahoo.com. They go to the right virtual server just fine, but Rails, and most other web app frameworks that I’ve seen, have decided that when HTTP_X_FORWARDED_HOST is present, it trumps HTTP_HOST.

I haven’t been able to find a good resource to tell me how it’s intended to be used, so I can determine whether Yahoo is mistaken, and what the best way to handle it is. It would seem to be used with proxies, but that’s about all I dare guess.

It seems to happen when people save a search for a B&A story to their My Web 2.0 Search, in which case Yahoo sends this weird request from a proxy server of theirs, proxy2.search.scd.yahoo.net.

Can anyone help clarify?

6 comments

Steven Bristol

What kind of server are you running? Apache?
Read more
Read less
  Cancel
lighttpd
Read more
Read less
  Cancel
You might have more luck googling for HTTP_X_FORWARDED_FOR, which seems to be Apache's equivalent. As I understand it it's added by proxies to record the originating client request. However, I personally disagree with the way some frameworks including Rails implicitly favor it -- like anything sent by the client it should be viewed with suspicion. If you're doing any kind of IP-based authentication (even simple things like relying on Rail's local_request?) it could be spoofed by just adding something like 'X-FORWARDED-FOR: 127.0.0.1' to the request headers... On the other hand, the REMOTE_ADDR header is set directly by Apache based on the originating IP address of the request. Not that it's impossible to spoof that of course, but it's a little harder than just adding a simple HTTP header. The above is obviously apache-specific but should be easy to translate into lighttpd
Read more
Read less
  Cancel
Of course, after I wrote that I looked at Rail's implementation of local_request? and it does use request.remote_addr rather than request.remote_ip which will be set to the X-Forwarded-For value if it exists in the reqest headers. So that's good.
Read more
Read less
  Cancel
I'm using this for subdomain-based virtual hosting, as in my backpack account, "larspind.backpackit.com". Rails' @request.host will use HTTP_X_FORWARDED_HOST over REMOTE_HOST if present, and if I just ignore that and always look at REMOTE_HOST, that seems to work. But I haven't been able to trace down the docs for how these are supposed to work, why (or how) Yahoo would choose to set HTTP_X_FORWARDED_HOST, and why Rails (and various Perl modules I found) would choose that over REMOTE_HOST in this case.
Read more
Read less
  Cancel
I may know why, since I'm dealing with this now. I am setting up a site, such as www.public.com. This host needs to proxy the content from another private host. We've setup Apache using something like: <pre>RewriteRule ^/(.*) http://private-server.com:8080/$1 [P]</pre> When requests are received by the private host, HTTP_HOST is set to 'private-server.com', instead of the actual hostname requested by the client ('www.public.com'). Thus, HTTP_X_FORWARDED_HOST is used to pass the hostname that the client requested along to the proxied server. I hope this makes sense.
Read more
Read less
  Cancel

Leave a comment