Logging in production with Rails

I use the Rails server logs primarily to dig through a user's interaction with the site across requests to track down intricate bugs, like how did the user manage to get the system into this state which it shouldn't be in.

But using the default Rails logs to do this piece of detective work is horrible.

I'm using the Hodel3000CompliantLogger, which prefixes each log line with the process id, which makes it possible to trace through each individual request.

But I haven't found any tools that help me dig through those large log files yet.

I discovered Loggly today, and I briefly got my hopes up, but it does nothing like what I'm looking for. Or if it is, I haven't noticed. All I can see that it does for me, is aggregate log files from various servers. I only have one server. There's some searching, but it still just shows me the plain old log file.

What I want to see is a list of each request served by Rails, along with the IP address, the user agent, the parameters, info from the session such as logged in user ID, the response code, the redirected to URL if redirected, perhaps processing time, and memory consumption.

Then I want to be able to filter by IP address, so I can follow just one user's path through the system, looking at each request, one by one.

Then I want to be able to dig into each request, one by one, seeing everything my app logged, which includes database statements and my own debugging statements and such.

I think having such a tool would help me tremendously.

Does any such tool exist already?

Possibly a customized Rails logger along with a web-based interface that lets me search and browse my logs.

I really thought Loggly was it, but I can't see that it does the job for me :(

Your insight is greatly appreciated.

Thanks! //Lars

5 comments

Loot at splunk - www.splunk.com I've used it to do some of what you want with application logs.
By Brian Dunbar on Tue, Nov 22, 11 at 19:07 · Reply
  Cancel
Which parts?
By Lars Pind on Wed, Nov 23, 11 at 03:10 · Reply
  Cancel
Splunk is great - you can build it to show the pieces you want, and to follow sessions throughout a log file.
By Joel Parker Henderson on Mon, Nov 28, 11 at 00:50 · Reply
  Cancel
And how would I get started with that? Any tips?
By Lars Pind on Mon, Nov 28, 11 at 02:30 · Reply
  Cancel
I found my perfect solution - mongo_db_logger seems to be exactly what I was looking for: http://blog.philburrows.com/articles/2009/09/28/rails-logging-with-mongodb/ //Lars
By Lars Pind on Sat, Dec 10, 11 at 11:20 · Reply
  Cancel

Leave a comment