I’ve been trying out the demo version of Splunk to aggregate and analyze all the log files messages generated on the servers we used to host the debut of Intersect.com. It seems pretty sweet.
I’m not sure how it works, but it correctly breaks up multi-line log messages from Ruby on Rails. It also makes it pretty easy to pluck specific fields out of arbitrary log files. You pick a type of log, it shows you a couple dozen example entries. You pick some example values from the field you want to isolate and splunk creates a regular expression to match it. If the match is imperfect, you can add other example values, otherwise, you can save the field.
There are still things I’m trying to figure out, but I’m a little reluctant to invest too much in it. The published pricing of $2000/year or $6000 for a perpetual license with 1y of support might be doable, but the lack of visibility into where pricing goes from there gives me serious pause.