Hey All,
I enabled the squid app for splunk and threw a log file into it. Pretty quick and easy, and I whipped out an additional dashboard. (Thanks to who put this together)
I noticed an issue, and in my noobness, looking for some direction. When I loaded the log file, splunk recorded 80,000 records loaded at 8:00pm. Well, it's true I loaded them, but I think it should have parsed the timestamp so I can do historical reporting. (Correct me if I'm wrong)
So, I looked at the transform and the regex is:
^\d+\.\d+\s+(\d+)\s+([0-9\.]*)\s+([^/]+)/(\d+)\s+(\d+)\s+(\w+)\s+((?:([^:]*)://)?([^/:]+):?(\d+)?(/?[^ ]*))\s+(\S+)\s+([^/]+)/([^ ]+)\s+(.*)$
format is:
duration::$1 clientip::$2 action::$3 http_status::$4 bytes::$5 method::$6 uri::$7 proto::$8 uri_host::$9 uri_port::$10 uri_path::$11 username::$12 hierarchy::$13 server_ip::$14 content_type::$15
The first field should be timestamp. When looking at squid data in search, "fields" include "timestamp" but it's determine that there are "none".
As a refresher, the log file entries look so:
1301087053.193 182 10.2.40.179 TCP_MISS/400 1083 GET http://api.twitter.com/1/statuses/user_timeline.json? username DIRECT/199.59.148.87 application/json
My regex-foo is weak, and I'm definitely below average. However, shouldn't this include the timestamp in order for splunk to index it by time properly?
So, I want to load last months data, but I will not be able to report on February 2011, because it appears to be all new data as of the load data.
Thanks for the advice. Moving forward, the records are correct. Obviously, splunk is doing it's own timestamp.
... View more