topic Re: response time from syslog in Splunk Search

response time from syslog

xiaoyuew — Thu, 11 Aug 2011 22:16:05 GMT

how to calculate response time from syslog? which field to use?

Jun 4 04:02:18 vmlbsmt logger: 10.10.10.10 [04/Jun/2011:04:02:18 +0000] "GET /status.html HTTP/1.0" 200 35 174 "-" "-"

Thanks!

Re: response time from syslog

Ayn — Fri, 12 Aug 2011 05:30:08 GMT

Please clarify. Do you mean which field in the sample event? Syslog is just the means of transporting the event from the source host to a log server.

Re: response time from syslog

xiaoyuew — Fri, 12 Aug 2011 18:33:50 GMT

@Ayn, my question is actually in two folds,

(1) what log format is it? what is in each field?
Jun 4 04:02:18 vmlbsmt logger: 10.10.10.10 [04/Jun/2011:04:02:18 +0000] "GET /status.html HTTP/1.0" 200 35 174 "-" "-"
(2) how to compute response time based on these fields? @Mus mentioned to use - , but where can i find "endtime", and "starttime"?

Thanks again.

Re: response time from syslog

mikelanghorst — Fri, 12 Aug 2011 19:13:53 GMT

xiaoyuew - Your question really isn't about syslog in this case, but in the formatting of the log messages in your webserver.

Your message consists of 2 parts:
Jun 4 04:02:18 vmlbsmt logger - This is written by syslog
10.10.10.10 [04/Jun/2011:04:02:18 +0000] "GET /status.html HTTP/1.0" 200 35 174 "-" "-" - This is sent by your webserver to the syslog daemon, which adds it's info and writes the message.

Re: response time from syslog

mikelanghorst — Mon, 28 Sep 2020 09:48:01 GMT

Depending on how you're getting the data into Splunk, it's possible to use the strip syslog function to remove the portion written by syslog and have the event a "pure" access message.

http://httpd.apache.org/docs/current/logs.html - Under "Access Logs" describes the default format of access_common or access_combined.

It looks like the format of the log has been modified from these standard formats, which neither usually contain a response time.

Re: response time from syslog

xiaoyuew — Fri, 12 Aug 2011 20:08:56 GMT

@mikelanghorst, Thanks so much for ur explanation. Currently i am manually copying the files into the Splunk, so just assume that I can have the apache log part. then i guess i should ask, how to compute response time based on the standard apache logs format? if there is no field recording response time directly?

Re: response time from syslog

mikelanghorst — Fri, 12 Aug 2011 21:08:13 GMT

By default, no. But your example has additional fields. I'd need to see how your logging is configured. Look for lines similar to what's in Fedora's default httpd.conf:

CustomLog logs/access_log combined

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

LogFormat "%h %l %u %t \"%r\" %>s %b" common

LogFormat "%{Referer}i -> %U" referer

LogFormat "%{User-agent}i" agent

The first line is telling Apache where and which format to use, the others define those format names.

Re: response time from syslog

mikelanghorst — Fri, 12 Aug 2011 21:14:33 GMT

http://httpd.apache.org/docs/current/mod/mod_log_config.html#formats

One of these values would need to be in your LogFormat as mentioned above in my comments:

%D The time taken to serve the request, in microseconds. or %T The time taken to serve the request, in seconds.

http://httpd.apache.org/docs/current/mod/mod_log_config.html#LogFormat