topic Re: To display actual logs time by using Timechart command in Splunk Search

To display actual logs time by using Timechart command

Sagar0511 — Mon, 27 Nov 2017 12:31:57 GMT

Hi Everyone

I am trying to create a timechart report and I want to display the Output of the Log event time field instead of _time which is uploaded event time. I tried with the timechart command but it couldn't work. I think by default it takes the field "_time". I have tried rename the logs Time(extarcetd from the Logs) to Time(Actual time of Logs) by the command "eval _time=Time".

Find the snapshot for the sample Log file data

Re: To display actual logs time by using Timechart command

kamlesh_vaghela — Mon, 27 Nov 2017 12:41:38 GMT

Hi @Sagar0511,
Can you please provide more details?

Re: To display actual logs time by using Timechart command

DalJeanis — Mon, 27 Nov 2017 17:31:15 GMT

What, specifically, do you want to display, and why do you want to use timechart?

Timechart is really great for summarizing the flow of events, but it's just not usable for exact time data.

Re: To display actual logs time by using Timechart command

Sagar0511 — Tue, 28 Nov 2017 05:44:23 GMT

apologies... edited my original post now to show more details (formerly hidden in image tag)

Re: To display actual logs time by using Timechart command

niketn — Tue, 28 Nov 2017 05:51:17 GMT

@Sagar0511, can you add raw sample event data (mock/anonymize any sensitive information). Also tell us in the raw event as to what is the log time. Seems like your logs may have two time stamps and your props.conf setting is using the incorrect field as event timestamp or _time, which you would need to rectify. Share your props.conf will also be helpful.

Re: To display actual logs time by using Timechart command

Sagar0511 — Tue, 28 Nov 2017 06:26:44 GMT

The device is not sending the logs directly to splunk server. Instead i have a csv log file which i let rsyslog (on another ubuntu system) send to the splunk server. Hence the _time value is the rsyslog transmit time, whereas the Time is the actual log timestamp.

Sample log (1 event) below:
<133>Oct 23 07:25:25 ubuntu CPFW, 217,26Oct2017,23:59:00,eth1-02,10.2.2.189,Log,Accept,53,54080,10.28.0.16,165.21.100.88,udp,203,,203-CBIG-SIN-Consolidation,,service_id: domain-udp,Security Gateway/Management,,

rsyslog time is Oct 23 07:25:25 = _time

actual log time is 23:59:00 = Time

I have used field extraction feature of splunk to specify the comma delimited nature of the log. The result of the field extraction is shown in my original post.

Below props.conf file from Splunk/etc/system/local

[Hostnames]
DATETIME_CONFIG = 
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

[CBIG-SIN_Log1 Updated]
DATETIME_CONFIG = 
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

[csv]
DATETIME_CONFIG = 
FIELD_DELIMITER = space
FIELD_QUOTE = "
NO_BINARY_CHECK = true
disabled = false

[CBIG_SING_Log1]
DATETIME_CONFIG = 
FIELD_DELIMITER = space
FIELD_QUOTE = "
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

[test1]
DATETIME_CONFIG = 
FIELD_DELIMITER = ,
FIELD_QUOTE = "
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

[test]
DATETIME_CONFIG = 
FIELD_DELIMITER = ,
FIELD_QUOTE = "
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

[CBIG_SIN]
DATETIME_CONFIG = 
FIELD_DELIMITER = space
FIELD_QUOTE = "
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

[cbig_sin]
DATETIME_CONFIG = 
FIELD_DELIMITER = space
FIELD_QUOTE = "
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

[access_combined1]
DATETIME_CONFIG = 
FIELD_DELIMITER = ,
FIELD_QUOTE = "
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

edit: sourcetype for the events we are referring in this question is 'cplogs'.. which can't be seen in props.conf

Re: To display actual logs time by using Timechart command

niketn — Tue, 28 Nov 2017 06:44:26 GMT

@Sagar0511, is your event timestamp supposed to be 26Oct2017,23:59:00. Does your CSV file have a header? If so, what are these field names called? Which stanza in the props.conf applies to the above event? It should be the same as the sourcetype, that Splunk Search displays when you search raw data.

Re: To display actual logs time by using Timechart command

Sagar0511 — Tue, 28 Nov 2017 13:24:16 GMT

Yes, the event timestamp is 26Oct2017,23:59:00. The header is present in the csv log file but I have extracted the field names by doing field extraction; so in that there is no need of headers. There is no cplogs(Sourcetype) mentioned in the props.conf which has been uploaded in the previous post.

Re: To display actual logs time by using Timechart command

woodcock — Sat, 02 Dec 2017 03:45:25 GMT

Try this:

... | eval DateTime = Date . " " . Time
| eval _time = strptime(DateTime, "%d%b%Y %H:%M:%S")
| timechart foo bar blah

Re: To display actual logs time by using Timechart command

niketn — Sat, 02 Dec 2017 17:00:30 GMT

@Sagar0511, I was trying to see a feasibility of getting Date and Time fields from CSV clubbed as _time (event time) at the time of indexing itself using props.conf. So that you dont have to put additional load for the same at Search Time. However, if you are performing a Field Extraction during Search Time, then you can try @woodcock 's answer.

Re: To display actual logs time by using Timechart command

Sagar0511 — Wed, 06 Dec 2017 05:42:30 GMT

The Query had successfully executed and desired result has been achieved. Thank you very much.