How to search Logs with Actual Log Source Time not the indexer time.

[Victor999]

Hi Splunkies,

I am a very new to splunk. I was using HP arcsight. There are two timestamp in HP

1) Manager Receipt time, similar indexer logging time.
2) Another is actual log source time in HP. But while I search logs for last 2 hours in splunk we will get the logs by indexer in last two hours.

However I can define in HP Arcsight, whether to search logs based on Manager Receipt time or Actual event time.

Example why I require.

I have symantec events, for 2 months ago(actual log source time 21st Aug 2016 logs) showing as "Details Pending" but the event has received now by indexer(21st Oct 2016). Now I want to wait for "quarantined" logs. The quarantined logs(for 21st Aug 2016) might be received 1 month back(20th September 2016) by indexer and triggered 1 months ago.

I want to search the logs with actual event time so that I can Co-relate with the actual time that the logs received 2 months ago as "Details pending" is related to the logs recieved 1 month ago as "quarantined".

Kindly let me know if we have this feature to "Search" in Splunk to search based on events received by splunk and also with actual logs.

Regards,
Destiny

[gokadroid]

At search time you can use eval command to pick the timefield from your log and assign it to _time field and then complete your searches on this new log time. Try this.

yourBaseSearch that extracts the field yourLogTimestampField
| eval _time=strptime( yourLogTimestampField, <format of your Log Time>)
| complete the search where results returned will take _time as new time

What this will do is pick up the time from "yourLogTimestampField" field and assign it to _time variable, thereby ensuring anything you write after this eval command in your search picks up the event time as "yourLogTimestampField" .

How to achieve it on a timechart is here:
https://answers.splunk.com/answers/145562/how-to-use-a-field-as-timestamp-for-a-timechart.html

More about strptime and strftime here:
https://answers.splunk.com/answers/80521/time-function.html

More about _time here:
https://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/SearchTimeModifiers

[Victor999]

Thanks Gokaadroid,

[dmaislin_splunk]

Victor, as long as you properly identify the timestamp in the log using the props.conf configurations when you index the file, Splunk will show that actual date of that event using whichever date you specify.

https://docs.splunk.com/Documentation/Splunk/6.5.0/Admin/Propsconf

TIME_PREFIX =
MAX_TIMESTAMP_LOOKAHEAD =
TIME_FORMAT =

You can stitch together events using transaction command is the event has a unique sessionID or something similarly unique and that will roll the events into a nice time based view.

http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Searchfortransactions?r=searchtip

[Victor999]

Thanks dmaislin [Splunk]

[Victor999]

Anyone please help 😞

[richgalloway]

If you have the actual time from the logs as a field in Splunk then you can search on it.

It would help you shared some sample events as seen by Splunk.

[sundareshr]

Are both logs (quarantined and current) in the same index with same source & sourcetype in splunk? Have you tried any queries that you can share?

[lukejadamec]

It would be helpful to post some example events also.

[Victor999]

Anyone please help 😞