Hi All, Currently we are facing an problem in time stamp for a Symantec log data.
Problem: When we search with the below query, we could see that the splunk _time field is different from the event's "time" field.
Query details:
index=sem sourcetype="symantec:tap:incidents" time="2017-12-04T17:19:06.606Z"
Event detail:
12/4/17
12:26:52.000 PM
{ [-]
tap_host: 10.140.37.7
tap_incident_id: 104649
deviceUid: [ [+]
]
device_time: 2017-12-04T17:19:06.606Z
domainId: [ [+]
]
event_count: 6
filehash: [ [+]
]
first_event_seen: 2017-12-04T16:30:08.000Z
last_event_seen: 2017-12-04T17:10:37.000Z
log_name: epmp_incident-2017-12-04/incident
priority_level: 1
recommended_action: If this site is not business critical, consider adding it to the Blacklist. Otherwise, consider creating a sinkhole server in your DNS to block the site.
state: 1
summary: Multiple malicious behaviors have been detected from xxxx.sharepoint.com.
time: 2017-12-04T17:19:06.606Z
updated: 2017-12-04T17:19:07.193Z
uuid: 3ba258e0-d917-11e7-e89d-00000000005a
}
Show as raw text
eventtype = nix_errors error host = splunk01.xxxx.com source = symantec_tap sourcetype = symantec:tap:incidents
From the Event Action, I could see that in the event time field "2017-12-04T17:19:06.606Z" and in the _time field as "2017-12-04 12:26:52" for the same event, "_time" is not equal to "time".
_time is being calculated based on when it was indexed instead of when it was an event.
Props.conf details: We have placed this configuration in Heavy forwarder where the data first reaches the splunk then gets ingested into indexer.
[symantec:tap:incidents]
TRUNCATE = 0
TIME_PREFIX=time:\s
TIME_FORMAT=%FT%T.%3N
MAX_TIMESTAMP_LOOKAHEAD=32
TZ=EDT
Question :
How to make the _time field be the same as the time field ?
Kindly guide me on this.
The event as you posted it is in syntax highlighted
view, not raw
, so the patterns you used in your props.conf are likely incorrect. Try this:
TIME_PREFIX=time":\s*"
The above is based on my assumption that your raw data actually looks more like "time": "<timestamp>"
If that doesn't work please add the raw event, instead of the syntax highlighted representation.
It seems like the timestamp extraction is setup (based on props.conf entry) to take time starting from time:\s
, but it may not be working as it appear towards end of a large events. What the number of character into the event your time field appears?
This is what @somesoni2 is referring to MAX_TIMESTAMP_LOOKAHEAD=32
this is set too low and therefore Splunk never reaches the correct time stamp.
MAX_TIMESTAMP_LOOKAHEAD
is relative to the position determined by TIME_PREFIX
, so the initial setting of 32 should not have been too low to be valid.
MAX_TIMESTAMP_LOOKAHEAD = <integer>
* Specifies how far (in characters) into an event Splunk should look for a
timestamp.
* This constraint to timestamp extraction is applied from the point of the
TIME_PREFIX-set location.
* For example, if TIME_PREFIX positions a location 11 characters into the
event, and MAX_TIMESTAMP_LOOKAHEAD is set to 10, timestamp extraction will
be constrained to characters 11 through 20.
* If set to 0, or -1, the length constraint for timestamp recognition is
effectively disabled. This can have negative performance implications
which scale with the length of input lines (or with event size when
LINE_BREAKER is redefined for event splitting).
* Defaults to 150 (characters).
Hi somesoni2, thanks for your effort, but could you please guide me how much value should be set in-order to correlate the _time stamp with the event time field.
thanks in advance.
@Hemnaath, Is there a difference between device_time and time?
device_time: 2017-12-04T17:19:06.606Z
time: 2017-12-04T17:19:06.606Z
If not you can just switch to device_time
TIME_PREFIX=device_time:\s
MAX_TIMESTAMP_LOOKAHEAD=128
time:
data is pushed towards the end of your event and you seem to have couple of unexpanded JSON Nodes like deviceUid and filehash etc. So it is better you fetch from the first time field which gives the correct time for your event.
For Time Stamp Lookahead you can try some higher number like 128,1024 etc depending on how many characters including whitespaces you might have in your data prefixing timestamp to be extracted.
Hi Niketnilay, thanks for your effort on this, I will update the props.conf with the above mentioned stanza but I am not sure about how/ from where to calculate the value for MAX_TIMESTAMP_LOOKAHEAD=128 from the events. Kindly guide me on this.
[symantec:tap:incidents]
SHOULD_LINEMERGE = false
FIELDALIAS-event_host = tap_host as event_host
KV_MODE = json
TRUNCATE = 0
TIME_PREFIX=device_time:\s
TIME_FORMAT=%FT%T.%3N
MAX_TIMESTAMP_LOOKAHEAD=128
TZ=EDT
Hi Niketnilay, hey I tried the above stanza what you had mentioned in your comment but it did not work. I am getting the same output, so can you please guide me to fix this issue.
index=sem sourcetype="symantec:tap:incidents"
Event details captured after changing the props.conf
12/5/17
11:05:28.000 PM
{ [-]
atp_host: 10.x.x.x
atp_incident_id: 104656
deviceUid: [ [+]
]
device_time: 2017-12-06T04:03:08.713Z
domainId: [ [+]
]
event_count: 1
first_event_seen: 2017-12-05T09:49:58.118Z
last_event_seen: 2017-12-05T09:49:58.118Z
log_name: epmp_incident-2017-12-06/incident
priority_level: 1
recommended_action: Consider blacklisting the site. In addition, you may need to investigate the source of the exposure to see if further action is required.
state: 1
summary: Malicious domain www.xxxx.com detected
time: 2017-12-06T04:03:08.713Z
updated: 2017-12-06T04:03:08.950Z
uuid: 5e89b190-da3a-11e7-d301-000000000061
thanks in advance
Hi All, Can anyone guide me on this, how to make the _time field be the same as the time field in the event.
thanks in advance.
Hi All, still facing the issue, unable to make the _time field be the same as the time field in the event.
Kindly guide me on this.