topic Re: How to fix a timestamp issue for Symantec logs? in Getting Data In

How to fix a timestamp issue for Symantec logs?

Hemnaath — Tue, 29 Sep 2020 17:08:05 GMT

Hi All, Currently we are facing an problem in time stamp for a Symantec log data.
Problem: When we search with the below query, we could see that the splunk _time field is different from the event's "time" field.

Query details:

index=sem sourcetype="symantec:tap:incidents" time="2017-12-04T17:19:06.606Z"

Event detail:

12/4/17
12:26:52.000 PM
{ [-]
tap_host: 10.140.37.7

tap_incident_id: 104649
deviceUid: [ [+]
]

device_time: 2017-12-04T17:19:06.606Z

domainId: [ [+]
]

event_count: 6

filehash: [ [+]
]

first_event_seen: 2017-12-04T16:30:08.000Z

last_event_seen: 2017-12-04T17:10:37.000Z

log_name: epmp_incident-2017-12-04/incident

priority_level: 1

recommended_action: If this site is not business critical, consider adding it to the Blacklist. Otherwise, consider creating a sinkhole server in your DNS to block the site.

state: 1

summary: Multiple malicious behaviors have been detected from xxxx.sharepoint.com.

time: 2017-12-04T17:19:06.606Z

updated: 2017-12-04T17:19:07.193Z

uuid: 3ba258e0-d917-11e7-e89d-00000000005a

}
Show as raw text
eventtype = nix_errors error host = splunk01.xxxx.com source = symantec_tap sourcetype = symantec:tap:incidents

From the Event Action, I could see that in the event time field "2017-12-04T17:19:06.606Z" and in the _time field as "2017-12-04 12:26:52" for the same event, "_time" is not equal to "time".

_time is being calculated based on when it was indexed instead of when it was an event.

Props.conf details: We have placed this configuration in Heavy forwarder where the data first reaches the splunk then gets ingested into indexer.

[symantec:tap:incidents]
TRUNCATE = 0
TIME_PREFIX=time:\s
TIME_FORMAT=%FT%T.%3N
MAX_TIMESTAMP_LOOKAHEAD=32
TZ=EDT

Question :

How to make the _time field be the same as the time field ?

Kindly guide me on this.

Re: How to fix a timestamp issue for Symantec logs?

somesoni2 — Mon, 04 Dec 2017 20:02:34 GMT

It seems like the timestamp extraction is setup (based on props.conf entry) to take time starting from time:\s, but it may not be working as it appear towards end of a large events. What the number of character into the event your time field appears?

Re: How to fix a timestamp issue for Symantec logs?

MuS — Mon, 04 Dec 2017 20:08:38 GMT

This is what @somesoni2 is referring to MAX_TIMESTAMP_LOOKAHEAD=32 this is set too low and therefore Splunk never reaches the correct time stamp.

Re: How to fix a timestamp issue for Symantec logs?

Hemnaath — Tue, 05 Dec 2017 08:51:56 GMT

Hi somesoni2, thanks for your effort, but could you please guide me how much value should be set in-order to correlate the _time stamp with the event time field.

thanks in advance.

Re: How to fix a timestamp issue for Symantec logs?

niketn — Tue, 29 Sep 2020 17:03:54 GMT

@Hemnaath, Is there a difference between device_time and time?

device_time: 2017-12-04T17:19:06.606Z
time: 2017-12-04T17:19:06.606Z
If not you can just switch to device_time

TIME_PREFIX=device_time:\s
MAX_TIMESTAMP_LOOKAHEAD=128

time: data is pushed towards the end of your event and you seem to have couple of unexpanded JSON Nodes like deviceUid and filehash etc. So it is better you fetch from the first time field which gives the correct time for your event.
For Time Stamp Lookahead you can try some higher number like 128,1024 etc depending on how many characters including whitespaces you might have in your data prefixing timestamp to be extracted.

Re: How to fix a timestamp issue for Symantec logs?

Hemnaath — Tue, 29 Sep 2020 17:08:20 GMT

Hi Niketnilay, thanks for your effort on this, I will update the props.conf with the above mentioned stanza but I am not sure about how/ from where to calculate the value for MAX_TIMESTAMP_LOOKAHEAD=128 from the events. Kindly guide me on this.

[symantec:tap:incidents]
SHOULD_LINEMERGE = false
FIELDALIAS-event_host = tap_host as event_host
KV_MODE = json
TRUNCATE = 0
TIME_PREFIX=device_time:\s
TIME_FORMAT=%FT%T.%3N
MAX_TIMESTAMP_LOOKAHEAD=128
TZ=EDT

Re: How to fix a timestamp issue for Symantec logs?

Hemnaath — Tue, 29 Sep 2020 17:05:16 GMT

Hi Niketnilay, hey I tried the above stanza what you had mentioned in your comment but it did not work. I am getting the same output, so can you please guide me to fix this issue.

index=sem sourcetype="symantec:tap:incidents"

Event details captured after changing the props.conf

12/5/17
11:05:28.000 PM
{ [-]
atp_host: 10.x.x.x

atp_incident_id: 104656
deviceUid: [ [+]
]

device_time: 2017-12-06T04:03:08.713Z

domainId: [ [+]
]

event_count: 1

first_event_seen: 2017-12-05T09:49:58.118Z

last_event_seen: 2017-12-05T09:49:58.118Z

log_name: epmp_incident-2017-12-06/incident

priority_level: 1

recommended_action: Consider blacklisting the site. In addition, you may need to investigate the source of the exposure to see if further action is required.

state: 1

summary: Malicious domain www.xxxx.com detected
time: 2017-12-06T04:03:08.713Z

updated: 2017-12-06T04:03:08.950Z

uuid: 5e89b190-da3a-11e7-d301-000000000061

thanks in advance

Re: How to fix a timestamp issue for Symantec logs?

Hemnaath — Wed, 06 Dec 2017 12:30:56 GMT

Hi All, Can anyone guide me on this, how to make the _time field be the same as the time field in the event.

thanks in advance.

Re: How to fix a timestamp issue for Symantec logs?

Hemnaath — Fri, 08 Dec 2017 14:09:04 GMT

Hi All, still facing the issue, unable to make the _time field be the same as the time field in the event.

Kindly guide me on this.

Re: How to fix a timestamp issue for Symantec logs?

micahkemp — Wed, 03 Jan 2018 17:00:43 GMT

MAX_TIMESTAMP_LOOKAHEAD is relative to the position determined by TIME_PREFIX, so the initial setting of 32 should not have been too low to be valid.

MAX_TIMESTAMP_LOOKAHEAD = <integer>
* Specifies how far (in characters) into an event Splunk should look for a
  timestamp.
* This constraint to timestamp extraction is applied from the point of the
  TIME_PREFIX-set location.
* For example, if TIME_PREFIX positions a location 11 characters into the
  event, and MAX_TIMESTAMP_LOOKAHEAD is set to 10, timestamp extraction will
  be constrained to characters 11 through 20.
* If set to 0, or -1, the length constraint for timestamp recognition is
  effectively disabled.  This can have negative performance implications
  which scale with the length of input lines (or with event size when
  LINE_BREAKER is redefined for event splitting).
* Defaults to 150 (characters).

Re: How to fix a timestamp issue for Symantec logs?

micahkemp — Wed, 03 Jan 2018 17:05:20 GMT

The event as you posted it is in syntax highlighted view, not raw, so the patterns you used in your props.conf are likely incorrect. Try this:

TIME_PREFIX=time":\s*"

The above is based on my assumption that your raw data actually looks more like "time": "<timestamp>"

If that doesn't work please add the raw event, instead of the syntax highlighted representation.