topic Help with splunk search for unix timestamp? in Splunk Search

Help with splunk search for unix timestamp?

vinothkumark — Wed, 12 Jul 2023 21:01:19 GMT

I want to create an alert for which I am writing a search query but I am unable to filter using the time range picker. since the events contains unix timestamp, I tried to convert but it fails during time range picker.
can you help me what is wrong here?

Query:

index=isilon sourcetype="emc:isilon:rest" "memory threshold" | eval "Start Time" = strftime('events.start', "%d/%m/%Y %I:%M:%S %p") | table "Start Time" events.message

Ideally when I run this query with time range picker on June 12th then there should be NO results, but the results contains June8th events(attachment provided)

Sample event:

{"events": {"devid": 8, "event": 400020001, "id": "8.794044", "lnn": 8, "message": "The SMB server (LWIO) is throttling due to current memory threshold settings. Current memory usage is 90% (23556 MB) and the memory threshold is set to 90%.", "resolve_time": 1686266238, "severity": "critical", "specifier": {"PercentMemoryUsed": 90, "PercentThreshold": 90, "ProcessMemConsumedInMB": 23556, "antime": 1686266290.600042, "devid": 8, "extime": 1686266290.490373, "kmtime": 1686266238.984405, "lnn": 8, "val": 90.0}, "time": 1686266238, "value": 90.0}, "timestamp": "2023-06-12 23:46:57", "node": "0.0.0.0", "namespace": "event"}

{"events": {"devid": 8, "event": 400020001, "id": "8.793138", "lnn": 8, "message": "The SMB server (LWIO) is throttling due to current memory threshold settings. Current memory usage is 90% (23556 MB) and the memory threshold is set to 90%.", "resolve_time": 1686248504, "severity": "critical", "specifier": {"PercentMemoryUsed": 90, "PercentThreshold": 90, "ProcessMemConsumedInMB": 23556, "antime": 1686248570.519368, "devid": 8, "extime": 1686248570.447457, "kmtime": 1686248504.901769, "lnn": 8, "val": 90.0}, "time": 1686248504, "value": 90.0}, "timestamp": "2023-06-12 23:46:57", "node": "0.0.0.0", "namespace": "event"}

Re: splunk search query for unix timestamp

burwell — Tue, 11 Jul 2023 21:48:43 GMT

Hello. We can't see what time Splunk thinks those events arrived in.

Do you have a props.conf for that sourcetype/source to tell the Splunk indexer which field you are using as the actual event time?

https://docs.splunk.com/Documentation/Splunk/latest/Admin/propsconf

Something like

TIME_FORMAT=%s (for unixtime format)
TIME_PREFIX = time\":\s*\"

Re: splunk search query for unix timestamp

vinothkumark — Wed, 12 Jul 2023 05:21:34 GMT

Not on the Indexer, but I can see the props on the Heavy forwarder.

[PureStorage_REST]
INDEXED_EXTRACTIONS = JSON
TIMESTAMP_FIELDS = time,opened
TIME_FORMAT = %Y-%m-%dT%H:%M:%SZ
TZ = UTC
detect_trailing_nulls = auto
SHOULD_LINEMERGE = false
KV_MODE = none
AUTO_KV_JSON = false

Re: splunk search query for unix timestamp

burwell — Wed, 12 Jul 2023 20:45:49 GMT

So in your query where you show the events, can you display the _time as well?

Re: splunk search query for unix timestamp

inventsekar — Thu, 13 Jul 2023 00:28:03 GMT

index=isilon sourcetype="emc:isilon:rest" "memory threshold" | table _time _indextime "Start Time" events.message

Can you pls run this and update us the results..

do you know if the UF, heavy forwarder and indexer are having same clock times(are they using NTP for time sync?)

Re: splunk search query for unix timestamp

vinothkumark — Thu, 13 Jul 2023 04:46:08 GMT

_time always picks the time range. for example if I set during June 8th, the results look like:

If I set the time range as JUne12th, the results look like,