topic Re: Splunk timeformat issue in Getting Data In

Splunk timeformat issue

uagraw01 — Mon, 15 Jul 2024 09:42:59 GMT

Hello Splunkers!!

I have a below event and I want to parse. But the event is not parsing with time format in Splunk. Please help me to get it fix .

TIME_FORMAT :

%dT%H:%M:%S.%3QZ

TIME_PREFIX :

\<eqtext\:EventTime\>

I have used the above setting but nothings works. StillI can see isse with indexed and event time. Please help me to get it fix.

Below are the raw events:

<eqtext:EquipmentEvent xmlns:eqtext="http:///FM/EqtEvent/EqtEventExtTypes/V1/1/5" xmlns:sbt="http://FM/Common/Services/ServicesBaseTypes/V1/8/4" xmlns:eqtexo="http://FM/EqtEvent/EqtEventExtOut/V1/1/5"><eqtext:ID><eqtext:Location><eqtext:PhysicalLocation><AreaID>7053</AreaID><ZoneID>33</ZoneID><EquipmentID>25</EquipmentID><ElementID>0</ElementID></eqtext:PhysicalLocation></eqtext:Location><eqtext:Description> Welder cold</eqtext:Description><eqtext:MIS_Address>6.2</eqtext:MIS_Address></eqtext:ID><eqtext:Detail><State>CAME_IN</State><eqtext:EventTime>2024-07-13T16:21:31.287Z</eqtext:EventTime><eqtext:MsgNr>7751154552301783480</eqtext:MsgNr><Severity>INFO</Severity><eqtext:OperatorID>WALVAU-SCADA-1</eqtext:OperatorID><ErrorType>TECHNICAL</ErrorType></eqtext:Detail></eqtext:EquipmentEvent></eqtexo:EquipmentEventReport>

Re: Splunk timeformat issue

yuanliu — Mon, 15 Jul 2024 04:20:20 GMT

Should be "%FT%H:%M:%S.%3Q%Z". You can always test your time format with an emulation, like

| makeresults format=csv data="eqtext:EventTime 2024-07-13T16:21:31.287Z" | eval _time = strptime('eqtext:EventTime', "%FT%H:%M:%S.%3Q%Z")

Re: Splunk timeformat issue

inventsekar — Mon, 15 Jul 2024 04:32:46 GMT

Hi @uagraw01

... pls check "%Y-%m-%dT%H:%M:%S.%Q"

Re: Splunk timeformat issue

uagraw01 — Mon, 15 Jul 2024 04:36:39 GMT

@yuanliu @inventsekar When I ingested to Splunk both the times varies ( index and event time). Please see below screenshot.

Re: Splunk timeformat issue

inventsekar — Mon, 15 Jul 2024 04:48:49 GMT

Hi @uagraw01 pls show us your props.conf

(if you dont know where to find the props....

are you using HF or not... if u use HF, then props.conf should be there

if you dont use HF, then, you should have props.conf for this should be in indexer)

Re: Splunk timeformat issue

uagraw01 — Mon, 15 Jul 2024 04:51:49 GMT

@inventsekar

[Scada_walmart_alarm]
DATETIME_CONFIG =
KV_MODE = xml
NO_BINARY_CHECK = true
CHARSET = UTF-8
LINE_BREAKER = <\/eqtext\:EquipmentEvent
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_FORMAT = %FT%H:%M:%S.%3Q%Z
#TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N%Z
TIME_PREFIX = \<eqtext\:EventTime\>
SEDCMD-first = s/^.*<eqtext:EquipmentEvent/<eqtext:EquipmentEvent/g

Re: Splunk timeformat issue

inventsekar — Mon, 15 Jul 2024 04:57:34 GMT

Hi @uagraw01 .. for the timeformat.. did you apply both mine and @yuanliu 's timeformat's?..

(after updating the props.conf, you must restart the splunk services.. then only the changes will be inserted to Splunk)

Re: Splunk timeformat issue

uagraw01 — Mon, 15 Jul 2024 04:59:56 GMT

@inventsekar I have updated this "TIME_FORMAT = %FT%H:%M:%S.%3Q%Z" other is in #

I have already restarted the Splunkd services.

Re: Splunk timeformat issue

inventsekar — Mon, 15 Jul 2024 05:20:07 GMT

1) Pls give us the search query you are using..

what you see on the results is your splunk user profile's timezone setting.

2) on your Splunk user profile, pls make sure you have the right timezone settings (click on your username---- > Account settings---- > Time Zone)

Re: Splunk timeformat issue

uagraw01 — Mon, 15 Jul 2024 05:25:56 GMT

@inventsekar
If you see the below attached screenshot. The first three events is matching but the last one event is always creating an issue.

FYI : I am using default timezone setting.

Re: Splunk timeformat issue

inventsekar — Mon, 15 Jul 2024 05:33:44 GMT

if 3 results got good timestamp meaning, props.conf is working fine.

lets troubleshoot the 4th one..

pls copy paste your search query.. (remove the hostnames, confidential info etc.. )

Re: Splunk timeformat issue

PickleRick — Mon, 15 Jul 2024 06:09:01 GMT

1. For "ASAP" you pay your friendly consultant or PS. This is a community-driven forum - people help others in their own spare time. Saying "help me ASAP" can be perceived as rude.

2. How do you ingest your data? UF->indexer? HF->indexer? UF->HF->indexer? What input do the events come in by. Where do you have the props.conf for the sourcetype?

3. You have the timestamp relatively late in the event and - as you've shown - your MAX_TIMESTAMP_LOOKAHEAD is set to only 24.

4. When posting config excerpts or data samples please use code block or preformatted style. It greatly helps readability.

Re: Splunk timeformat issue

uagraw01 — Mon, 15 Jul 2024 06:16:47 GMT

@inventsekar I have added these three corrected settings in props.conf. I am waiting for the real event to come in, if this works then the job will be done.

LINE_BREAKER = <\/eqtext:EquipmentEvent>()
TIME_PREFIX = ((?<!ReceiverFmInstanceName>))<eqtext:EventTime>
TZ = America/Glace_Bay

Re: Splunk timeformat issue

uagraw01 — Mon, 15 Jul 2024 06:20:05 GMT

@PickleRick Please don't take my words otherwise. I didn't mean to say that. Btw way thanks for correcting me. I will take care with my words from the next time.

Re: Splunk timeformat issue

PickleRick — Mon, 15 Jul 2024 06:23:16 GMT

Thought as much, it's just worth noting that things can be perceived differently than what we wanted to say 😉

Now, check the technical part of my response 🙂

Most probably, you need to increase the lookahead because you have no timestamp in first 24 chars of your event. The architectural issue might also mean that when you fix that you'll be doing the right thing but in wrong place.

Re: Splunk timeformat issue

uagraw01 — Mon, 15 Jul 2024 06:38:38 GMT

@PickleRick According to your suggestion my settings will be as below

MAX_TIMESTAMP_LOOKAHEAD = 520
( timestamps comes after 520 character of events)

Re: Splunk timeformat issue

PickleRick — Mon, 15 Jul 2024 06:43:26 GMT

You might want to set it to a bit higher value. The timestamp is relatively late in the event and the part before the timestamp contains dynamic data which can be of varying length so you have to account for that.

Bonus question - you're not using INDEXED_EXTRACTIONS, are you?

Re: Splunk timeformat issue

uagraw01 — Mon, 15 Jul 2024 07:26:44 GMT

@PickleRick No, I am not using INDEXED_EXTRACTIONS.

I am using KV_MODE=xml in my setting ( props). Is there any other significance of INDEXED_EXTRACTIONS ?

Re: Splunk timeformat issue

ITWhisperer — Mon, 15 Jul 2024 08:51:16 GMT

The first three may be working because Splunk might not be finding the timestamp you are searching for within 520 characters, so it is finding the sbt:MessageTimeStamp, which happens to be the same as the EventTime in these events.

sbt:MessageTimeStamp does not exist in the failing event so Splunk is using the ingest time in the fourth event.

The fourth event is a different format to the the other three events "eqtext:EquipmentEvent" instead of "eqtexo:EquipmentEventReport" so should ideally be in a different sourcetype (at least the source file names are different so it should be relatively easy to split them off).

The timestamp in the fourth event is at least around 627 characters in so your lookahead should at least cover that (and as @PickleRick said, it looks like you are dealing with variable length data, so 627 may not be enough).

Re: Splunk timeformat issue

PickleRick — Mon, 15 Jul 2024 10:35:06 GMT

Yes, INDEXED_EXTRACTIONS can alter the procesing path of your event. Without it the event is parsed on the first "heavy" component the event goes through - typically either the intermediate HF or the destination indexer. When you enable indexed extractions on a UF, the data is parsed directly on the originating UF and is not touched after that (apart from possible ingest actions).