Splunk event time and the raw event time difference

[uagraw01]

Hello Splunkers!!

Please help me to fix this time zone issue.

Thanks in advance!!

[PickleRick]

And your current settings are...?

[uagraw01]

@PickleRick 

Props.conf setting 

KV_MODE = xml
NO_BINARY_CHECK = true
CHARSET = UTF-8
LINE_BREAKER = <\/eqtext:EquipmentEvent>()
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 650
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3QZ
TIME_PREFIX = ((?<!ReceiverFmInstanceName>))<eqtext:EventTime>

User time preference setting

[PickleRick]

At first glance it looks relatively ok. Are you using indexed extractions?

[uagraw01]

No, I am not using that attribute in props.conf.

[uagraw01]

@PickleRick 

When I am using this time preference then there is no difference showing. So its good to setup this setting ?

Is there anything else you want me to  suggest for fix ?

[PickleRick]

Wait a second. You're trying to say that regardless of what timezone you set in your preferences the event is still shown at the same time for the same event? (The time on the left, not the time within the event itself obviously since this one is already ingested, indexed and it won't change). That should be impossible.

BTW, what does your ingestion architecture look like for this source? File->UF->indexer? Where do you have your props.conf settings (on which component)?

[uagraw01]

@PickleRick  I am using single standalone machine and the data coming through the nework directory. That network directory produces files and then using inputs.conf I am monitoring into Splunk.

[PickleRick]

Are you sure your lookahead is big enough? I haven't counted exactly but your event seems close to exceeding that 650 characters mark before reaching the timestamp.

Also - have you verified your TIMESTAMP_PREFIX? That capture group looks strange and you have a very strange lookbehind which seems to not do what you think it should do. Verify it on regex101.com

[uagraw01]

@PickleRick Sorry for the delayed response. But my concern I am mentioning below in the screenshot. Please help to fix. Is there any setting I need to add in props.conf.

[PickleRick]

Have you read my previous response? Did you check any of the things I mentioned?

[uagraw01]

@PickleRick I have increased the max_timestamp from 650 to 750 as well as corrected the TIME_PREFIX setting also but still the issue persist.

[uagraw01]

@PickleRick  Actually I got the issue; in my data there is two pattern of events as mentioned below. Therefore, in  props.conf I am using TIME_PREFIX = \<\/ReceiverFmInstanceName\>\<eqtext\:EventTime\> & TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3QZ  and because the TIME_PREFIX setting  as mentioned Splunk is picking only Pattern 1 and skipping Pattern 2 . So please suggest can I remove TIME_PREFIX setting from the props.conf so Splunk will cover or pick both the events (pattern 1 & pattern 2?

================================

Patterrn 1:

In this pattern Time_Prefix is looking different 
</ReceiverFmInstanceName><eqtext:EventTime>2024-08-01T21:23:37.560Z

=================================

Patterrn 2:

In this pattern Time_Prefix is looking different 
</State><eqtext:EventTime>2024-08-01T21:23:37.560Z

[PickleRick · Accepted solution]

No. If you remove it, Splunk will be trying to guess and that's performance-intensive.

I don't know your data and don't know whether you don't have more border cases but you can use a regex with alternative branches to match either form)

TIME_PREFIX = (</ReceiverFmInstanceName><eqtext:EventTime>|</State><eqtext:EventTime>)

(I'm not sure if some of those characters do not require ecaping so take it with a pinch of salt)

[uagraw01]

@PickleRick Thanks for this suggestion. I will incorporate this change into the props settings.