topic Re: Timestamp extraction config (props) is not working in Getting Data In

Timestamp extraction config (props) is not working

dongwonn — Tue, 26 Mar 2024 00:32:13 GMT

Hello,

Log :

Mar 22 10:50:51 x.x.x.21 Mar 22 11:55:00 Device version -: [2024-03-22 11:54:12] Event : , IP : , MAC : , Desc :

Props :

[host::x.x.x.21]

CHARSET = utf8
TIME_PREFIX = \-:\s\[
TIME_FORMAT = %Y-%m-%d %H:%M:%S

When I check _time field, value is still 2021-03-22 10:50:51.

I think Device's IP is x.x.x.21.

So it seems that 21 is recognized as the year and I config props.

But props is not working...

Help me Thank you.

Re: Timestamp extraction config (props) is not working

KendallW — Tue, 26 Mar 2024 02:39:49 GMT

Hi @dongwonn a few things to check
-check the host field in Splunk matches the host:: stanza in your props.conf
-Since you are not explicitly specifying a lot of configs, they may be taking default values from other places. Use btool to check the full props settings being applied to this host:
$SPLUNK_HOME/bin/splunk cmd btool props list host::x.x.x.21
-Update your TIME_PREFIX to capture the full string before the timestamp beginning at the start of the event, so that Splunk will definitely exclude the preceding timestamps. Example:
TIME_PREFIX=^\w{3}\s\d\d\s(\d{2}\:?){3}\s(\d{0,3}\.?){4}\s\w{3}\s\d\d\s(\d{2}\:?){3}\s[\w\s]+\-:\s\[

Re: Timestamp extraction config (props) is not working

gcusello — Tue, 26 Mar 2024 06:42:26 GMT

Hi @KendallW ,

check if the issue is related to the header or to thwe regex: use a sourcetype instead of host in the stanza header.

Sometimes I found an issue using host or source instead sourcetype.

Ciao.

Giuseppe

Re: Timestamp extraction config (props) is not working

dongwonn — Thu, 28 Mar 2024 01:09:23 GMT

Hello, Thank you for your answer. I already tried it but it doesn't work. I'll try it one more time!

Re: Timestamp extraction config (props) is not working

dongwonn — Thu, 28 Mar 2024 01:43:23 GMT

Thank you for answer! I tried specifying and applying all the regular expressions as you answered, but it doesn't work. It's difficult...

Re: Timestamp extraction config (props) is not working

dongwonn — Thu, 28 Mar 2024 04:10:27 GMT

@gcusello @KendallW

I receive the log via UDP from the heavy forwarder connected to the indexer. After setting the sourcetype to temp in the heavy forwarder (inputs), the sourcetype is set to override according to the host and regular expression. Is it correct to extract timestamps in the heavy forwarder props? No matter how many times I apply the settings you mentioned, it doesn't work.

Re: Timestamp extraction config (props) is not working

dongwonn — Thu, 28 Mar 2024 07:01:37 GMT

I don't know why, but after applying the settings and restarting, the year value was set normally.

[host::x.x.x.21]
TIME_PREFIX = ....

TIME_FORMAT = ....

So far, I have reloaded the settings with /debug/refresh, but this time I tried reloading the settings by restarting Splunk. Although the current operating environment is difficult to operate with just one server, is it possible that there may be cases where new settings are not reloaded?

Re: Timestamp extraction config (props) is not working

gcusello — Thu, 28 Mar 2024 06:27:00 GMT

Hi @dongwonn ,

not all the configuration are reloaded with /debug/refresh.

For this reason it's always better to restart Splunk.

Ciao.

Giuseppe

Re: Timestamp extraction config (props) is not working

dongwonn — Thu, 28 Mar 2024 07:09:33 GMT

Oh, in the future, timestamp issues will have to be resolved by restarting the instance. thank you @gcusello!