Timestamp extraction config (props) is not working

[dongwonn]

Hello, 

Log  :

Mar 22 10:50:51 x.x.x.21 Mar 22 11:55:00 Device version -: [2024-03-22 11:54:12] Event : , IP : , MAC : , Desc :

Props :

[host::x.x.x.21]

CHARSET = utf8
TIME_PREFIX = \-:\s\[
TIME_FORMAT = %Y-%m-%d %H:%M:%S

When I check _time field, value is still 2021-03-22 10:50:51.

I think Device's IP is x.x.x.21.

So it seems that 21 is recognized as the year and I config props.

But props is not working...

Help me Thank you.

[dongwonn]

@gcusello @KendallW 

I receive the log via UDP from the heavy forwarder connected to the indexer. After setting the sourcetype to temp in the heavy forwarder (inputs), the sourcetype is set to override according to the host and regular expression. Is it correct to extract timestamps in the heavy forwarder props? No matter how many times I apply the settings you mentioned, it doesn't work. 

[gcusello]

Hi @KendallW ,

check if the issue is related to the header or to thwe regex: use a sourcetype instead of host in the stanza header.

Sometimes I found an issue using host or source instead sourcetype.

Ciao.

Giuseppe

[dongwonn]

Hello, Thank you for your answer. I already tried it but it doesn't work. I'll try it one more time!

[KendallW · Accepted solution]

Hi @dongwonn a few things to check
-check the host field in Splunk matches the host:: stanza in your props.conf
-Since you are not explicitly specifying a lot of configs, they may be taking default values from other places. Use btool to check the full props settings being applied to this host:
$SPLUNK_HOME/bin/splunk cmd btool props list host::x.x.x.21
-Update your TIME_PREFIX to capture the full string before the timestamp beginning at the start of the event, so that Splunk will definitely exclude the preceding timestamps. Example:
TIME_PREFIX=^\w{3}\s\d\d\s(\d{2}\:?){3}\s(\d{0,3}\.?){4}\s\w{3}\s\d\d\s(\d{2}\:?){3}\s[\w\s]+\-:\s\[
 

[dongwonn]

Thank you for answer! I tried specifying and applying all the regular expressions as you answered, but it doesn't work. It's difficult...

[dongwonn]

I don't know why, but after applying the settings and restarting, the year value was set normally.

[host::x.x.x.21]
TIME_PREFIX = ....

TIME_FORMAT = ....

So far, I have reloaded the settings with /debug/refresh, but this time I tried reloading the settings by restarting Splunk. Although the current operating environment is difficult to operate with just one server, is it possible that there may be cases where new settings are not reloaded?

[gcusello]

Hi @dongwonn ,

not all the configuration are reloaded with /debug/refresh.

For this reason it's always better to restart Splunk.

Ciao.

Giuseppe

[dongwonn]

Oh, in the future, timestamp issues will have to be resolved by restarting the instance. thank you @gcusello!