topic Re: simple timestamp extraction from log file in Getting Data In

simple timestamp extraction from log file

a_splunk_user — Mon, 28 Sep 2020 12:08:03 GMT

All of my data from an snmp log file has timestamps which are the modified date of the log file:
7/5/12 2:50:50.000 PM

However, I need the associated timestamp for every event within that log file.
2012-07-23 16:18:32 abc.xyz.net [UDP: [111.222.333.444]:26263->[0.0.0.0]:0]:

This timestamp format seems to be fairly common, so I don't believe I will need to modify the $SPLUNK_HOME/etc/datetime.xml file.

I must be missing something obvious, but I'm a bit confused as to where else to look. I believe I have read all the docs and most of the questions out there regarding similar issues.

Here is my props.conf:
[source::C:\usr\log\snmptrapd.log] TIME_FORMAT = %Y-%m-%d %H:%M:%S MAX_TIMESTAMP_LOOKAHEAD = 25 SHOULD_LINEMERGE = true

Thanks in advance for any advice!

Re: simple timestamp extraction from log file

lguinn2 — Mon, 23 Jul 2012 20:52:12 GMT

Where is your props.conf - on the indexer(s) or on the forwarder? What kind of forwarder?

Timestamp extraction is part of the data parsing phase. It cannot be done on a Universal Forwarder. So, your props.conf needs to go wherever the parsing occurs - on the indexer(s). Or, if you are using a heavy forwarder, on the heavy forwarder.

Re: simple timestamp extraction from log file

a_splunk_user — Tue, 24 Jul 2012 13:44:22 GMT

Hi,

Sorry for not being clear on that. The props.conf is on the indexer, reading snmp data from a local log file.

Thx

Re: simple timestamp extraction from log file

a_splunk_user — Thu, 26 Jul 2012 14:00:35 GMT

This is still an issue - any help is appreciated please.

Re: simple timestamp extraction from log file

lguinn2 — Thu, 26 Jul 2012 20:02:45 GMT

Maybe this is the problem. I found this in the documentation for props.conf:

**Considerations for Windows file paths:**

When you specify Windows-based file paths as part of a [source::<source>] stanza, you must
escape any backslashes contained within the specified file path.

Example: [source::c:\\path_to\\file.txt]

So try this instead:

[source::C:\\usr\\log\\snmptrapd.log]
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = true

Also, just in case this isn't the problem - what sourcetype is assigned to this data?

Re: simple timestamp extraction from log file

a_splunk_user — Thu, 26 Jul 2012 21:53:48 GMT

Hi lguinn,

Thanks for getting back to me. Oddly the escaped paths were in fact there, not sure why the unescaped ones were posted - my bad.

The sourcetype is set to 'snmp', and has a number of hosts writing to it - all data in the 'snmp' sourcetype has this issue.

Thanks!

Re: simple timestamp extraction from log file

lguinn2 — Fri, 27 Jul 2012 08:08:59 GMT

I checked to see if "snmp" is a built-in sourcetype, and it is not. So I would love to see the props.conf that references the snmp sourcetype, and any transforms.conf stanzas as well.

Otherwise, I am out of ideas.... 😞

Re: simple timestamp extraction from log file

a_splunk_user — Thu, 09 Aug 2012 15:08:13 GMT

The solution ended up being a modified props.conf file like so:

[snmp]
NO_BINARY_CHECK = 1
pulldown_type = 1
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = TRUE
TIME_PREFIX = ^

Once that is done I can search on sourcetype=snmp, and the timestamps are correctly registered with Splunk.

Hope this helps others!