Getting Data In

simple timestamp extraction from log file

a_splunk_user
Path Finder

All of my data from an snmp log file has timestamps which are the modified date of the log file:

7/5/12
2:50:50.000 PM

However, I need the associated timestamp for every event within that log file.
2012-07-23 16:18:32 abc.xyz.net [UDP: [111.222.333.444]:26263->[0.0.0.0]:0]:

This timestamp format seems to be fairly common, so I don't believe I will need to modify the $SPLUNK_HOME/etc/datetime.xml file.

I must be missing something obvious, but I'm a bit confused as to where else to look. I believe I have read all the docs and most of the questions out there regarding similar issues.

Here is my props.conf:

[source::C:\usr\log\snmptrapd.log]
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = true

Thanks in advance for any advice!

Tags (1)
0 Karma

a_splunk_user
Path Finder

The solution ended up being a modified props.conf file like so:

[snmp]
NO_BINARY_CHECK = 1
pulldown_type = 1
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = TRUE
TIME_PREFIX = ^

Once that is done I can search on sourcetype=snmp, and the timestamps are correctly registered with Splunk.

Hope this helps others!

0 Karma

lguinn2
Legend

Maybe this is the problem. I found this in the documentation for props.conf:

**Considerations for Windows file paths:**

When you specify Windows-based file paths as part of a [source::<source>] stanza, you must
escape any backslashes contained within the specified file path.

Example: [source::c:\\path_to\\file.txt]

So try this instead:

[source::C:\\usr\\log\\snmptrapd.log]
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = true

Also, just in case this isn't the problem - what sourcetype is assigned to this data?

0 Karma

lguinn2
Legend

I checked to see if "snmp" is a built-in sourcetype, and it is not. So I would love to see the props.conf that references the snmp sourcetype, and any transforms.conf stanzas as well.

Otherwise, I am out of ideas.... 😞

0 Karma

a_splunk_user
Path Finder

Hi lguinn,

Thanks for getting back to me. Oddly the escaped paths were in fact there, not sure why the unescaped ones were posted - my bad.

The sourcetype is set to 'snmp', and has a number of hosts writing to it - all data in the 'snmp' sourcetype has this issue.

Thanks!

0 Karma

lguinn2
Legend

Where is your props.conf - on the indexer(s) or on the forwarder? What kind of forwarder?

Timestamp extraction is part of the data parsing phase. It cannot be done on a Universal Forwarder. So, your props.conf needs to go wherever the parsing occurs - on the indexer(s). Or, if you are using a heavy forwarder, on the heavy forwarder.

0 Karma

a_splunk_user
Path Finder

This is still an issue - any help is appreciated please.

0 Karma

a_splunk_user
Path Finder

Hi,

Sorry for not being clear on that. The props.conf is on the indexer, reading snmp data from a local log file.

Thx

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...