Getting Data In

How to configure props.conf to use the date string at the beginning of my sample event as the event _time?

nicocin
Path Finder

I have an event with multiple date strings, it looks like this:

2016-06-01 15:31:31 INFO  - Transfer[sourceName=xxx,sourceFile=xxx,sourceSize=xxx,sourceCheckSum=xxx,targetName=xxx,targetFile=xxx,targetSize=xxx,targetCheckSum=xxx,status=xxx,errorText=xxx,startTime=Wed Jun 01 15:29:26 CEST 2016,endTime=Wed Jun 01 15:29:27 CEST 2016,checkSumMethod=xxx,originalEntryDate=xxx]

Splunk uses the date string in startTime for the _time field. I want to use the date string found in the beginning of the event.

In the props.conf, I've added TIME_FORMAT to the stanza, but nothing changes..

TIME_FORMAT = %y-%m-%d %H:%M:%S

Any hints?

0 Karma
1 Solution

woodcock
Esteemed Legend

The y needs to be upper-case. Try this:

[yoursourcetype]
TIME_FORMAT = %Y-%m-%d %H:%M:%S
TIME_PREFIX=^

Then restart all the splunk processes on your Indexers and check that ONLY events that arrive after the restart are correct (the older ones will stay wrong).

View solution in original post

woodcock
Esteemed Legend

The y needs to be upper-case. Try this:

[yoursourcetype]
TIME_FORMAT = %Y-%m-%d %H:%M:%S
TIME_PREFIX=^

Then restart all the splunk processes on your Indexers and check that ONLY events that arrive after the restart are correct (the older ones will stay wrong).

nicocin
Path Finder

Its still not working. I've deployed an app to my Heavy Forwarder.

The app contains the following in the folder "default":

input.conf

[monitor://C:\log\mylog1.log]
sourcetype=mysourcetype
host=server1

[monitor://C:\log\mylog2.log]
sourcetype=mysourcetype
host=server2

props.conf

[mysourcetype]
 TIME_FORMAT = %Y-%m-%d %H:%M:%S
 TIME_PREFIX=^

I don't see my fault..

Edit: Problem solved! I've searched for "mysourcetype" in all deployed apps, and I've found another props.conf (containing my sourcetype) deployed to all forwarders.. The guy who has implemented this initially hasn't done a proper configuration.

Thank you for the help!

0 Karma

somesoni2
Revered Legend

YOu would need the attribute TIME_PREFIX to set this

[yoursourcetype]
TIME_FORMAT = %y-%m-%d %H:%M:%S
TIME_PREFIX=^
....
...
0 Karma

nicocin
Path Finder

Thank you, but unfortunately it is still not working (still using startTime).

0 Karma

somesoni2
Revered Legend

Where (Universal forwarder/Heavy Forwarder/Indexer) you kept your props.conf? It should be available in the first Full instance of SPlunk (Heavy forwarder OR Indexer whichever comes first in the data flow).

0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

Observability Newsletter Highlights | March 2023

 March 2023 | Check out the latest and greatestSplunk APM's New Tag Filter ExperienceSplunk APM has updated ...

Security Newsletter Updates | March 2023

 March 2023 | Check out the latest and greatestUnify Your Security Operations with Splunk Mission Control The ...