Getting Data In

Why is my props.conf configuration no longer working on my French timestamp and FileZilla server logs?

Builder

Hello guys,

I have a problem with French logs so I tried to create props.conf and deploy it :

[fzs]
TIME_PREFIX = ^\([0-9]*\)\s
TIME_FORMAT = %d/%m/%Y %H:%M:%S

Log example :

(1002561) 01/04/2017 23:59:01 - blablabla

I've understood that the time_prefix will ignore the (number) and space before the french date.

Should it work? My logs from April are not coming however it worked from January to March 2017.

Thanks a lot!

0 Karma
1 Solution

Splunk Employee
Splunk Employee

You need to deploy your props.conf settings where the event parsing happens. There are a few that take effect on the Universal Forwarder, but most of them need to be on your indexing tier (or any intermediary heavy forwarder, if present).
This Wiki page provides a bit more detail around the topic.

View solution in original post

Splunk Employee
Splunk Employee

You need to deploy your props.conf settings where the event parsing happens. There are a few that take effect on the Universal Forwarder, but most of them need to be on your indexing tier (or any intermediary heavy forwarder, if present).
This Wiki page provides a bit more detail around the topic.

View solution in original post

Builder

Ok, I've tried on a test machine and it works finally (I used the add data/upload web interface with the advanced sourcetype settings).

So I've to put my props.conf on both /etc/deployment-apps/_server... and therefore etc/master-apps/_cluster/local?

Thanks a lot!

0 Karma

Splunk Employee
Splunk Employee

Great to hear, you're welcome. Please accept answer when you get a chance. Thanks!

Splunk Employee
Splunk Employee

Where did you deploy your props.conf file?
If your Filezilla logs are being collected with a Universal Forwarder, props.conf needs to be on all the indexers, if you are using a Heavy forwarder somewhere between your Filezilla server and the indexers, it needs to go on the Heavy Forwarder.

In other words: props.conf needs to be on the Splunk role that does the event parsing.

Maybe it helps if you describe your data ingest path a bit more.

0 Karma

Builder

You are right, props.conf is only forwarder side. Should it be deployed on the indexer cluster?

0 Karma

SplunkTrust
SplunkTrust

Props.conf files usually deployed on the indexers, and for the functionality that you want, that is where the props.conf should be, because it is at index time, not at forwarding time that those configs are needed.

SplunkTrust
SplunkTrust

That should be working. Have you checked to make sure that there aren't extra spaces or something else that might have changed slightly in the log since April 1?

0 Karma

Builder

I never used TIME_PREFIX and TIME_FORMAT before, in fact april logs are now indexed as march which is the problem :

03/04/2017 (3rd april 2017 french format) => 4 march 2017

Thanks.

0 Karma

SplunkTrust
SplunkTrust

It looks like it is ignoring your TIME_FORMAT. Are you sure that the stanza is being used for your data? If not, then it would try to do formatting on its own, and that might make it use an American mon/day/year format, which looks like what you are seeing.

0 Karma