I am trying to extract a timestamp from this type of events. Here, 04 is the day of month and 12 is the month, Dec
on the search head, these events currently appear as 12th April
[04/12/2018 10:16:04] CAUAJMI40245 EVENT: CHANGESTATUS STATUS: SUCCESS JOB: esysprodNOA5minbox
[04/12/2018 10:26:03] CAUAJMI40245 EVENT: CHANGESTATUS STATUS: SUCCESS JOB: esysprod_EX900 MACHINE:
It looks pretty straightforward, but I cannot figure out what I am doing wrong.
The source type for these events is called : "autosyseventsprod"
So, I created a props.conf as below, and located it in the app that gets distributed from my deployment server:
I also verify on the server where the log is created that the props.conf file is updated, and I also restart Splunk on the Universal Forwarder.
[splunk@msplunkutil01 local]$ cat props.conf [autosys_events_prod] TIME_PREFIX = ^[ TIME_FORMAT = %d/%m/%Y %H:%M:%S SHOULD_LINEMERGE = false MAX_TIMESTAMP_LOOKAHEAD = 19
I have tried different time prefix(es) without success.
How do I know if my props.conf is actually being used?
Everything I have tried seems to have no effect so far.
What is the best way to troubleshoot this ?
Thank you for your help in advance.
it is the first time I am trying to extract a timestamp from an event, so I might be doing something wrong.
TIME_PREFIX is a regular expression, but yours is not a valid regex.
^[ starts a character set, but doesn't finish it. Try `^[', which treats the bracket as a literal character.
I have tried your suggestion and it still is showing events for the 12th April, instead of the 4th Dec
I tried those 2:
TIMEPREFIX = '^['
TIMEPREFIX = '['
to confirm my props setttings on the universal forwarder, I found this great command:
[splunk@bautoprod01 local]$ splunk cmd btool --app=autosys props list
MAXTIMESTAMPLOOKAHEAD = 19
SHOULDLINEMERGE = false
TIMEFORMAT = %d/%m/%Y %H:%M:%S
TIME_PREFIX = '['
[splunk@bautoprod01 local]$ pwd
So the above confirms that the settings are applied ("distributed"), but yet it still is not working
Thank you for your help anyway
This should work, give it a try....
[autosys_events_prod] SHOULD_LINEMERGE = false TIME_PREFIX = ^\[ TIME_FORMAT = %m/%d/%Y %H:%M:%S MAX_TIMESTAMP_LOOKAHEAD = 19
Hi Rich and Prakash
I have tried both suggestions and it still is not working
thank you both for your replies, you both suggested to use :
TIME_FORMAT to %m/%d/%Y %H:%M:%S
but my raw events timestamps shows as : [05/12/2018 10:32:03] text text ...
where 05 is the day of the month %d
and 12 is the month %m
so the correct TIME FORMAT should be : %d/%m/%Y %H:%M:%S
please explain why you suggested otherwise, I am getting really confused ...
I am also wondering why all my attempts are failing, is it possible that another definition or config somewhere could take precedence over the app's props.conf ?
Thank you again
@blaise, I originally recommended
%d/%m/%Y %H:%M:%S, but you said it was wrong so I suggested
@blaise: I tested it on my local with your sample data, it's working for me, except you need to make changes to TIME_FORMAT based on your requirements...
##this configs should be on indexers(data parsing happens on indexers) props.conf [autosys_events_prod] SHOULD_LINEMERGE = false TIME_PREFIX = ^\[ TIME_FORMAT = %d/%m/%Y %H:%M:%S MAX_TIMESTAMP_LOOKAHEAD = 19 try running this command to check all the props in that particular app... ./splunk btool props list --debug ./splunk btool props list --debug --app=search
thank you , you are correct and that was my mistake, the props.conf definition needs to be on the indexers.
Once I did that , it started working
Thank you heaps for your help, it is appreciated 🙂
I have finally resolved the issue, the problem was I have a distributed environment ...
so like Prakash suggested, the props.conf needs to be on the indexers, where the timestamp extraction is done.
I have completely removed the props.conf from the universal forwarder server, where I only left the inputs.conf to define the inputs.
Thank you for all your help