creating a _masheddate3 in datetime.xml

[hiddenkirby]

This is a question stemmed from http://answers.splunk.com/questions/2996/strptime-format-for-yyyymmddhhmmss and http://answers.splunk.com/questions/2831/index-on-regex-field-from-source

my event source looks like this ".../scripts/201005271243/data/file.txt".

End goal is to parse the date from the source.

I have copied the datetime.xml file to myapp/default dir.

i have modified props.conf to

[my_sourcetype]
DATETIME_CONFIG = /opt/splunk/etc/apps/myapp/default/datetime.xml

i have modified datetime.xml to

<define name="_masheddate3" extract="year, month, day, hour, minute">
    <text><![CDATA[(?:^|source::).*\/scripts\/(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})\/data.*]]></text>
</define>

as well as the

<datePatterns><use name="_masheddate3"/>

...to no avail. Index doesn't seem to populate at all.

[Lowell · Accepted solution]

Well, your regex does seem to work, however, you may want to tweak it to (1) only match source:: patterns, although you have enough other path-like matching it's unlikely to match else where, and (2) remove the unnecessary escaping for /, but I'm not sure that would cause you a problem.

I'd suggest, something like this:

<text><![CDATA[source::.*?/scripts/(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})/data/]]></text>

I also dropped off the .* from the end since the other rules don't have any such matching at the end. I also replaced your .* at the front with the non-greedy .*? which, should help performance.

Hmmm, I think this could be your problem.... Your _masheddate3 is contains not only a date but also a time. It looks like _combdatetime for example, is setup in both the timePatterns listing as well as the datePatterns. So if you don't have your entry in both, then I'd give that a try.

[nitesh218ss]

Hi have a log which is inside folder which folder name is date
i give folder name or path is=
C:\Users\T_NiteshS1\Documents\My Received Files\20150511\log2.log

If you see before log2.log you get folder 20150511 This is date
if you expend 20150511 this yyyymmdd

how is set in datetime.xml i don't now xml

I try many way but fail
my try
[CDATA[source::.*?\[My]+\ [Received]+\ [Files]+\(\d{4})(\d{2})(\d{2})\[NB92-Transaction07.log]+]

[CDATA[source::.*?\(\d{4})(\d{2})(\d{2})\[NB92-Transaction07.log]+]

[CDATA[source::.*?\(\d{4})(\d{2})(\d{2})\]

[CDATA[source::.*?\(\d{4})(\d{2})(\d{2})]

[CDATA[source::.*?\My Received Files(\d{4})(\d{2})(\d{2})]

and so many way i try

[hiddenkirby]

I added it to both time and date .. still no go. Seems to still run off of the last modified date. New question... http://answers.splunk.com/questions/3102/datetime-xml-change-doesnt-seem-to-be-working

[gkanapathy]

You do need to list the name of your rule in both the datePatterns and timePatterns part of the datetime.xml if you want it to get both date and time. It's okay to list the same rule in both places.

[Lowell]

lol. Sometimes it's the simple things that take the most time to find. (I've been there many many times). Good catch. Hey, let me know about the whole time/date Patterns thing, does it make a difference?

[hiddenkirby]

bad path to datetime.xml

[hiddenkirby]

Ah hah! "Can't open DatePaser XML configuration file ...datetime.xml" No such file or directory.

[hiddenkirby]

how do i know if it failed.. other than finding the index did not populate? is an error in _internal ?

[hiddenkirby]

i suspect the regex to be incorrect.