Splunk Search

How i set date by log path address ?

nitesh218ss
Communicator

Hi have a log which is inside folder which folder name is date
i give folder name or path is=
C:\Users\T_NiteshS1\Documents\My Received Files\20150511\log2.log

If you see before log2.log you get folder 20150511 This is date
if you expend 20150511 this yyyymmdd

how is set in datetime.xml i don't now xml

Tags (2)
0 Karma

woodcock
Esteemed Legend

Try this for your datetime.xml (hopefully markdown will not mangle the text):

<datetime>
<define name="_dateFromDirectorySegment" extract="year, month, day">
<text><![CDATA[source::.*?(\d{4})(\d{2})(\d{2})]]></text>
</define>
<define name="_timeFromEventData" extract="hour, minute, second, subsecond">
<text><![CDATA[^(\d{2}):(\d{2}:(\d{2}.(\d{3}]]></text>
</define>
<timePatterns>
<use name="_timeFromEventData"/>
</timePatterns>
<datePatterns>
<use name="_dateFromDirectorySegment"/>
</datePatterns>
</datetime>
0 Karma

nitesh218ss
Communicator

in this you not given file path any where

C:\Users\T_NiteshS1\Documents\My Received Files\20150511\log2.log

This is my file path
with out given file path how they take date?

Today i try with file path address that y they not take date
ok thanks i try this in office tomorrow

0 Karma

woodcock
Esteemed Legend

Yes, I skip over the path because you might change the segment-level (add/remove path segments) so I left is as wildcarded. This configuration WILL work if you try it.

0 Karma

nitesh218ss
Communicator

I try your answer but they not work after i try many way but fail
itry

[CDATA[source::.*?\\[My]+\ [Received]+\ [Files]+\\(\d{4})(\d{2})(\d{2})\\[NB92\-Transaction07.log]+]

[CDATA[source::.*?\\(\d{4})(\d{2})(\d{2})\\[NB92\-Transaction07.log]+]

[CDATA[source::.*?\\(\d{4})(\d{2})(\d{2})\\]

[CDATA[source::.*?\\(\d{4})(\d{2})(\d{2})]

[CDATA[source::.*?\My Received Files\(\d{4})(\d{2})(\d{2})]

and so many way i try

logfile not contain date logfile present inside folder that folder have date as name

0 Karma

woodcock
Esteemed Legend

The problem is your entry in props.conf; it uses a relative path and you have given it a full path. Use my original datetime.xml file and use this in props.conf:
DATETIME_CONFIG = /etc/apps/search/datetime.xml

0 Karma

nitesh218ss
Communicator

i used both full address and /etc/apps/search/datetime.xml but they not work

but when i put datetime.xml in /etc/apps/search/local/datetime.xml then at indexing time
splunk not read file give message check sourcetype setting

when i used /etc/apps/search/datetime.xml they read at indexing time normally

0 Karma

woodcock
Esteemed Legend

So you have datetime.xml working generally but we cannot get the exat text string to do it by path. That is a good start. To summarize, your desired path is like this:

C:\Users\T_NiteshS1\Documents\My Received Files\20150511\log2.log

So this line should work (do not worry that it is not fully qualified):

<define name="_dateFromDirectorySegment" extract="year, month, day">
<text><![CDATA[source::.*?(\d{4})(\d{2})(\d{2})]]></text>

Yes, this is the same thing that I wrote way back in the beginning but I stand by my position that it should work (unless you have copied it by hand and typoed something).

Do note that the first line is very important because it specifies the order of the captured strings and how they apply to the variables.

0 Karma

nitesh218ss
Communicator

sir i tired this so many time but they not worked
i thing some thing is wrong in splunk to take date from folder name
this is bug in spluk possible

0 Karma

woodcock
Esteemed Legend

Maybe because you are windows, the problem is the path so instead of this:

DATETIME_CONFIG = /etc/apps/search/datetime.xml

Try this:

DATETIME_CONFIG = \etc\apps\search\datetime.xml
0 Karma

nitesh218ss
Communicator

when i put date in file name like tran20150511.log
and in datetime.xml seeting for date part i useis: [CDATA[source::.*?tran(\d{4})(\d{2})(\d{2})]]
and in props.conf

DATETIME_CONFIG = C:\Program Files\Splunk\etc\apps\search\datetimee.xml
EXTRACT-fields = (?&lt;time&gt;\d{2}:\d{2}:\d{2}\.\d{3})\|(?&lt;audit&gt;[^\|]+)\|(?&lt;serviceType&gt;[^\|]+)\|(?&lt;processId&gt;[^\|]+)\|(?&lt;serviceName&gt;[^\|]+)\|(?&lt;indicator&gt;[^\|]+)\|(?P&lt;message&gt;
LINE_BREAKER = ([\r\n]+)\d{2}:\d{2}:\d{2}\.\d{3}\|[^\|]+\|[^\|]+\|[^\|]+\|[^\|]+\|[^\|]+\|
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
disabled = false
pulldown_type = true

Then they take date correctly but

by folder they not take date

0 Karma

nitesh218ss
Communicator

i use in xml

&lt;define name="_masheddate2" extract="month, day, year"&gt;
    &lt;text&gt;&lt;![CDATA[(?:^|C:\Program Files\Splunk\etc\apps\search::).*?(20\d{2})(\d{2})(\d{2})]]&gt;&lt;/text&gt;
&lt;/define&gt;

i use in props.conf
DATETIME_CONFIG = C:\Program Files\Splunk\etc\apps\search\datetime.xml

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...