Hi Experts,
I don't have a time stamp field in any of my events. As of now, the default system time is added as _time.
File comes by hourly with the file name (filename_201503121000.csv). [filename_YYYYMMDDHHSS.csv ]. Now I am trying to extract the _time from the source file name. I created a transformation and then calculated _time based on source.
It seems like _time has changed,
Earlier: search duration is <2015-Mar-12 00:00 to 2015-Mar-12 12:00>
Filename_201503121000.csv [ 12-Mar-2015 10:00] comes at 11.10 so all the events are looking like this,
03/12/2015 11.10 <event 1>
03/12/2015 11.10 <event 2>
03/12/2015 11.10 <event 3>
...
03/12/2015 11.10 <event n>
After the change, the _time : search duration is
<2015-Mar-12 00:00 to 2015-Mar-12 12:00>
03/12/2015 10.00 <event 1>
03/12/2015 10.00 <event 2>
03/12/2015 10.00 <event 3>
...
03/12/2015 10.00 <event n>
When I select the duration <2015-Mar-12 10:00 to 2015-Mar-12 10:59>
the records are not available. Seems like _time has been changed and it's not possible to search using that updated _time.
Some times there would be some delay in third party system which posts the file so it's required to change the time. Help me to change _time based on the file name(source).
Thanks in advance.
Cheers!
This can be done with datetime.xml
. This should work for you:
<datetime>
<define name="datefromfile" extract="year, month, day">
<text><![CDATA[source::.*?_(\d{4})(\d{2})(\d{2}]]></text>
</define>
<define name="timefromfile" extract="hour, second">
<text><![CDATA[source::.*?_\d{8}(\d{2})(\d{2}]]></text>
</define>
<timePatterns>
<use name="timefromfile"/>
</timePatterns>
<datePatterns>
<use name="datefromfile"/>
</datePatterns>
</datetime>
This can be done with datetime.xml
. This should work for you:
<datetime>
<define name="datefromfile" extract="year, month, day">
<text><![CDATA[source::.*?_(\d{4})(\d{2})(\d{2}]]></text>
</define>
<define name="timefromfile" extract="hour, second">
<text><![CDATA[source::.*?_\d{8}(\d{2})(\d{2}]]></text>
</define>
<timePatterns>
<use name="timefromfile"/>
</timePatterns>
<datePatterns>
<use name="datefromfile"/>
</datePatterns>
</datetime>