Hello, Splunkers!
I'm trying to add a new log file, but I can't extract the correct timestamp.
Help me to write any Timestamp format, which will use date and time from events.
Here in these 3 sample events, timestamp should be 01.09.2015 00:20:05
for the first event,
01.09.2015 00:20:05
for the second event, and so on.
<tr style="height:21px">
<td colspan="3" class="s18-90D19DFDD9934A0F8EEAA283057A16E6">01.09.15</td><td colspan="2" class="s19-90D19DFDD9934A0F8EEAA283057A16E6">00:20:05</td><td class="s20-90D19DFDD9934A0F8EEAA283057A16E6" style="font-size:1px;background-image:none"> </td><td colspan="2" class="s20-90D19DFDD9934A0F8EEAA283057A16E6">0.039</td><td class="s20-90D19DFDD9934A0F8EEAA283057A16E6">Мб.</td><td class="s20-90D19DFDD9934A0F8EEAA283057A16E6">Мобильный интернет</td><td colspan="2" class="s20-90D19DFDD9934A0F8EEAA283057A16E6" style="font-size:1px;background-image:none"> </td><td colspan="5" class="s21-90D19DFDD9934A0F8EEAA283057A16E6">0.00</td>
</tr>
<tr style="height:21px">
<td colspan="3" class="s22-90D19DFDD9934A0F8EEAA283057A16E6">01.09.15</td><td colspan="2" class="s23-90D19DFDD9934A0F8EEAA283057A16E6">00:26:18</td><td class="s24-90D19DFDD9934A0F8EEAA283057A16E6">900</td><td colspan="2" class="s24-90D19DFDD9934A0F8EEAA283057A16E6">1</td><td class="s24-90D19DFDD9934A0F8EEAA283057A16E6">Шт.</td><td class="s24-90D19DFDD9934A0F8EEAA283057A16E6">Входящее SMS</td><td colspan="2" class="s24-90D19DFDD9934A0F8EEAA283057A16E6" style="font-size:1px;background-image:none"> </td><td colspan="5" class="s25-90D19DFDD9934A0F8EEAA283057A16E6">0.00</td>
</tr>
<tr style="height:21px">
<td colspan="3" class="s18-90D19DFDD9934A0F8EEAA283057A16E6">01.09.15</td><td colspan="2" class="s19-90D19DFDD9934A0F8EEAA283057A16E6">00:26:59</td><td class="s20-90D19DFDD9934A0F8EEAA283057A16E6">900</td><td colspan="2" class="s20-90D19DFDD9934A0F8EEAA283057A16E6">1</td><td class="s20-90D19DFDD9934A0F8EEAA283057A16E6">Шт.</td><td class="s20-90D19DFDD9934A0F8EEAA283057A16E6">Входящее SMS</td><td colspan="2" class="s20-90D19DFDD9934A0F8EEAA283057A16E6" style="font-size:1px;background-image:none"> </td><td colspan="5" class="s21-90D19DFDD9934A0F8EEAA283057A16E6">0.00</td>
</tr>
Something to consider is modifying the datetime_config file or, better yet, create a separate datetime_config file for this sourcetype. I should emphasize that this is completely untested.
Copy the existing SPLUNK_HOME/etc/datetime.xml file to SPLUNK_HOME/etc/mydatetime.xml. Add a new define near the bottom of the file.
<define name="mydatetime" extract="month, day, year, hour, minute, second">
<text><![CDATA[\>(?P<month>[012]\d)\.(?P<day>[012]\d|3[01])\.(?P<year>\d{2})\<.*?\>(?P<hour>\d{2}):(?P<minute>\d{2}):(?P<second>\d{2})\<]]></text>
</define>
Then add `to each of the
datePatternsand
timePatterns` stanzas.
In your props.conf file put:
[mysourcetype]
DATETIME_CONFIG = /etc/mydatetime.xml
Yes, sure. I have restarted my splunk server several times. The log file is on splunk servers local disk.
Thanks!
You found my mistake in my props.conf I have wrote DATETIME_CONFIG = /etc/mydatetime.xml instead of DATETIME_CONFIG = /etc/megafon.xml
now name of xml file in etc folder and parameter in DATETIME_CONFIG = are the same
In SPLUNK_HOME/etc/megafon.xml I have specified extract
<define name="megafon" extract="day, month, year, hour, minute, second">
<text><![CDATA[\>(?P<month>[012]\d)\.(?P<day>[012]\d|3[01])\.(?P<year>\d{2})\<.*?\>(?P<hour>\d{2}):(?P<minute>\d{2}):(?P<second>\d{2})\<]]></text>
</define>
<timePatterns>
<use name="megafon"/>
<use name="_time"/>
<use name="_hmtime"/>
<use name="_hmtime"/>
<use name="_dottime"/>
<use name="_combdatetime"/>
<use name="_utcepoch"/>
<use name="_combdatetime2"/>
</timePatterns>
<datePatterns>
<use name="megafon"/>
<use name="_usdate1"/>
<use name="_usdate2"/>
<use name="_isodate"/>
<use name="_eurodate1"/>
<use name="_eurodate2"/>
<use name="_bareurlitdate"/>
<use name="_orddate"/>
<use name="_combdatetime"/>
<use name="_masheddate"/>
<use name="_masheddate2"/>
<use name="_combdatetime2"/>
</datePatterns>
</datetime>
But it is still only date in the timestamp
I am at a loss. Did you restart Splunk after modifying props.conf?
Thank you, man!
I have copied the existing SPLUNK_HOME/etc/datetime.xml file to SPLUNK_HOME/etc/megafon.xml.
I have added your code
<define name="megafon" extract="">
<text><![CDATA[\>(?P<month>[012]\d)\.(?P<day>[012]\d|3[01])\.(?P<year>\d{2})\<.*?\>(?P<hour>\d{2}):(?P<minute>\d{2}):(?P<second>\d{2})\<]]></text>
</define>
<timePatterns>
<use name="megafon"/>
<use name="_time"/>
<use name="_hmtime"/>
<use name="_hmtime"/>
<use name="_dottime"/>
<use name="_combdatetime"/>
<use name="_utcepoch"/>
<use name="_combdatetime2"/>
</timePatterns>
<datePatterns>
<use name="megafon"/>
<use name="_usdate1"/>
<use name="_usdate2"/>
<use name="_isodate"/>
<use name="_eurodate1"/>
<use name="_eurodate2"/>
<use name="_bareurlitdate"/>
<use name="_orddate"/>
<use name="_combdatetime"/>
<use name="_masheddate"/>
<use name="_masheddate2"/>
<use name="_combdatetime2"/>
</datePatterns>
</datetime>
I have modified C:\Program Files\Splunk\etc\apps\search\local\props.conf
[Megafon]
DATETIME_CONFIG = /etc/mydatetime.xml
NO_BINARY_CHECK = true
category = Custom
disabled = false
pulldown_type = true
Source type Megafon was created in Search app context.
And now it is still only date in the timestamp
A modified my answer to include field names in the 'extract' clause.
Double-check the DATETIME_CONFIG setting in your props.conf.