topic How do I index one _time field from separated different time data? in Getting Data In

How do I index one _time field from separated different time data?

goji — Sat, 04 Aug 2018 07:17:54 GMT

This is my data from with html tags. This is just single line.

03/<tr class="mtx" style="text-align:right;"><td style="white-space:nowrap"><div class="a_print"><a href="hourly_s1.php?prec_no=44&block_no=47662&year=1980&month=03&day=1&view=p1">1</a></div></td><td class="data_0_0">997.3</td><td class="data_0_0">1001.6</td><td class="data_0_0">13.5</td><td class="data_0_0">5.5</td><td class="data_0_0">1.5</td><td class="data_0_0">9.1</td><td class="data_0_0">14.2</td><td class="data_0_0">4.5</td><td class="data_0_0"> 59</td><td class="data_0_0"> 40</td><td class="data_0_0">6.5</td><td class="data_0_0">13.6</td><td class="data_0_0" style="text-align:center">–k–k¼</td><td class="data_0_0">24.6</td><td class="data_0_0" style="text-align:center">–k</td><td class="data_0_0">--</td><td class="data_0_0">--</td><td class="data_0_0">--</td><td class="data_0_0" style="text-align:left">‰J</td><td class="data_0_0" style="text-align:left">~
‚È‚µ</td></tr>

The time of this data consists of three pieces of information: year=, month=, and date=.
So, I just use custom DATETIME_CONFIG to ingest the data but it doesn't work. This is my created .xml file.

<datetime>
  <define name="test" extract="year, month, day">
    <text><![CDATA[(?year\=(\d{4})&month\=(\d{2})&day\=(\d{1,2})]]></text>
  </define>

  <timePatterns>
    <use name="test"/>
  </timePatterns>
  <datePatterns>
    <use name="test"/>
  </datePatterns>
</datetime>

I think there is no grammatical error because there was no error at splunk restarted.
How do I index one _time field from three separated time data at index time?

Thank you,

Re: How do I index one _time field from separated different time data?

dcharboneau_spl — Tue, 07 Aug 2018 20:05:58 GMT

Looks like your regex may have an issue.

Try
&year=(\d{4})&month=(\d{2})&day=(\d{1,2})

Re: How do I index one _time field from separated different time data?

goji — Fri, 10 Aug 2018 09:38:01 GMT

I use "&" instead of "?" in front of year, but it doesn't work.

Re: How do I index one _time field from separated different time data?

sudosplunk — Fri, 10 Aug 2018 16:24:23 GMT

Your datetime.xml looks good to me. Please make sure to put these configurations on both source(forwarders) and destination (indexers).

If nothing works, give this a try instead of datetime.xml

Append below to your props.conf

TIME_PREFIX = year\=
TIME_FORMAT = %y&month=%d&day=%d

Re: How do I index one _time field from separated different time data?

goji — Tue, 29 Sep 2020 20:53:53 GMT

nittala_surya,

This data was a bit old, so I could read it by increasing the value of MAX_DAYS_AGO.

Thank you!!!