topic Timestamp extraction from event data ?? in Getting Data In

Timestamp extraction from event data ??

rakesh_498115 — Tue, 28 Jan 2014 12:12:29 GMT

Hi,

I have eventdata as follows ,

05NOV13 XYZ1 21:40:21 GMI User JESSD11 GMI sessn 1 timed-out token 2872827 revoked
26JAN14 ABC1 21:36:50 GMI User JESSE05 Token #0442422 removed from CMW SOT( 139)
26JAN14 ABC1 21:36:50 GMI User JESSE05 GMI sessn 1 logged off token 0442422 deleted
26JAN14 XYZ1 21:40:21 TOW USER JESSD91 TERM #2872827 WAS FORCED OFF BY TIMEOUT
26JAN14 XYZ1 21:40:21 GMI User JESSD91 GMI sessn 1 timed-out token 2872827 revoked
26JAN14 ABC1 21:40:50 GMI User JESSD91 Token #2872827 removed from CMW SOT( 140)
26JAN14 ABC1 21:40:50 GMI User JESSD91 GMI sessn 1 logged off token 2872827 deleted

in which i have date mentioned in one place and timestamp mentioned in another place ,

How can i use my TIME_FORMAT attribute for this case ? to pick the ryt time ???

i.e 26JAN14 ABC1 21:40:50 as 26JAN14 21:40:50

tried the following in props.conf , but didnt work any other solution for this ??

TIME_FORMAT = %d%b%y\s[A-Z0-9]{4}\s%H:%M:%S

Re: Timestamp extraction from event data ??

kristian_kolb — Tue, 28 Jan 2014 13:35:30 GMT

What type of information is represented by the XYZ1 and ABC1? Time Zone? Or something else?

I don't think TIME_FORMAT can handle the regex pattern in a nice way - in fact I believe that it tries to match the literal string, square brackets and all.

Re: Timestamp extraction from event data ??

richgalloway — Tue, 28 Jan 2014 13:55:04 GMT

I agree that TIME_FORMAT cannot accept a regex string. Here's something I thought of, but haven't yet tried. Use a transform to parse the event data then an eval to build a datetime string.

props.conf:

[test] REPORT-logdata = parseLogData EVAL-datetime = date." ".time ...

transforms.conf:

[parseLogData] REGEX = "(?<date>.*?)\s(?<foo>.*?)\s(?<time>.*?)\s"

Re: Timestamp extraction from event data ??

rakesh_498115 — Tue, 28 Jan 2014 14:54:07 GMT

Hi Kristian..XYZ1 and ABC1 represents my system codes...

Re: Timestamp extraction from event data ??

kristian_kolb — Wed, 29 Jan 2014 02:25:10 GMT

Sorry, that won't work. The tranformations take place after timestamp extractions.

Re: Timestamp extraction from event data ??

adityapavan18 — Thu, 30 Jan 2014 10:53:36 GMT

If the timestamp is not recognized, you need to make changes to datetime.xml

you can use following in datetime.xml

<define name="_cssdatetime" extract="day, litmonth, year, ignored_sep3, hour, minute, second">
        <text><![CDATA[([012]\d|3[01])(?i)(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)(?:20)?([901]\d)(?!\d| {2,})\s+(\w+)\s+([012]?\d):([0-6]?\d):([0-6]?\d)]]></text>
</define>

And point your sourcetype to this new datetime xml using DATETIME_CONFIG in props.conf

Re: Timestamp extraction from event data ??

rakesh_498115 — Fri, 31 Jan 2014 09:38:07 GMT

Excellent Pavan .. 🙂