topic how to extract time from multi line log in Getting Data In

how to extract time from multi line log

ahmedhassanean — Sun, 21 Feb 2016 14:18:01 GMT

Dears,

i have log that repeated every 10 min as below
16-02-08 Name Succ drop

04:26:50 Searches 12 0
04:27:00 Searches 17 0
04:27:10 Searches 12 0

firts line contain Date of the Day and each line contain different Timestamp
i need to know how to extract each line with exact time
i know that i can break events using Break_line option and also break multiple events using multikv
but i couldn't extract Correct time for every event So please advise

Re: how to extract time from multi line log

s2_splunk — Sun, 21 Feb 2016 20:56:04 GMT

Not sure I understand. Can you describe your desired outcome?
Do you want these to be one event? Do you want each line to be one event, with all the lines that don't have a date in it using... which date?
Do you have the opportunity to change the application generating these logs?

Re: how to extract time from multi line log

ahmedhassanean — Sun, 21 Feb 2016 21:18:21 GMT

i would like to have each line as new event but with correct time and column name as below ( note : date is come in first line only in our case (16-02-08 ) and for each line there is different date and all this table is repeated every 2 min in log with header )

16-02-08 04:26:50 Searches 12 0
16-02-08 04:27:00 Searches 17 0
16-02-08 04:27:10 Searches 12 0

Re: how to extract time from multi line log

vbumgarner — Sun, 13 Mar 2016 07:45:55 GMT

That's super ugly. If the event really looks like that, and there's nothing you can do about it, then you could do something like this:

|stats count
| eval _raw="16-02-08 Name Succ drop\n04:26:50 Searches 12 0\n04:27:00 Searches 17 0\n04:27:10 Searches 12 0"
| rex "^(?<date>\d+-\d+-\d+) "
| eval line=split(_raw,"\n")
| mvexpand line
| rex field=line "^(?<time>\d+:\d+:\d+) "
| eval _time=strptime(date + " " + time, "%d-%m-%y %H:%M:%S")

It isn't going to be fast, though. You'd be far better off figuring out how to parse the logs line by line, if at all possible.