Getting Data In

how to extract time from multi line log

ahmedhassanean
Explorer

Dears,

i have log that repeated every 10 min as below
16-02-08 Name Succ drop

04:26:50 Searches 12 0
04:27:00 Searches 17 0
04:27:10 Searches 12 0

firts line contain Date of the Day and each line contain different Timestamp
i need to know how to extract each line with exact time
i know that i can break events using Break_line option and also break multiple events using multikv
but i couldn't extract Correct time for every event So please advise

0 Karma
1 Solution

vbumgarner
Contributor

That's super ugly. If the event really looks like that, and there's nothing you can do about it, then you could do something like this:

|stats count
| eval _raw="16-02-08 Name Succ drop\n04:26:50 Searches 12 0\n04:27:00 Searches 17 0\n04:27:10 Searches 12 0"
| rex "^(?<date>\d+-\d+-\d+) "
| eval line=split(_raw,"\n")
| mvexpand line
| rex field=line "^(?<time>\d+:\d+:\d+) "
| eval _time=strptime(date + " " + time, "%d-%m-%y %H:%M:%S")

It isn't going to be fast, though. You'd be far better off figuring out how to parse the logs line by line, if at all possible.

View solution in original post

0 Karma

vbumgarner
Contributor

That's super ugly. If the event really looks like that, and there's nothing you can do about it, then you could do something like this:

|stats count
| eval _raw="16-02-08 Name Succ drop\n04:26:50 Searches 12 0\n04:27:00 Searches 17 0\n04:27:10 Searches 12 0"
| rex "^(?<date>\d+-\d+-\d+) "
| eval line=split(_raw,"\n")
| mvexpand line
| rex field=line "^(?<time>\d+:\d+:\d+) "
| eval _time=strptime(date + " " + time, "%d-%m-%y %H:%M:%S")

It isn't going to be fast, though. You'd be far better off figuring out how to parse the logs line by line, if at all possible.

0 Karma

ahmedhassanean
Explorer

i would like to have each line as new event but with correct time and column name as below ( note : date is come in first line only in our case (16-02-08 ) and for each line there is different date and all this table is repeated every 2 min in log with header )

16-02-08 04:26:50 Searches 12 0
16-02-08 04:27:00 Searches 17 0
16-02-08 04:27:10 Searches 12 0

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Not sure I understand. Can you describe your desired outcome?
Do you want these to be one event? Do you want each line to be one event, with all the lines that don't have a date in it using... which date?
Do you have the opportunity to change the application generating these logs?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...