We have numerous log lines that are in a format similar to the following:-
2019-04-09 13:00:03 DEBUG DynamicName1 1000 (1.00) ; DynamicName2 2000 (2.00) ;
2019-04-09 13:00:02 DEBUG DynamicName2 500 (0.50) ; DynamicName4 3100 (3.10) ; DynamicName5 12000 (12.00) ;
2019-04-09 13:00:00 DEBUG DynamicName1 600 (0.60) ; DynamicName5 2100 (2.10) ;
The DynamicName# is a dynamic string that can have multiple values per line (but never the same value per line), the numbers after it represent a timing in milliseconds and then seconds.
What I want to get is a table of all the unique DynamicName(s), their average execution times and counts
However, I can't quite get the extraction correct. When I use a rex, for example
rex field=_raw "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)" | table name time
However this creates a table of multiple values per row and then I can't use other commands on it correctly. For example:-
rex field=_raw "(?<name>\w+) (?<time>\d+) \(\d+.\d+\) ; " | table name time | sort -time
Does not result in the correct result I am expecting.
Is there a way I can correctly extract the data to get true dynamic multiple values that I can then table with 1 DynamicName per table row
Try splitting it up into a mv field after stripping out the first characters that aren't needed:
| eval foo=replace(_raw, "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} DEBUG", "")
| makemv delim=";" foo
| mvexpand foo
| rex field=foo "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"
| table name time | sort -time
Then you can use mvexpand to split it up into multiple events and your regex can work on that.
Try splitting it up into a mv field after stripping out the first characters that aren't needed:
| eval foo=replace(_raw, "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} DEBUG", "")
| makemv delim=";" foo
| mvexpand foo
| rex field=foo "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"
| table name time | sort -time
Then you can use mvexpand to split it up into multiple events and your regex can work on that.
Awesome that worked. I had played with the mv functions before but couldn't get it to work. Much appreciated