[beginner] Provide simple stat with different string from the field _raw

[chrispappo]

Hi,

if I have some logs like this:
ID DATE _RAW
1 10/06/2015 text .. ERROR text...
2 10/06/2015 text .. ERROR text...
3 10/06/2015 text .. INFO text...
4 10/06/2015 text .. WARN text...

With that, I would like to find the right commande to provide me somethig like that

ID | ERROR| INFO| WARN
1 | 1 | 0 | 0
2 | 1 | 0 | 0
3 | 0 | 1 | 0
4 | 0 | 0 | 1

So that count the number of ERROR, INFO, WARN for each ID. My main problem is that their is any field who detect the type of the event ERROR, INFO, WARN so I have to match it in the _raw field. I guess i have to use some regex but I can't find the good combination.

Is someone can help me ?

Thanks

[stephanefotso · Accepted solution]

Here you go!

 index=...|rex field=_raw "^\S+(?<myfield>\w+)\s+"|stats count as totalcount values(ID) as ID by myfield|table ID myfield totalcount

Is not optimal but it can help
Thanks

[chrispappo]

I think it works i just re work the command like that

index=source | rex field=_raw "\]\s+(?<informations>\S+)\s+(?<ID>[^\-]+)"| chart count over ID by informations

Thanks :). But if one day the words INFO, ERROR or WARN are not followed anymore by the ID, it will not work isn't ?

[stephanefotso]

. My regex just means, after ]extract the informationsfield, and after that field, take all caracters, except the -, and create a ID field with that values.

This means that, if INFO, ERROR or WARN are not followed by ID, the informations field will still be extracted, but the ID field will be populated with wrong values

Thanks

Do not forget to accept the answer, if are satisfy .

[stephanefotso]

please add your comment under the answer.

ok i think i have updated my query. Test it again, and let me know

[chrispappo]

Steel not working. How can you catch the different type of events without use INFO, ERROR, WARN it in the regexe ?

(Because i told you the _raw are like that
text ERROR text
text ERROR text
text INFO text
text WARN text)

thanks

[stephanefotso]

test this and let me know if your values are extracted.

         index=...|rex field=_raw "[^\n]+\s+(?<myfield>\w+)\s+"|table myfield

if not working, let me get a sample event please.

[chrispappo]

it's like that, in bold this is the field ID. The information I want is for each ID how many INFO, ERROR and WARN

2015-06-10 17:20:37,838 [Thread] INFO c.b.w.a.c.AbstractRepository - SE_USE_TTL was not specified. Defaulting to: false

2015-06-10 17:21:37,838 [Thread] ERROR c.b.w.a.c.AbstractRepository - SE_USE_TTL was not specified. Defaulting to: false

2015-06-10 17:22:37,838 [Thread] WARN c.b.w.a.c.AbstractRepository - SE_USE_TTL was not specified. Defaulting to: trye

many thanks

[stephanefotso]

Means, you also need to extrac the ID field. Here you go for the extraction:

...|rex field=_raw "\]\s+(?<informations>\S+)\s+(?<ID>[^\-]+)"|table ID informations