topic Re: How to remove the field from raw data in Dashboards & Visualizations

How to remove the field from raw data

aditsss — Sat, 13 Feb 2021 18:32:54 GMT

Hi Team,

I have one requirement.

I have raw logs as shown below:

2021-02-12 09:22:32,936 INFO [ Web -4092] AuthenticationFilter Attempting request for (<asriva22><lgposputb500910.ghp.bcp.com><CN=lgposputb50010.ghp.aexp.com, OU=Middleware Utilities, O=ABC Company, L=Phoenix, ST=Arizona

2021-02-12 09:22:38,689 INFO [ Web -4099] o.a.n.w.s.AuthenticationFilter Authentication success for smennen

2021-02-12 08:45:05,277 INFO [Web -3253] o.a.n.w.s.AuthenticationFilter Attempting request for (<JWT token>) GET https://ebac/api/flow/controller/bulletins

I want to remove highlighted time from the logs.

How can I do that.

Thanks in Advance

Re: How to remove the field from raw data

ITWhisperer — Sat, 13 Feb 2021 18:35:55 GMT

| rex mode=sed "s/^.*INFO/INFO/g"

Re: How to remove the field from raw data

aditsss — Sat, 13 Feb 2021 18:40:23 GMT

@ITWhisperer

It will not only be INFO it could be ERROR as well in raw logs

Can you help me with the regex.

Thanks in advance.

Re: How to remove the field from raw data

jotne — Mon, 15 Feb 2021 07:09:19 GMT

| rex mode=sed "s/.*?,[0-9]+ //"

Try som like this (non greedy to stop after first , with number behind)

Re: How to remove the field from raw data

bowesmana — Sun, 14 Feb 2021 23:36:47 GMT

@aditsss

Either of these are valid

| rex mode=sed "s/^[^ ]* [^ ]* //" | rex mode=sed "s/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d+ //"

be careful using .* as it will be greedy, so if you have a matching pattern at the end of the data, you will lose the line up to that point.

First is just delineating by the date/time which have trailing spaces and the second is stricter in matching the date/time format as shown.