topic Re: How to extract values depending on field, props.conf, transforms.conf, and regex? in Getting Data In

How to extract values depending on field, props.conf, transforms.conf, and regex?

Atchyuth_P — Tue, 23 May 2023 04:37:01 GMT

Hi Team,

Kindly check with below logs

[19-May-2023 06:15:55.341][INFO] abc@abc.com@ABC-CB-NOC, 1.1.1.1:61, create, user:Test, appliances: ALR-prod1 , changeset:devices { device{ALR-prod1} { config { orgs { org-services{ATCHYUTH-NOC} { user-identification { local-database { users { + user{Test} { + email-address test@abc.com + passwd + status CREATED} } } } } } } } }

[28-Jun-2022 08:35:48.010][INFO] abc@abc.com@ABC-CB-NOC, 1.1.1.1:61, create, authentication-method:Dummy-Auth, template:Access-Template , changeset:devices { template{Access-Template} { config { orgs { org-services{Atchyuth-NOC} { user-identification { authentication-methods { + authentication-method{Dummy-Auth} { + method { + local }} } } } } } } }

[28-Sep-2020 12:13:07.137][INFO] abc@abc.com@ABC-CB-NOC, CLI 'set devices template ABC config orgs org-services ATCHYUTH-NOC security captive-portal url abc.net'

[28-Jun-2022 08:35:48.010][INFO] abc@abc.com@ABC-CB-NOC, 1.1.1.1:53, create, authentication-method:Dummy-Auth, template:Access-Template, changeset:devices { template{Access-Template} { config { orgs { org-services{Atchyuth} { user-identification { authentication-methods { + authentication-method{Dummy-Auth} { + method { + local }} } } } } } } }

[28-Sep-2020 12:13:07.137][INFO] abc@abc.com@ABC-CB-NOC, CLI 'set devices template ABC config orgs org-services ATCHYUTH security portal url abc.net'

Based on above logs we have one common field org-services. Depending on org-services need to create new field and extract the values to the field. Kindly help me in props.conf and transforms.conf and also let me know the regex pattern that will create a new field based on org-services.

Re: How to extract values depending on field, props.conf, transforms.conf, and regex?

isoutamo — Sat, 27 May 2023 13:52:29 GMT

you could try this

props.conf

[<your sourcetype here>] REPORT-org_services = org_services

and transforms.conf

[org_services] REGEX = org-services[\s\{]*(?<org_services>[^\}\s]+)

Another way is use only props.conf with EXTRACT instead of REPORT. Especially if you use this only in one sourcetype then maybe EXTRACT is better option.

r. Ismo

Re: How to extract values depending on field, props.conf, transforms.conf, and regex?

PickleRick — Sun, 28 May 2023 10:05:34 GMT

One possible solution is of course the one shown by @isoutamo but there is more to it than meets the eye. There are two different kinds of events here so it would probably be best to define two separate definitions for field extractions. Or even two separate sourcetypes.

As a side note - that application log seems to be overly complicated - I'd try to work with the app team to make them log in a less nested and easier parseable format.