topic Re: Need to parse Bluecoat VPM-XML file. this is not a XML format in Dashboards & Visualizations

Need to parse Bluecoat VPM-XML file. this is not a XML format

danje57 — Wed, 30 Sep 2020 04:51:43 GMT

Dear all,

I need your help as I need to parse a file generated by bluecoat, wich contain data relative to our web proxy policy.

The format is like this:

;; CPL generated by Visual Policy Manager: [Thu Mar 26 14:00:04 CET 2020]
;*************************************************************
; WARNING:
;     THIS FILE IS AUTOMATICALLY GENERATED - DO NOT EDIT!
;     ANY MANUAL CHANGES TO THIS FILE WILL BE LOST WHEN VPM
;     POLICY IS REINSTALLED.
;*************************************************************


define category "Blacklisted"
    isdsdsd.com
    *sdsds.com
end category "Blacklisted"

define condition __GROUP5
    realm=admin group="admonui"
end condition __GROUP5

define condition __GROUP7
    realm=admin group="user1"
end condition __GROUP7

define condition __GROUP25
    realm=blablablabla"
end condition __GROUP25

define condition __GROUP28
    realm=bliblibli
end condition __GROUP28

;; Description:
define condition __CondList1
    url.domain="*ecurity.com"
    url.domain="sdsds*ecurity.com"
end condition __CondList1

It seams that value are between words:
define XXXXX and end XXXXX

We cannot predict the XXXX

However XXXXX are the same to start with define and end for example
define MY_OWN_Policy
value1="dsdsds"
value2="fdfdfdfd"
end MY_OWN_Policy

In addition, comments are allowed using ;; before the define statement.

Do you have idea on how to parse such format?

Regards

Re: Need to parse Bluecoat VPM-XML file. this is not a XML format

to4kawa — Thu, 02 Apr 2020 10:02:20 GMT

|makeresults
| eval _raw=" ;; CPL generated by Visual Policy Manager: [Thu Mar 26 14:00:04 CET 2020]
 ;*************************************************************
 ; WARNING:
 ;     THIS FILE IS AUTOMATICALLY GENERATED - DO NOT EDIT!
 ;     ANY MANUAL CHANGES TO THIS FILE WILL BE LOST WHEN VPM
 ;     POLICY IS REINSTALLED.
 ;*************************************************************


 define category \"Blacklisted\"
     isdsdsd.com
     *sdsds.com
 end category \"Blacklisted\"

 define condition __GROUP5
     realm=admin group=\"admonui\"
 end condition __GROUP5

 define condition __GROUP7
     realm=admin group=\"user1\"
 end condition __GROUP7

 define condition __GROUP25
     realm=blablablabla\"
 end condition __GROUP25

 define condition __GROUP28
     realm=bliblibli
 end condition __GROUP28

 ;; Description:
 define condition __CondList1
     url.domain=\"*ecurity.com\"
     url.domain=\"sdsds*ecurity.com\"
 end condition __CondList1"
 | rex max_match=0 "(?ms)define (category|condition) (\"|__)(?<fieldname>\w+)[\"\s]+(?<fieldvalue>.*?)end"
 | rex field=fieldvalue mode=sed "s/\s+/ /g"
 | eval counter=mvrange(0,mvcount(fieldname))
 | stats list(field*) as field* by counter
 | foreach field* [ eval <<FIELD>> = mvindex('<<FIELD>>', counter) ]
 | eval {fieldname} = fieldvalue
 | fields - counter field*
 | stats values(*) as *

Re: Need to parse Bluecoat VPM-XML file. this is not a XML format

danje57 — Thu, 02 Apr 2020 11:57:56 GMT

Thanks to4kawa,

There is no way to create a source type instead to parse inline the file?

Indeed the file has 300line like this.

Re: Need to parse Bluecoat VPM-XML file. this is not a XML format

to4kawa — Thu, 02 Apr 2020 21:22:57 GMT

make transforms.conf with REGEX and FORMAT
good luck