Filter out unnecessary data before sending the data to indexer

[sahabhi606]

Dear Splunkers, I need your help in filtering out the data which I am recieving before storing it into the indexer. Below is a sample data that I am recieving, and here I am intrested to keep data in below tags and discard others.

Intrested data:

<name>MACHINE_HOSTNAME</name> and <ip_address>11.22.33.44</ip_address>

Sample Data:

<computer><general><id>1234</id><name>MACHINE_HOSTNAME</name><network_adapter_type>XXXXXX</network_adapter_type><mac_address>XX:XX:XX:XX:XX:XX</mac_address><alt_network_adapter_type>Ethernet</alt_network_adapter_type><alt_mac_address>XX:XX:XX:XX:XX:XX</alt_mac_address><ip_address>11.22.33.44</ip_address><last_reported_ip>12.34.56.78</last_reported_ip><serial_number>XXXXXXXXXX</serial_number><udid>XXXX-XXXX-XX-XX</udid><jamf_version>10.X.0-tXXXXXX</jamf_version><platform>Mac</platform><barcode_1 /><barcode_2 /><asset_tag /><remote_management><managed>true</managed>

Regards,
Abhi

[thambisetty]

add below to props.conf on HF or Indexer, don't forget to replace yoursourcetype with the one for which you want to apply below extraction. It doesn't have timestamp after removing everything and keeping only name and ip_address, you should manage that.

[yoursourcetype]
SEDCMD-remove=s/.*(\<name\>.*\<\/name\>).*(\<ip_address\>.*\<\/ip_address\>).*/\1,\2/g

[sahabhi606]

Hi @thambisetty , below configuration works only for the events which has these fields. 

[yoursourcetype]
SEDCMD-remove=s/.*(\<name\>.*\<\/name\>).*(\<ip_address\>.*\<\/ip_address\>).*/\1,\2/g

How can I discard rest events which do not have above values? or is there a way, where I get to keep only a few set of field values from the data, and discard others?

Thanks,

Abhishek

[thambisetty]

is the above sample data  single event ?

[sahabhi606]

Hi @thambisetty ,

Thanks for your post. Yes it is a single event. This is just a small snippet of a very long event

Regards,

Abhishek

[gcusello]

Hi @sahabhi606,

you have to use the SEDCMD command in props.conf.

If the sample you shared is only one event, you have to use something like this:

SEDCMD-removeunwanted1 = s/^\<computer\>\<general\>\<id\>\d+<\/id\>//
SEDCMD-removeunwanted2 = s/\<network_adapter_type\>.*\<\/alt_mac_address\>//
SEDCMD-removeunwanted3 = s/\<last_reported_ip\>.*//

 Ciao.

Giuseppe

[sahabhi606]

Hello @gcusello ,

Thanks for your post. yes it is a single evenyt, but this is just a snapshot of a very long event, and it won't be feasible to use SEDCMD to remove most of the tags. 

I would actually prefer to filter out all data except the relevant ones. I tried  it using below config in transforms.conf, but no luck

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \<name\>.*\<\/name\>
DEST_KEY = queue
FORMAT = indexQueue

Regards,

Abhishek

[gcusello]

Hi @sahabhi606,

if it's acceptable for you to delete all the events it's easier for you!

Ciao and good splunking.

Giuseppe