Getting Data In

Filter out unnecessary data before sending the data to indexer

sahabhi606
Path Finder

Dear Splunkers, I need your help in filtering out the data which I am recieving before storing it into the indexer. Below is a sample data that I am recieving, and here I am intrested to keep data in below tags and discard others.

Intrested data:

<name>MACHINE_HOSTNAME</name> and <ip_address>11.22.33.44</ip_address>


Sample Data:

<computer><general><id>1234</id><name>MACHINE_HOSTNAME</name><network_adapter_type>XXXXXX</network_adapter_type><mac_address>XX:XX:XX:XX:XX:XX</mac_address><alt_network_adapter_type>Ethernet</alt_network_adapter_type><alt_mac_address>XX:XX:XX:XX:XX:XX</alt_mac_address><ip_address>11.22.33.44</ip_address><last_reported_ip>12.34.56.78</last_reported_ip><serial_number>XXXXXXXXXX</serial_number><udid>XXXX-XXXX-XX-XX</udid><jamf_version>10.X.0-tXXXXXX</jamf_version><platform>Mac</platform><barcode_1 /><barcode_2 /><asset_tag /><remote_management><managed>true</managed>


Regards,
Abhi

Labels (2)
0 Karma

thambisetty
SplunkTrust
SplunkTrust

add below to props.conf on HF or Indexer, don't forget to replace yoursourcetype with the one for which you want to apply below extraction. It doesn't have timestamp after removing everything and keeping only name and ip_address, you should manage that.

[yoursourcetype]
SEDCMD-remove=s/.*(\<name\>.*\<\/name\>).*(\<ip_address\>.*\<\/ip_address\>).*/\1,\2/g

 

————————————
If this helps, give a like below.

sahabhi606
Path Finder

Hi @thambisetty , below configuration works only for the events which has these fields. 

 

[yoursourcetype]
SEDCMD-remove=s/.*(\<name\>.*\<\/name\>).*(\<ip_address\>.*\<\/ip_address\>).*/\1,\2/g

How can I discard rest events which do not have above values? or is there a way, where I get to keep only a few set of field values from the data, and discard others?

 

Thanks,

Abhishek

0 Karma

thambisetty
SplunkTrust
SplunkTrust

is the above sample data  single event ?

————————————
If this helps, give a like below.
0 Karma

sahabhi606
Path Finder

Hi @thambisetty ,

 

Thanks for your post. Yes it is a single event. This is just a small snippet of a very long event

 

Regards,

Abhishek

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sahabhi606,

you have to use the SEDCMD command in props.conf.

If the sample you shared is only one event, you have to use something like this:

SEDCMD-removeunwanted1 = s/^\<computer\>\<general\>\<id\>\d+<\/id\>//
SEDCMD-removeunwanted2 = s/\<network_adapter_type\>.*\<\/alt_mac_address\>//
SEDCMD-removeunwanted3 = s/\<last_reported_ip\>.*//

 Ciao.

Giuseppe

0 Karma

sahabhi606
Path Finder

Hello @gcusello ,

 

 

Thanks for your post. yes it is a single evenyt, but this is just a snapshot of a very long event, and it won't be feasible to use SEDCMD to remove most of the tags. 

I would actually prefer to filter out all data except the relevant ones. I tried  it using below config in transforms.conf, but no luck

 

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \<name\>.*\<\/name\>
DEST_KEY = queue
FORMAT = indexQueue

 

Regards,

Abhishek

 

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sahabhi606,

if it's acceptable for you to delete all the events it's easier for you!

Ciao and good splunking.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...