Getting Data In

Filter out unnecessary data before sending the data to indexer

sahabhi606
Path Finder

Dear Splunkers, I need your help in filtering out the data which I am recieving before storing it into the indexer. Below is a sample data that I am recieving, and here I am intrested to keep data in below tags and discard others.

Intrested data:

<name>MACHINE_HOSTNAME</name> and <ip_address>11.22.33.44</ip_address>


Sample Data:

<computer><general><id>1234</id><name>MACHINE_HOSTNAME</name><network_adapter_type>XXXXXX</network_adapter_type><mac_address>XX:XX:XX:XX:XX:XX</mac_address><alt_network_adapter_type>Ethernet</alt_network_adapter_type><alt_mac_address>XX:XX:XX:XX:XX:XX</alt_mac_address><ip_address>11.22.33.44</ip_address><last_reported_ip>12.34.56.78</last_reported_ip><serial_number>XXXXXXXXXX</serial_number><udid>XXXX-XXXX-XX-XX</udid><jamf_version>10.X.0-tXXXXXX</jamf_version><platform>Mac</platform><barcode_1 /><barcode_2 /><asset_tag /><remote_management><managed>true</managed>


Regards,
Abhi

Labels (2)
0 Karma

thambisetty
SplunkTrust
SplunkTrust

add below to props.conf on HF or Indexer, don't forget to replace yoursourcetype with the one for which you want to apply below extraction. It doesn't have timestamp after removing everything and keeping only name and ip_address, you should manage that.

[yoursourcetype]
SEDCMD-remove=s/.*(\<name\>.*\<\/name\>).*(\<ip_address\>.*\<\/ip_address\>).*/\1,\2/g

 

————————————
If this helps, give a like below.

sahabhi606
Path Finder

Hi @thambisetty , below configuration works only for the events which has these fields. 

 

[yoursourcetype]
SEDCMD-remove=s/.*(\<name\>.*\<\/name\>).*(\<ip_address\>.*\<\/ip_address\>).*/\1,\2/g

How can I discard rest events which do not have above values? or is there a way, where I get to keep only a few set of field values from the data, and discard others?

 

Thanks,

Abhishek

0 Karma

thambisetty
SplunkTrust
SplunkTrust

is the above sample data  single event ?

————————————
If this helps, give a like below.
0 Karma

sahabhi606
Path Finder

Hi @thambisetty ,

 

Thanks for your post. Yes it is a single event. This is just a small snippet of a very long event

 

Regards,

Abhishek

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sahabhi606,

you have to use the SEDCMD command in props.conf.

If the sample you shared is only one event, you have to use something like this:

SEDCMD-removeunwanted1 = s/^\<computer\>\<general\>\<id\>\d+<\/id\>//
SEDCMD-removeunwanted2 = s/\<network_adapter_type\>.*\<\/alt_mac_address\>//
SEDCMD-removeunwanted3 = s/\<last_reported_ip\>.*//

 Ciao.

Giuseppe

0 Karma

sahabhi606
Path Finder

Hello @gcusello ,

 

 

Thanks for your post. yes it is a single evenyt, but this is just a snapshot of a very long event, and it won't be feasible to use SEDCMD to remove most of the tags. 

I would actually prefer to filter out all data except the relevant ones. I tried  it using below config in transforms.conf, but no luck

 

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \<name\>.*\<\/name\>
DEST_KEY = queue
FORMAT = indexQueue

 

Regards,

Abhishek

 

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sahabhi606,

if it's acceptable for you to delete all the events it's easier for you!

Ciao and good splunking.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...