Splunk - Log Ingestion Modification

[fmandelli]

Hi everyone.

I am still learning Splunk so that I will need your assistance on this, please.

I am currently working on a PoC where our firewalls are sending traffic logs to Splunk.

In order to shorten the size of data being ingested into Splunk I would like to know if there is an option to remove “variable” names from traffic being transmitted.

In summary, this is a typical traffic log that we receive from our firewalls in order to ingest it into Splunk:fw_log.jpg

Instead, we would like to ingest only "values" from each “key/value” pair.

In this case, instead of time=17:28:50 devname=”my3700d”, only 17:28:50 “my3700d” would be ingested.

Could you guys please assist me on this issue?

Thank you very much.

Fmandelli

[richgalloway]

Understand that by removing the key you make it harder for Splunk to parse the events.  The current format is one Splunk handles natively.

If you really want to remove the keys then you may be able to do so using SEDCMD such as

SEDCMD-nokey = s/\s\w+=/\s/g

By doing so, however, you'll have to assign the values to fields somehow.

[isoutamo]

As @richgalloway said you can do it, but to utilize those fields in queries you basically need to add those back on search time. If you want to save license then it’s probably better to remove some unneeded fields with values?

r. Ismo