topic Splunk - Log Ingestion Modification in Getting Data In

Splunk - Log Ingestion Modification

fmandelli — Fri, 11 Sep 2020 14:47:38 GMT

Hi everyone.

I am still learning Splunk so that I will need your assistance on this, please.

I am currently working on a PoC where our firewalls are sending traffic logs to Splunk.

In order to shorten the size of data being ingested into Splunk I would like to know if there is an option to remove “variable” names from traffic being transmitted.

In summary, this is a typical traffic log that we receive from our firewalls in order to ingest it into Splunk:

Instead, we would like to ingest only "values" from each “key/value” pair.

In this case, instead of time=17:28:50 devname=”my3700d”, only 17:28:50 “my3700d” would be ingested.

Could you guys please assist me on this issue?

Thank you very much.

Fmandelli

Re: Splunk - Log Ingestion Modification

richgalloway — Fri, 11 Sep 2020 15:26:46 GMT

Understand that by removing the key you make it harder for Splunk to parse the events. The current format is one Splunk handles natively.

If you really want to remove the keys then you may be able to do so using SEDCMD such as

SEDCMD-nokey = s/\s\w+=/\s/g

By doing so, however, you'll have to assign the values to fields somehow.

Re: Splunk - Log Ingestion Modification

isoutamo — Fri, 11 Sep 2020 15:38:44 GMT

As @richgalloway said you can do it, but to utilize those fields in queries you basically need to add those back on search time. If you want to save license then it’s probably better to remove some unneeded fields with values?

r. Ismo