Getting Data In

Filter the data of a logfile at Universal Forwarder?

sarvesh_11
Communicator

Hi Splunker,

I am using splunkforwarder 6.5 on windows 2k8 servers. I am monitoring a log file, from splunk.
I have modified inputs.conf at Universal Forwarder.
The size of the file that i am monitoring is 130mb, out of which my useful data is of somewhere around 20Mb.
Can i restrict the unwanted data?
I have list of keywords, for which log event is required and to be indexed.

Is it possible to do the same at Universal Fowarder level?

TIA

0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi òsarvesh_11,
you can filter your logs on Indexers or (if present) on Heavy Forwarders.
The instructions to filter events are at https://docs.splunk.com/Documentation/Splunk/8.0.3/Forwarding/Routeandfilterdatad#Filter_event_data_...

As you can read, you can:

  • discard all events and take some of them,
  • take all events and discard some of them.

The job to do is to find one or more regex to filter your data.

Only one hint: put attention to your architecture, because the props.conf and transforms.conf files to filter events must be located on:

  • Indexers (if you haven't Heavy Forwarders),
  • on Heavy Forwarders (if present and oll the traffic pass through them),
  • both on Indexers and Heavy Forwarders, if you have a mixed situation (some Universal Forwarders directly send data to Indexers and sone others through HFs).

Ciao.
Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi òsarvesh_11,
you can filter your logs on Indexers or (if present) on Heavy Forwarders.
The instructions to filter events are at https://docs.splunk.com/Documentation/Splunk/8.0.3/Forwarding/Routeandfilterdatad#Filter_event_data_...

As you can read, you can:

  • discard all events and take some of them,
  • take all events and discard some of them.

The job to do is to find one or more regex to filter your data.

Only one hint: put attention to your architecture, because the props.conf and transforms.conf files to filter events must be located on:

  • Indexers (if you haven't Heavy Forwarders),
  • on Heavy Forwarders (if present and oll the traffic pass through them),
  • both on Indexers and Heavy Forwarders, if you have a mixed situation (some Universal Forwarders directly send data to Indexers and sone others through HFs).

Ciao.
Giuseppe

0 Karma

sarvesh_11
Communicator

Hi @gcusello ,
Yeah i am clear about indexer and/or Heavy forwarder, i was checking if we can place props.conf and transforms.conf at Universal Forwarder.
To filter out the logs at forwarder level only.

Because i ready mixed views for this.

Thanks,

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sarvesh_11,
at Universal Forwarders level, you can filter events only from Windows eventlogs, as you can see at https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#WINDOWS_INPUTS:

Ciao.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...