Getting Data In

Filter the data of a logfile at Universal Forwarder?

sarvesh_11
Communicator

Hi Splunker,

I am using splunkforwarder 6.5 on windows 2k8 servers. I am monitoring a log file, from splunk.
I have modified inputs.conf at Universal Forwarder.
The size of the file that i am monitoring is 130mb, out of which my useful data is of somewhere around 20Mb.
Can i restrict the unwanted data?
I have list of keywords, for which log event is required and to be indexed.

Is it possible to do the same at Universal Fowarder level?

TIA

0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi òsarvesh_11,
you can filter your logs on Indexers or (if present) on Heavy Forwarders.
The instructions to filter events are at https://docs.splunk.com/Documentation/Splunk/8.0.3/Forwarding/Routeandfilterdatad#Filter_event_data_...

As you can read, you can:

  • discard all events and take some of them,
  • take all events and discard some of them.

The job to do is to find one or more regex to filter your data.

Only one hint: put attention to your architecture, because the props.conf and transforms.conf files to filter events must be located on:

  • Indexers (if you haven't Heavy Forwarders),
  • on Heavy Forwarders (if present and oll the traffic pass through them),
  • both on Indexers and Heavy Forwarders, if you have a mixed situation (some Universal Forwarders directly send data to Indexers and sone others through HFs).

Ciao.
Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi òsarvesh_11,
you can filter your logs on Indexers or (if present) on Heavy Forwarders.
The instructions to filter events are at https://docs.splunk.com/Documentation/Splunk/8.0.3/Forwarding/Routeandfilterdatad#Filter_event_data_...

As you can read, you can:

  • discard all events and take some of them,
  • take all events and discard some of them.

The job to do is to find one or more regex to filter your data.

Only one hint: put attention to your architecture, because the props.conf and transforms.conf files to filter events must be located on:

  • Indexers (if you haven't Heavy Forwarders),
  • on Heavy Forwarders (if present and oll the traffic pass through them),
  • both on Indexers and Heavy Forwarders, if you have a mixed situation (some Universal Forwarders directly send data to Indexers and sone others through HFs).

Ciao.
Giuseppe

0 Karma

sarvesh_11
Communicator

Hi @gcusello ,
Yeah i am clear about indexer and/or Heavy forwarder, i was checking if we can place props.conf and transforms.conf at Universal Forwarder.
To filter out the logs at forwarder level only.

Because i ready mixed views for this.

Thanks,

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sarvesh_11,
at Universal Forwarders level, you can filter events only from Windows eventlogs, as you can see at https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#WINDOWS_INPUTS:

Ciao.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...