Getting Data In

How to Remove the data getting indexed?

Inayath_khan
Path Finder

Hi Folks,

Can anyone suggest how to remove the below data getting indexed to indexer and also how to remove the data which is already indexed?

timestamp syslog_host user remote_host connection_id query_id operation database object

Labels (1)
Tags (1)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @Inayath_khan,
Splunk isn't a database, so you cannot modify or remove data after indexing:

Ciao.
Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Inayath_khan,
Splunk isn't a database, so you cannot modify or remove data after indexing:

Ciao.
Giuseppe

0 Karma

Inayath_khan
Path Finder

Thanks @gcusello can you help me at a configuration level what changes has to be made to stop selective data getting indexed?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Inayath_khan,
as described at https://docs.splunk.com/Documentation/Splunk/8.0.2/Forwarding/Routeandfilterdatad#Filter_event_data_... , you can filter data before indexing in two ways:

  • take all the data and delete some identified events,
  • delete all the data and take identified events.

In both cases you have to follow these steps:

  • identify events to filter (sourcetype or source or host);
  • identify rule to discard or take logs (regex);
  • modify props.conf and transforms.conf on indexers or (when present) on Heavy Forwarders;
  • put props.conf and transforms.conf in an App (better) or in $SPLUNK_HOME/etc/system/local .

Discard specific events and keep the rest:
In props.conf, set the TRANSFORMS-null attribute:

[your_sourcetype]
TRANSFORMS-null= setnull

in transforms.conf, set DEST_KEY to "queue" and FORMAT to "nullQueue":

[setnull]
REGEX = your_regex
DEST_KEY = queue
FORMAT = nullQueue

At the end restart Splunk Enterprise.

Keep specific events and discard the rest
In props.conf:

[your_sourcetype]
TRANSFORMS-set= setnull,setparsing

In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = your-regex
DEST_KEY = queue
FORMAT = indexQueue

At the end restart Splunk Enterprise.

Ciao.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...