- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How to configure in props/transforms.conf to remove the event data which does not contain any information in it?
Hi Splunk experts,
Just want to know how can I remove events which does not contain any information in it?
Example
sample events which have information on them.
I know that, we have to configure Route and Filter data in Props/transforms to achieve this but not sure about the Regex.
Props.conf details:
[who]
TRANSFORMS-null= setnull
Transforms.conf details:
[setnull]
REGEX =
DEST_KEY = queue
FORMAT = nullQueue
Kindly guide me on this
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Give this a try (changed strategy from 'Drop specific events' to 'Keep specific event and drop remaining').
Reference: http://docs.splunk.com/Documentation/Splunk/7.1.1/Forwarding/Routeandfilterdatad#Keep_specific_event...
Edit props.conf and add the following:
[who]
TRANSFORMS-set= setnull,setparsing
Edit transforms.conf and add the following:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
#basically keeping anything that has header and at least one data row
[setparsing]
REGEX = ^(USERNAME).+[\r\n]+\w+
DEST_KEY = queue
FORMAT = indexQueue
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi SomeSoni2,
Yes I had given a try with the above set of configuration in my test environment by uploading the data in raw text format via Add data -->upload --> select sourcetype but it did not fetch the required output.
Instead getting the below message.
No results found. Please change source type, adjust source type settings, or check your source file
Props.conf details:
[who]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
TRUNCATE=1000000
DATETIME_CONFIG = CURRENT
TRANSFORMS-set= setnull,setparsing
Transforms.conf details:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
#basically keeping anything that has header and at least one data row
[setparsing]
REGEX = ^(USERNAME).+[\r\n]+\w+
DEST_KEY = queue
FORMAT = indexQueue
Kindly guide me on this.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

YOu said you're uploading it from an instance. Did you setup these (props and transforms) on the same server/instance? If this a test environment, try setting up a monitoring (using inputs.conf) and test the ingestion, instead of using Add Data wizard.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
thanks Somesoni2, let me try that.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey it did not work, could you please guide me on this
USERNAME LINE HOSTNAME TIME
host = tt3crp00 source = who sourcetype = who
6/21/18
1:11:25.000 PM
USERNAME LINE HOSTNAME TIME
solarwinds pts/21 vmswpep02.XXXX.com Jun 21 13:11
host = tt3dev00 source = who sourcetype = who
6/21/18
1:11:25.000 PM
USERNAME LINE HOSTNAME TIME
solarwinds pts/0 vmswpep05.xxxx.com 2018-06-21 13:11
solarwinds pts/1 vmswpep05.xxxx.com 2018-06-21 13:11
the above props/transforms.conf are placed in the Heavy forwarder instances.
Kindly guide me to fix this issue.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Somesoni2, by applying the above stanza in props.conf and transforms.conf, the entire event information are removed from splunk and it throws the below message.
No results found. Please change source type, adjust source type settings, or check your source file
Could you please guide me on this to events which does not contain any information in it.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content


If the Question contains the example data, then perhaps something like:
REGEX = ^USERNAME\s+LINE\s+HOSTNAME\s+TIME$
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi cpetterborg, thanks for your effort on this, I had added the sample events containing the data, so can I use the above regex to remove events which does not contain any information in it.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content


If you are making the events be multi-line events (one for each time the command is run), then you can use the SEDCMD
in props.conf
to remove only the header lines.
If you are making each line be a separate event (as it appeared in your question), you can use something like my suggestion.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey we are splitting multiple events in to single individual events by using the line_breaker stanza in props.conf. And also we wanted to remove the events which does not contain any information in it.
1) To split multiple events in to single individual events.
[who]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
TRUNCATE=1000000
DATETIME_CONFIG = CURRENT
2) For removing the events which does not contain any information.
Props.conf
[who]
TRANSFORMS-null= setnull
Transforms.conf details:
[setnull]
REGEX = ^USERNAME\s+LINE\s+HOSTNAME\s+TIME$
DEST_KEY = queue
FORMAT = nullQueue
Kindly guide me on this.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

The REGEX will depend upon how your event looks like when it has information. Could you provide some sample events which have information on them?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Somesoni2, thanks for your effort on this, we are monitoring wtmpx file from the unix machines using the Splunk Add-on for Unix. Below are the sample events contains the information.
Index=unix sourcetype=who host=*
6/20/18
3:06:05.000 PM
USERNAME LINE HOSTNAME TIME
DBB019 pts/1 w442xty1.XXXX.com Jun 19 18:40
AMM007 pts/5 yb33gnn1.XXXX.com Jun 19 08:53
host = ttbmt02 source = who sourcetype = who
6/20/18
3:03:35.000 PM
USERNAME LINE HOSTNAME TIME
DBB019 pts/1 w442xty1.XXXX.com Jun 19 18:40
AMM007 pts/5 yb33gnn1.XXXX.com Jun 19 08:53
