https://community.splunk.com/t5/Getting-Data-In/How-to-configure-nullQueue-to-filter-out-repetitive-lines-from-a/m-p/229067#M44561 <P>Hi rturecek,</P> <P>This example discards the example line in your events by sending them to <CODE>nullQueue</CODE>:</P> <P>In <CODE>props.conf</CODE>, set the <CODE>TRANSFORMS-</CODE> attribute:</P> <PRE><CODE>[source::/var/log/*/CheckPointReconciler.log*] TRANSFORMS-001_CheckPointReconciler_NullQueue = CheckPointReconciler_NullQueue </CODE></PRE> <P>Create a corresponding stanza in <CODE>transforms.conf</CODE>. Set DEST_KEY to <CODE>queue</CODE> and FORMAT to <CODE>nullQueue</CODE>:</P> <PRE><CODE>[CheckPointReconciler_NullQueue] REGEX = [\d-\s:\.]+INFO\s+[^\s]+\snone\s+main\s+Reading\scheckpoint\sdirectory DEST_KEY = queue FORMAT = nullQueue </CODE></PRE> <P>You can also set the sourcetype instead of the source in <CODE>props.conf</CODE>. Do this either on a heavyweight forwarder or the indexer and remember to restart Splunk afterwards. </P> <P>Hope this helps ...</P> <P>cheers, MuS</P> Mon, 11 Jan 2016 21:16:35 GMT MuS 2016-01-11T21:16:35Z https://community.splunk.com/t5/Getting-Data-In/How-to-configure-nullQueue-to-filter-out-repetitive-lines-from-a/m-p/229066#M44560 <P>Splunk 6.1 Linux indexers feeding server with master license.</P> <P>I am trying to filter out repetitive lines from a log file before they are indexed. Need to configure the 3 conf files: inputs, props and transform.</P> <P>The server where the log file is located(different from indexer server where conf files are located): mmd5<BR /> mmd5 path/log: <CODE>/var/log/*/CheckPointReconciler.log*</CODE><BR /> Log line I want to filter out to nullQueue ( filter on 'Reading')</P> <PRE><CODE> 2015-12-30 2:02:12.736 14181:4 INFO job_id none main Reading checkpoint directory /mm/feeder/chkpt </CODE></PRE> Mon, 11 Jan 2016 19:42:14 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-configure-nullQueue-to-filter-out-repetitive-lines-from-a/m-p/229066#M44560 rturecek 2016-01-11T19:42:14Z https://community.splunk.com/t5/Getting-Data-In/How-to-configure-nullQueue-to-filter-out-repetitive-lines-from-a/m-p/229067#M44561 <P>Hi rturecek,</P> <P>This example discards the example line in your events by sending them to <CODE>nullQueue</CODE>:</P> <P>In <CODE>props.conf</CODE>, set the <CODE>TRANSFORMS-</CODE> attribute:</P> <PRE><CODE>[source::/var/log/*/CheckPointReconciler.log*] TRANSFORMS-001_CheckPointReconciler_NullQueue = CheckPointReconciler_NullQueue </CODE></PRE> <P>Create a corresponding stanza in <CODE>transforms.conf</CODE>. Set DEST_KEY to <CODE>queue</CODE> and FORMAT to <CODE>nullQueue</CODE>:</P> <PRE><CODE>[CheckPointReconciler_NullQueue] REGEX = [\d-\s:\.]+INFO\s+[^\s]+\snone\s+main\s+Reading\scheckpoint\sdirectory DEST_KEY = queue FORMAT = nullQueue </CODE></PRE> <P>You can also set the sourcetype instead of the source in <CODE>props.conf</CODE>. Do this either on a heavyweight forwarder or the indexer and remember to restart Splunk afterwards. </P> <P>Hope this helps ...</P> <P>cheers, MuS</P> Mon, 11 Jan 2016 21:16:35 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-configure-nullQueue-to-filter-out-repetitive-lines-from-a/m-p/229067#M44561 MuS 2016-01-11T21:16:35Z https://community.splunk.com/t5/Getting-Data-In/How-to-configure-nullQueue-to-filter-out-repetitive-lines-from-a/m-p/229068#M44562 <P>Thank you, The * in the path had to defined a little more precisely but now works.</P> Fri, 15 Jan 2016 20:18:18 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-configure-nullQueue-to-filter-out-repetitive-lines-from-a/m-p/229068#M44562 rturecek 2016-01-15T20:18:18Z