https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132198#M27200 <P>Hello All,</P> <P>I am working on props.conf and transforms.conf files to clean some data before indexing the data into splunk so i have some data like </P> <H1>Version: 1.0</H1> <H1>Fields: date time cs-method cs-uri sc-status cs-username cs(User-Agent) s-sitename s-ip s-port time-taken cs(Cookie) sc-substatus cs-host cs-version</H1> <H1>Software: WebLogic</H1> <H1>Start-Date: 2014-09-04 00:00:00</H1> <P>I want to exclude these lines while indexing data into Splunk. i know we can use blacklist and whitelist to exclude the files in inputs.conf file </P> <P>Can anyone help me out here.</P> <P>Thanks</P> Fri, 19 Sep 2014 14:19:14 GMT gajananh999 2014-09-19T14:19:14Z https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132198#M27200 <P>Hello All,</P> <P>I am working on props.conf and transforms.conf files to clean some data before indexing the data into splunk so i have some data like </P> <H1>Version: 1.0</H1> <H1>Fields: date time cs-method cs-uri sc-status cs-username cs(User-Agent) s-sitename s-ip s-port time-taken cs(Cookie) sc-substatus cs-host cs-version</H1> <H1>Software: WebLogic</H1> <H1>Start-Date: 2014-09-04 00:00:00</H1> <P>I want to exclude these lines while indexing data into Splunk. i know we can use blacklist and whitelist to exclude the files in inputs.conf file </P> <P>Can anyone help me out here.</P> <P>Thanks</P> Fri, 19 Sep 2014 14:19:14 GMT https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132198#M27200 gajananh999 2014-09-19T14:19:14Z https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132199#M27201 <P>Are you trying to ignore the preamble of a log file? If so, give <CODE>PREAMBLE_REGEX</CODE> in props.conf a shot. See <A href="http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Extractfieldsfromfileheadersatindextime#Props.conf_attributes_for_structured_data">http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Extractfieldsfromfileheadersatindextime#Props.conf_attributes_for_structured_data</A> for documentation.</P> Fri, 19 Sep 2014 14:29:26 GMT https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132199#M27201 martin_mueller 2014-09-19T14:29:26Z https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132200#M27202 <P>Another option is the nullQueue filtering, but has more cost at indextime to parse all the events.</P> <P><A href="http://docs.splunk.com/Documentation/Splunk/6.1.3/Forwarding/Routeandfilterdatad#Discard_specific_events_and_keep_the_rest">http://docs.splunk.com/Documentation/Splunk/6.1.3/Forwarding/Routeandfilterdatad#Discard_specific_events_and_keep_the_rest</A></P> Fri, 19 Sep 2014 14:47:54 GMT https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132200#M27202 yannK 2014-09-19T14:47:54Z https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132201#M27203 <P>Hello Martin,</P> <P>will this work to remove lines from file which are starts with #?</P> <P>PREAMBLE_REGEX=^#</P> <P>Thanks<BR /> Gajanan HIroji</P> Fri, 19 Sep 2014 15:45:36 GMT https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132201#M27203 gajananh999 2014-09-19T15:45:36Z https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132202#M27204 <P>Hello Yannk,</P> <P>nullQueue is not able to remove # lines from file here is the prop<BR /> 9/17/01 1:02:50.000 AM<BR /><BR /> Example:</P> <H1>Version: 1.0</H1> <H1>Fields: date time cs-method cs-uri sc-status cs-username cs(User-Agent) s-sitename s-ip s-port time-taken cs(Cookie) sc-substatus cs-host cs-version</H1> <H1>Software: WebLogic</H1> <P>It taking this whole thing as one event</P> <P>Thanks<BR /> Gajanan Hiroji</P> Fri, 19 Sep 2014 16:02:53 GMT https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132202#M27204 gajananh999 2014-09-19T16:02:53Z https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132203#M27205 <P>That should do it.</P> Fri, 19 Sep 2014 21:35:20 GMT https://community.splunk.com/t5/Getting-Data-In/how-to-blacklist-events-from-file/m-p/132203#M27205 martin_mueller 2014-09-19T21:35:20Z