I know it is possible to skip lines in an input, however, I have the case where I want to skip part of a line.
For example, I have an inputs.conf stanza like the following:
[monitor://C:\temp\example.log]
...
And I have the following log file, example.log:
time/fieldb/fieldc 13:50,200,300
time/fieldb/fieldc 14:00,210,310
time/fieldb/fieldc 14:10,223,305
time/fieldb/fieldc 14:20,215,307
...
I want to only index the part after the space, due to having the index size as small as possible.
Is it possible to somehow skip the "time/fieldb/fieldc"-part from being indexed?
It is possible to remove/replace certain part of the data before it's indexed. It's generally used for masking of sensitive data. I would throw a caution before going into details that it add overhead to indexer/heavy forwarder as Splunk now has to to additional processing of each event.
You can use SEDCMD script in props.conf OR add a transforms.conf to achieve the same. See this for more details on data masking
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Anonymizedata
Do you have an example explaining how to use the SEDCMD here? That would help me a lot.
In your props.conf on INdexer/Heavy Forwarder (if any), add this to your sourcetype.
props.conf (on Indexer/Heavy Forwarder)
[YourSourceType]
..other settings..
SEDCMD-removeheader = s/^(\S+\s+)(.*)/\2/g