I know it is possible to skip lines in an input, however, I have the case where I want to skip part of a line.
For example, I have an inputs.conf stanza like the following:
And I have the following log file, example.log:
time/fieldb/fieldc 13:50,200,300 time/fieldb/fieldc 14:00,210,310 time/fieldb/fieldc 14:10,223,305 time/fieldb/fieldc 14:20,215,307 ...
I want to only index the part after the space, due to having the index size as small as possible.
Is it possible to somehow skip the "time/fieldb/fieldc"-part from being indexed?
It is possible to remove/replace certain part of the data before it's indexed. It's generally used for masking of sensitive data. I would throw a caution before going into details that it add overhead to indexer/heavy forwarder as Splunk now has to to additional processing of each event.
You can use SEDCMD script in props.conf OR add a transforms.conf to achieve the same. See this for more details on data masking
In your props.conf on INdexer/Heavy Forwarder (if any), add this to your sourcetype.
props.conf (on Indexer/Heavy Forwarder)
[YourSourceType] ..other settings.. SEDCMD-removeheader = s/^(\S+\s+)(.*)/\2/g