Hi,
I've a full forwarder on machine A pointing at some log files in c:\temp*.log. These are being forwared to the full splunk install on machine B. I only want send the lines in the log files on machine A that contain the string [1:] or a [2:] to the splunk indexer from the forwarder on Machine B.
My \etc\apps\search\local\inputs.conf file looks like:
[monitor://c:\temp\log\*.log]
disabled = false
My \etc\apps\search\local\props.conf file looks like:
[splunkd]
EXTRACT-fields = (?i)^(?:[^ ]* ){2}(?:[+\-]\d+ )?(?P<log_level>[^ ]*)\s+(?P<component>[^ ]+) - (?P<message>.+)
[splunk_web_service]
EXTRACT-useragent = userAgent=(?P<browser>[^ (]+)
[source:c:\temp\log\...]
TRANSFORMS-dp = setTypeOne, setTypeTwo
My \etc\apps\search\local\transforms.conf looks like:
# Version 6.0.2
[setTypeOne]
DEST_KEY = MetaData:Sourcetype
REGEX = \[1:1\]
FORMAT = sourcetype::dp
[setTypeTwo]
DEST_KEY = MetaData:Sourcetype
REGEX = \[1:2\]
FORMAT = sourcetype::dp
The problem are:
Any pointers would be gratefully received.
Thanks in advance.
Maybe I seriously misunderstanding something, but these configs do not do anything of the sort.
On your full forwarder, do the following assuming that you keep events that contain [1:]
or [2:]
;
keep your inputs.conf as it is.
props.conf
[source::c:\temp\log\*.log]
TRANSFORMS-do_stuff = setOneTwo, setnull, keepOneTwo
transforms.conf
[setOneTwo]
DEST_KEY = MetaData:Sourcetype
REGEX = \[(1|2):\]
FORMAT = sourcetype::dp
[setnull]
DEST_KEY = queue
REGEX = .
FORMAT = nullQueue
[keepOneTwo]
DEST_KEY = queue
REGEX = \[(1|2):\]
FORMAT = indexQueue
The field extractions and that stuff you do one the search head/indexer.
EDIT: updated the source
stanza in props.conf to reflect on the actual path (included wildcard). Should work better now.
Hope this helps,
K
see update to answer above.
As long as the search app is enabled on the forwarder, you can put your settings there. However, to be absolutely sure that they are being honoured, you should put them in $SPLUNK_HOME/etc/system/local
, where $SPLUNK_HOME
the installation directory - normally c:\program files\splunk
or /opt/splunk
. This location overrides any other setting, but has the disadvantage that configurations here cannot be altered from a Deployment Server. However, I don't think config file precedence plays a part in the behaviour you're (not) seeing.
Am I correct in where the config files are kept - \Splunk\etc\apps\search\local. I notice that there is a \Splunk\etc\apps\SplunkForwarder\local directory. What( if anything ) should be kept in there with respect to my aim of filtering the data sent to the remote splunk server.
Thanks
Thanks, but this hasn't changed the behaviour at all. All the log file lines are still being posted, so no filtering is occuring at the heavy forwarder. The sourcetype is the name of the file, rather than dp.