Getting Data In

How to assign syslog file to specific index based on file name in transforms.conf?

Builder

Can you please provide sample configuration for the below, We have multiple forwarding sources and they are using syslog-ng. how to assign index based on log filename at indexer side.

Example :

mynbc-syslog-2015-07-30.log pattern files to go "mynbc" index
msnbc-syslog-2015-07-29.log patter files to go "msnbc" index
bravotv-syslog-2015-08-01_1.log patter files to go "bravotv" index

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Filename-based transforms.conf use the source key from the metadata like this:

[send_to_index_by_source]
SOURCE_KEY = MetaData:Source
REGEX = ^source::/path/to/files/(\w+)-syslog
DEST_KEY = _MetaData:Index
FORMAT = $1

Reference that in props.conf with TRANSFORMS-foo = send_to_index_by_source.

View solution in original post

SplunkTrust
SplunkTrust

Filename-based transforms.conf use the source key from the metadata like this:

[send_to_index_by_source]
SOURCE_KEY = MetaData:Source
REGEX = ^source::/path/to/files/(\w+)-syslog
DEST_KEY = _MetaData:Index
FORMAT = $1

Reference that in props.conf with TRANSFORMS-foo = send_to_index_by_source.

View solution in original post

Builder

Forwarder- outputs.conf

[tcpout:splunkssl]
server = splunk.abc.com:9997
compressed = true

[tcpout-server://splunk.abc.com:9997]
sslCertPath = $SPLUNK_HOME/etc/certs/forwarder.pem
sslPassword = password
sslRootCAPath = $SPLUNK_HOME/etc/certs/cacert.pem

0 Karma

Builder

trying with the below config, seems its not working for me, Can you please correct this if anything wrong,

props.conf

[syslog]
TRANSFORMS-idx_routing = generic_idx_routing

transforms.conf (created index abc)

[generic_idx_routing]
SOURCE_KEY = MetaData:Host
REGEX = abc.xyz.\nbcu.com
DEST_KEY = _MetaData:Index
FORMAT = $1

inputs.conf

[splunktcp-ssl:9997]
compressed = true
sourcetype = syslog

[SSL]
password = password
requireClientCert = false
rootCA = $SPLUNK_HOME/etc/certs/cacert.pem
serverCert = $SPLUNK_HOME/etc/certs/splunknode.pem

0 Karma

Builder

Can you please provide the sample REGEX while using the the file name pattern like this ${sitename}.${logname} ${Existing MSG}. we want to parse the sitename and assign to index name.

0 Karma