RFC Splunk App syslog problems

[ency79]

Hello Community,

i hope you can support. I have a CloudFoundry Environment which send all logs to my splunk-forwarder on which i have installed syslog-ng 4.6. On the Splunk Server Side the Splunk App for RFC5424 has been installed and configured as documented.

My current syslog-ng.conf (without RFC5424) looks as follows (with syslog-ng 3.23):

@version:3.23

options {
    flush_lines(0);
    time_reopen(10);
    log_fifo_size(16384);
    chain_hostnames(off);
    use_dns(no);
    use_fqdn(no);
    create_dirs(yes);
    keep_hostname(yes);
    owner();  dir-owner();
    group();  dir-group();
    perm(-1);     dir-perm(-1);
    keep-timestamp(no);
    threaded(yes);
};

source s_tcp555 { tcp (ip("0.0.0.0") port(555) keep-alive(yes) max-connections(100) log-iw-size(10000)); };

destination env_logs { file("/var/log/syslog2splunk/env/${LOGHOST}/${HOST}/${YEAR}-${MONTH}-${DAY}_${HOUR}.log" template("${UNIXTIME} ${MSGHDR} ${MESSAGE}\n") frac-digits(3) time_zone("UTC") owner("splunk") dir-owner("splunk") group("splunk") dir-group("splunk")); };

log { source(s_tcp514); destination(env_logs); };

The inputs.conf:

[default]
host = my-splk-fwd
index = <my-splk-index-xxx>

[monitor:///var/log/syslog2splunk/env/*/*/*.log]
disabled = false
sourcetype = CF:syslog
host_segment = 6
crcSalt = <SOURCE>

You see that my CloudFoundry Environment is sending syslog over port 514 to the splunk forwarder which is then shipping them to the splunk server.

Now i have configured RFC5424 in syslog-ng.conf and also in the inputs.conf. My CF syslogs should be only formatted to RFC5424 and therefore i do not want to have in my syslog-ng.conf 2 sources/destinations and a new port. I would only like that my current syslogs will be formatted to rfc5424. But i also know that in the inputs.conf its not possible to configure 2 sourcetypes. So therefore i need to know how to configure those both files that my almost incoming syslog files will be formatted with rfc5424. I do not want to have two directories with exactly the same logs.

Here is my syslog-ng.conf (with syslog-ng 4.6):

@version: 4.6

options {
    flush_lines(0);
    time_reopen(10);
    log_fifo_size(16384);
    chain_hostnames(off);
    use_dns(no);
    use_fqdn(no);
    create_dirs(yes);
    keep_hostname(yes);
    owner(); dir-owner();
    group(); dir-group();
    perm(-1); dir-perm(-1);
    keep-timestamp(no);
    threaded(yes);
};

source s_tcp514 { tcp (ip("0.0.0.0") port(514)); };

destination env_logs { file("/var/log/syslog2splunk/env/${LOGHOST}/${HOST}/${YEAR}-${MONTH}-${DAY}_${HOUR}.log" template("<${PRI}>1 ${ISODATE} ${HOST} ${PROGRAM} ${PID} ${MSGID} ${STRUCTURED-DATA} ${MESSAGE}\n") frac-digits(3) time_zone("UTC") owner("splunk") dir-owner("splunk") group("splunk") dir-group("splunk")); };

destination rfc5424_logs { file("/var/log/syslog2splunk/rfc5424/${LOGHOST}/${HOST}/${YEAR}-${MONTH}-${DAY}_${HOUR}.log" template("<${PRI}>1 ${ISODATE} ${HOST} ${PROGRAM} ${PID} ${MSGID} ${STRUCTURED-DATA} ${MESSAGE}\n") frac-digits(3) time_zone("UTC") owner("splunk") dir-owner("splunk") group("splunk") dir-group("splunk")); };

# Log routing
log { source(s_tcp514); destination(env_logs); };

==> do i need here to add an additional source/destination or is this conf ok?

The new inputs.conf looks as follows:

[default]
host = my-splk-fwd
index = my-splk-index_xxx

[monitor:///var/log/syslog2splunk/env/*/*/*.log]
disabled = false
sourcetype = ENV:syslog
host_segment = 6
crcSalt = <SOURCE>

[monitor:///var/log/syslog2splunk/rfc5424/*/*/*.log]
disabled = false
sourcetype = rfc5424_syslog
host_segment = 6
crcSalt = <SOURCE>

With the syslog-ng.conf and inputs.conf i can see the source-type for rfc but from my opinion it is exactly the same output as before - so i do not recognize any difference.