Getting Data In

On a heavy forwarder, can I forward a subset of data to syslog and drop everything else?

reswob4
Builder

Here is my situation: I have a Windows HF that is collecting a lot of different data. Some via powershell scripts, some via WMI, some via file monitoring locally and over UNC paths. All of that data is being forwarded to two indexes.

A few weeks ago I configured one of the file monitoring inputs to send a copy of the data it collected to a syslog server.

I now need to send that data (collected via file monitoring) to the syslog server and NOT to the indexers. IOW, I want all data collected by this HF to go to the indexers, EXCEPT this data which should be sent to the syslog server ONLY.

How do I do that?

I've read through this which helped me get the current configuration:
http://docs.splunk.com/Documentation/SplunkCloud/6.6.1/Forwarding/Forwarddatatothird-partysystemsd

Here are my config files:

.../etc/apps/myapp/local/props.conf:

[WinDNS]
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE_DATE = True
MAX_EVENTS = 1000
EXTRACT-Domain = (?i) .*? \.(?P<Domain>[-a-zA-Z0-9@:%_\+.~#?;//=]{2,256}\.[a-z]{2,6})
EXTRACT-src = (?i) [Rcv|Snd] (?P<source_address>\d+\.\d+\.\d+\.\d+)
EXTRACT-Threat_ID,Context,Int_packet_ID,proto,mode,Xid,type,Opcode,Flags_Hex,char_code,ResponseCode,question_type = .+?[AM|PM]\s+(?<Threat_ID>\w+)\s+(?<Context>\w+)\s+(?<Int_packet_ID>\w+)\s+(?<proto>\w+)\s+(?<mode>\w+)\s+\d+\.\d+\.\d+\.\d+\s+(?<Xid>\w+)\s(?<type>(?:R)?)\s+(?<Opcode>\w+)\s+\[(?<Flags_Hex>\w+)\s(?<char_codes>.+?)(?<ResponseCode>[A-Z]+)\]\s+(?<question_type>\w+)\s
EXTRACT-Authoritative_Answer,TrunCation,Recursion_Desired,Recursion_Available = (?m) .+?Message:\W.+\W.+\W.+\W.+\W.+AA\s+(?<Authoritative_Answer>\d)\W.+TC\s+(?<TrunCation>\d)\W.+RD\s+(?<Recursion_Desired>\d)\W.+RA\s+(?<Recursion_Available>\d)
TRANSFORMS-droplocal2 = droplocal2
TRANSFORMS-dropbach = dropbach
#TRANSFORMS-dropall = dropall
SEDCMD-win_dns = s/\(\d+\)/./g
TRANSFORMS-dns = send_to_syslog

.../etc/apps/myapp/local/transforms.conf

[dropbach]
REGEX = \[.+?\]\s+\w+\s+.+?BACH
DEST_KEY=queue
FORMAT=nullQueue

[droplocal2]
REGEX = \[.+?\]\s+\w+\s+.+?local
DEST_KEY=queue
FORMAT=nullQueue

[send_to_syslog]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = my_syslog_group

#[dropall]
#REGEX = .
#DEST_KEY=queue
#FORMAT=nullQueue

.../etc/system/local/outputs.conf

[tcpout]
defaultGroup = default-autolb-group
indexAndForward = 0

[tcpout-server://splunk-01:9997]

[tcpout:default-autolb-group]
disabled = false
server = splunk-01:9997,splunk-02:9997

[tcpout-server://splunk-02:9997]  # not sure why this is here....

[syslog:my_syslog_group]
server = 1.1.1.5:514

As you can tell, I tried to add a 'dropall', but that just dropped everything without sending a copy to the syslog server first.

I then found this forum post:

https://answers.splunk.com/answers/4083/can-i-route-some-data-as-syslog-output-to-multiple-destinati...

which seems to imply that to do what I want to do, I need to modify the outputs.conf so the defaultgroup=nothing and then modify all the props.conf and transforms.conf files for all my inputs to point to the "default-autolb-group" in outputs.conf that sends to the indexers, and then for this app have the ONLY output reference pointing to the "my_syslog_group" in outputs.conf.

Is that correct or something else?

0 Karma

reswob4
Builder
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...