Getting Data In

Input data getting source type changed

Jon_Irish
Explorer

I used to have a PaloAlto firewall and i had it setup to syslog on ump/5514. I was also running a couple of PaloAlto applications. I have retired the PaloAlto firewall and I uninstalled the apps via the "splunk remove app [appname] -auth :" command. I have recently installed a pFsense firewall in its place, and it to is setup to syslog via udp/5514. I am ingesting the new syslog data fine, but all of it is getting tagged with a source type of "pan:log". This is what the old PaloAlto data was tagged with so it worked with the PA applications. I have verified that my Data Inputs setting for udp/5514 is set to use a source type of "pfsense_syslog". Thus, something is overriding this. I have searched my system for a non-default transforms.conf, but all I see are the "default" examples.

Any ideas where I can look to determine what is causing this?

Thanks!
Jon

0 Karma

mattymo
Splunk Employee
Splunk Employee

Hey Jon_Irish,

Can you check the output of ./splunk btool inputs list udp://5514 --debug? Or just ./splunk btool inputs list udp --debug

[splunker@n00bserver bin]$ ./splunk btool inputs list udp://5514 --debug
/home/splunker/splunk/etc/apps/launcher/local/inputs.conf            [udp://5514]
/home/splunker/splunk/etc/system/default/inputs.conf                 _rcvbuf = 1572864
/home/splunker/splunk/etc/apps/launcher/local/inputs.conf            connection_host = ip
/home/splunker/splunk/etc/apps/Splunk_TA_windows/default/inputs.conf evt_dc_name = 
/home/splunker/splunk/etc/apps/Splunk_TA_windows/default/inputs.conf evt_dns_name = 
/home/splunker/splunk/etc/system/local/inputs.conf                   host = n00bserver
/home/splunker/splunk/etc/system/default/inputs.conf                 index = default
/home/splunker/splunk/etc/apps/launcher/local/inputs.conf            source = syslog
/home/splunker/splunk/etc/apps/launcher/local/inputs.conf            sourcetype = syslog

Or in the GUI (if standalone) Settings > Data Inputs and check for the 5514 config.

That sourcetype is set in the inputs.conf, then the pan:log sourcetype is in the props.conf, which you can look for with ./splunk btool props pan:log --debug`

I assume that because its coming in on the same listener, it is simply applying the same settings??

- MattyMo
0 Karma

Jon_Irish
Explorer

Thanks for the suggestions mmodestino,
I tried all three suggestions, but nothing really grabs my attention:

# ./splunk btool props list pan:log --debug ==> no output
# ./splunk btool inputs list udp://5514 --debug ==> no output
# ./splunk btool inputs list udp --debug
/Applications/Splunk/etc/system/default/inputs.conf              [udp]
/Applications/Splunk/etc/system/default/inputs.conf              _rcvbuf = 1572864
/Applications/Splunk/etc/system/default/inputs.conf              connection_host = ip
/Applications/Splunk/etc/system/local/inputs.conf                host = Jons-iMac.local
/Applications/Splunk/etc/system/default/inputs.conf              index = default
/Applications/Splunk/etc/apps/TA-pfsense_a3sec/local/inputs.conf [udp://192.168.1.2:5514]
/Applications/Splunk/etc/system/default/inputs.conf              _rcvbuf = 1572864
/Applications/Splunk/etc/apps/TA-pfsense_a3sec/local/inputs.conf connection_host = ip
/Applications/Splunk/etc/system/local/inputs.conf                host = Jons-iMac.local
/Applications/Splunk/etc/apps/TA-pfsense_a3sec/local/inputs.conf index = gw_pfsense
/Applications/Splunk/etc/apps/TA-pfsense_a3sec/local/inputs.conf sourcetype = pfsense_syslog
/Applications/Splunk/etc/apps/search/local/inputs.conf           [udp://514]
/Applications/Splunk/etc/system/default/inputs.conf              _rcvbuf = 1572864
/Applications/Splunk/etc/apps/search/local/inputs.conf           connection_host = ip
/Applications/Splunk/etc/system/local/inputs.conf                host = Jons-iMac.local
/Applications/Splunk/etc/apps/search/local/inputs.conf           index = main
/Applications/Splunk/etc/apps/search/local/inputs.conf           sourcetype = syslog
# ./splunk btool props pan:log --debug
Invalid command: pan:log

Thanks!
Jon

0 Karma

mattymo
Splunk Employee
Splunk Employee

looks good, did you restart splunk?

./splunk restart

- MattyMo
0 Karma

jon_d_irish_ctr
Path Finder

After restarting, it appears that the sourcetype is now correct. Odd that a restart was required. I would have thought that I would have been notified of a need to reboot when I uninstalled the applications. Oh well, all is well now. Thanks for the help!

Jon

0 Karma

mattymo
Splunk Employee
Splunk Employee

ah nice! now I can sleep better at night! 😉

Good reference here regarding what config changes require restart, cause editing the conf files won't alert you to needing to restart...

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Configurationfilechangesthatrequirerestart

- MattyMo
0 Karma

jon_d_irish_ctr
Path Finder

LOL 😉

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...