All Apps and Add-ons

Integrating Splunk with TA and APP for pfSense by A3Sec, I can search log data, but why is no data shown on the app dashboard?

cesarfabre
Explorer

Hi guys,

I'm having problems in integrating between Splunk and pfSense APP.

  1. In pfSense Firewall version 2.2.4, I did the following: Status -> System logs -> Settings Remote Syslog Servers: 10.xx.xx.xx:25514 (IP do Splunk Server) Save button

The data are being sent to the splunk. Looks like this:

14:07:25.769718 IP 10.xx.xx.xx.53366 > 10.xx.xx.xx.25514: UDP, length 98
14:07:25.769744 IP 10.xx.xx.xx.53366 > 10.xx.xx.xx.25514: UDP, length 98
  1. In Splunk 6.2.3 I did the following: First, I created an index named gw_pfsense. Second, I created a Data Input as: UDP: Configuration Manual source: pfsense_syslog sourcetype: pfsense_webui index: gw_pfsense

In search, I can see the logs from the web UI, but no data is shown on the Dashboard of pfsense APP.

Can you help me?

Tks,
Cesar

0 Karma
1 Solution

Richfez
SplunkTrust
SplunkTrust

According to the docs for that app,

Input requirements: Data must be source typed as 'pfsense_syslog' and stored in a index named 'gw_pfsense'.

That statement isn't 100% clear, but you may have to set "sourcetype=pfsense_syslog". It's worth a shot, anyway.

If that works, you may want to provide some kind feedback to the author on how awesome and useful his app is, but that perhaps he should be more clear that "source typed as 'pfsense_syslog' " does actually mean "sourcetype=pfsense_syslog".

If it doesn't work, you'll probably have to review what the app's searches are looking for and go from there.

View solution in original post

0 Karma

Richfez
SplunkTrust
SplunkTrust

According to the docs for that app,

Input requirements: Data must be source typed as 'pfsense_syslog' and stored in a index named 'gw_pfsense'.

That statement isn't 100% clear, but you may have to set "sourcetype=pfsense_syslog". It's worth a shot, anyway.

If that works, you may want to provide some kind feedback to the author on how awesome and useful his app is, but that perhaps he should be more clear that "source typed as 'pfsense_syslog' " does actually mean "sourcetype=pfsense_syslog".

If it doesn't work, you'll probably have to review what the app's searches are looking for and go from there.

View solution in original post

0 Karma

Richfez
SplunkTrust
SplunkTrust

All right, I think this is fixable. So that future "troubleshooting" sessions on other problems are easier, I'll explain how I got to where I got. This is helpful in a general sense, but if you want to skip to the fix, check the "Problem!" section below.

First, let's recount what we know is working and see what's left. The input is coming in from the pfsense box and is tagged as sourcetype pfsense_syslog. We also know that the various transforms are working, since sourcetype is being changed on some events from pfsense_syslog to pfsense_webui. That's all working because you can search those events and see the events with changed sourcetypes.

Now what isn't working is those events showing on the dashboard.

I downloaded the app, extracted it (but didn't install it, I'm just browsing files) and took a look at some of the searches. You can browse the filesystem and read files, or you can click Edit, Edit Panels, then pick a search and edit the search string, or Edit, Edit Source to view the source files.

Problem!

A typical search string on a dashboard starts like so...

index=homemonitor sourcetype=$sourcetype$ | ...

So, it's looking specifically for data in an index called "homemonitor". But your data is in an index called "gw_pfsense". That's why it can't find it.

Honestly, I think the easiest fix for this is to a) Create an index "homemonitor", then b) change your input to save to that index instead of gw_pfsense. Your dashboards should all start working after that. (Well, once data comes in, anyway).

There are other solutions, but they aren't probably something you want to maintain long term (like changing all the "index=..." strings in all the searches.

Hope this helps!

0 Karma

cesarfabre
Explorer

Hi Rich,

Still not working with "sourcetype=pfsense_syslog" and restart on Splunk Server. Look my inputs.conf file:
[udp://25514]
connection_host = ip
index = gw_pfsense
source = syslog
sourcetype = pfsense_syslog

Fallows the results from search "index=gw_pfsense | chart count by sourcetype"

Events: 113
sourcetype (2) - pfsense_webui and pfsense_syslog

pfsense_webui data
Aug 24 08:58:22 10.xx.xx.xx Aug 24 08:58:22 php-fpm[34674]: /index.php: Successful login for user 'admin' from: 10.xx.xx.xx host = 10.xx.xx.xx source = syslog sourcetype = pfsense_webui authentication

pfsense_syslog data
Aug 23 07:58:17 10.xx.xx.xx Aug 23 07:58:17 kernel: arp: 172.xx.xx.xx moved from f8:a9:d0:6a:b6:cb to d8:bb:2c:7d:dd:90 on bce1 host = 10.xx.xx.xx source = syslog sourcetype = pfsense_syslog

Do you have another ideas?

Thank you so much!
Cesar

0 Karma

cesarfabre
Explorer

Hi Rich,

Still not working with "sourcetype=pfsense_syslog". Fallows the standard XML queries in this APP:

index=gw_pfsense sourcetype=pfsense_webui | timechart count(action) by action usenull=f useother=f
index=gw_pfsense sourcetype=pfsense_webui action=* | table _time,action,user

Do you have another idea?

Tks,
Cesar

0 Karma

Richfez
SplunkTrust
SplunkTrust

So, on the UDP input you have set up, it now says incoming data on that port is set to a sourcetype of pfsense_syslog? And you restarted Splunk afterwards?

If it is not working after those things, let's check that it's indeed right. Try searching "index=gw_pfsense | chart count by sourcetype" over some reasonable time frame, which will confirm we have a bunch of correctly-sourcetyped events in the right index. I don't know the ratio of each type to the other, but you should see events of various types (look at the docs near the bottom for examples).

Check that and post back the results!

Oh, one other VERY simple thing to check - what's the time set to on the pfsense box? Is that OK? Also, if the above search I suggest doesn't return data, change it to "all time" and check your events. Maybe - it's a long shot but an easy one - the data's coming in and it's just "out of the time frame" the app is set to by default.

0 Karma