modsecurity / Source doesn't show up

[thierryit]

Hi,

Running both Splunk server and Splunkforwarder on V6.0.2.
Both machine (web server and Splunk server) have their FW off.
After an "netstat -a" on both machine, I can see that there is a TCP connection established between my web server (port TCP 56xxx) and my Splunk server (port TCP 9997).
My inputs.conf is:

[monitor:///var/log/apache2/modsec_audit.log]
disabled = false
host = name_of_my_server
index = main
sourcetype = modsec_audit

On my Splunk server when going to: Search & Reporting/Search/Data Summary I only see one source (udp:514 -> my firewall) nothing else.
On hosts, I can see only my firewall .....

If I add in my inputs.conf one of my apache2 log, as example access.log, it will work like a charm ...
But not for my modsecurity log file .....

Any ideas ?

Thx

[thierryit · Accepted solution]

Hi,

WORKING !!!

I add to declare in "/opt/splunkforwarder/etc/system/default/props.conf"

[modsec_audit]
CHARSET = AUTO
NO_BINARY_CHECK = true

And in modsecurity.conf, I had to change this:
SecAuditLogType Concurrent (instead of Serial)

Thx martin for your help.

[thierryit]

TaillingProcessor - Ignoring path=\"/var/log/apache2/modsec_audit.log\" due to: Bug: tried to check/configure STData processing but have no pending metadata.

[thierryit]

TailingProcessor-Ignoring file '/var/log/apache2/modsec_audit.log' due to: binary

In props.conf, I have add:

NO_BINARY_CHECK = true
CHARSET = AUTO

Thx

[martin_mueller]

Take a look at this similar problem: http://answers.splunk.com/answers/117915/tailingprocessor-ignoring-pathpathtoxyz-due-to-bug-tried-to...

[thierryit]

props.conf of the forwarder ?

[thierryit]

index=_internal host="hostname"

I have as example:

3/29/14
6:28:09.847 PM

03-29-2014 18:28:09.847 +0200 ERROR TailingProcessor - Ignoring path="/var/log/apache2/modsec_audit.log" due to: Bug: tried to check/configure STData processing but have no pending metadata.
host = hostname source = /opt/splunkforwarder/var/log/splunk/splunkd.log sourcetype = splunkd
3/29/14
5:58:17.970 PM

03-29-2014 17:58:17.970 +0200 ERROR TailingProcessor - Ignoring path="/var/log/apache2/modsec_audit.log" due to: Bug: tried to check/configure STData processing but have no pending metadata.

[martin_mueller]

Run a search on the indexer, something like this:

index=_internal host=yourforwarderhost modsec_audit.log

[thierryit]

Can you be more precise ? What entry ? Where ?
Thx for your help even during the week end 🙂

[martin_mueller]

No entries in the _internal log files from that host?

[thierryit]

sudo ps auxxx |grep splunk*
root 1247 0.8 0.7 161860 32520 ? Sl 15:18 0:06 splunkd -p 8089 start
root 1251 0.0 0.0 49116 2884 ? Ss 15:18 0:00 [splunkd pid=1247] splunkd -p 8089 start [process-runner]

-rw-r--r-- 1 root adm 8528077 Mar 29 14:52 modsec_audit.log

Thx

[martin_mueller]

Okay, is that file readable by the user running the forwarder?

[thierryit]

[monitor:///opt/splunkforwarder/etc/splunk.version]
_TCP_ROUTING = *
_rcvbuf = 1572864
host = server_hostname
index = _internal
sourcetype = splunk_version
[monitor:///opt/splunkforwarder/var/log/splunk]
_rcvbuf = 1572864
host = server_hostname
index = _internal
[monitor:///opt/splunkforwarder/var/log/splunk/splunkd.log]
_TCP_ROUTING = *
_rcvbuf = 1572864
host = server_hostname
index = _internal
[monitor:///var/log/apache2/modsec_audit.log]
_rcvbuf = 1572864
crcSalt =
disabled = false
host = toto.domain.org
index = main
sourcetype = modsec_audit

With or without crcSalt, same pb.

[martin_mueller]

That's utterly unreadable, but it seems to me as if it only lists Splunk's own internal log files - so it's not even trying to read your log.

Run this from the CLI of the forwarder:

/opt/splunkforwarder/bin/splunk cmd btool inputs list monitor

[thierryit]

file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished reading /opt/splunkforwarder/var/log/splunk/licenseusage.log
file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished reading /opt/splunkforwarder/var/log/splunk/metrics.log file position 1144937 file size 1144937 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100.00 type finished
reading /opt/splunkforwarder/var/log/splunk/scheduler.log
file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished

[thierryit]

A part of the answer, seems to be too big:
app canlist 1 canwrite 1 modifiable 0 owner system perms
read
* write
removable 0 sharing system eai:attributes
optionalFields
requiredFields
wildcardFields
inputs
/opt/splunkforwarder/var/log/splunk/audit.log
file position 50835 file size 50835 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100.00 type finished reading /opt/splunkforwarder/var/log/splunk/btool.log
file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished reading /opt/splunkforwarder/var/log/splunk/licenseaudit.log

[martin_mueller]

...in an https-capable client, such as your browser.

[thierryit]

splunk ~ # https://192.168.1.xx:8089/services/admin/inputstatus/TailingProcessor:FileStatus
-bash: https://192.168.1.xx:8089/services/admin/inputstatus/TailingProcessor:FileStatus: Aucun fichier ou dossier de ce type (No file or folder of this type)

[martin_mueller]

As suggested by the script run on the server, go to https://127.0.0.1:8089/services/admin/inputstatus/TailingProcessor:FileStatus (replace 127.0.0.1 with the forwarder's host).

[thierryit]

I am not a developer ... I do not understand your answer ... Sorry.

[martin_mueller]

You can call that REST endpoint manually.