Getting Data In

modsecurity / Source doesn't show up

Path Finder

Hi,

Running both Splunk server and Splunkforwarder on V6.0.2.
Both machine (web server and Splunk server) have their FW off.
After an "netstat -a" on both machine, I can see that there is a TCP connection established between my web server (port TCP 56xxx) and my Splunk server (port TCP 9997).
My inputs.conf is:

[monitor:///var/log/apache2/modsecaudit.log]
disabled = false
host = name
ofmyserver
index = main
sourcetype = modsec_audit

On my Splunk server when going to: Search & Reporting/Search/Data Summary I only see one source (udp:514 -> my firewall) nothing else.
On hosts, I can see only my firewall .....

If I add in my inputs.conf one of my apache2 log, as example access.log, it will work like a charm ...
But not for my modsecurity log file .....

Any ideas ?

Thx

Tags (2)
0 Karma
1 Solution

Path Finder

Hi,

WORKING !!!

I add to declare in "/opt/splunkforwarder/etc/system/default/props.conf"

[modsecaudit]
CHARSET = AUTO
NO
BINARY_CHECK = true

And in modsecurity.conf, I had to change this:
SecAuditLogType Concurrent (instead of Serial)

Thx martin for your help.

View solution in original post

0 Karma

Path Finder

Hi,

WORKING !!!

I add to declare in "/opt/splunkforwarder/etc/system/default/props.conf"

[modsecaudit]
CHARSET = AUTO
NO
BINARY_CHECK = true

And in modsecurity.conf, I had to change this:
SecAuditLogType Concurrent (instead of Serial)

Thx martin for your help.

View solution in original post

0 Karma

Path Finder

TaillingProcessor - Ignoring path=\"/var/log/apache2/modsec_audit.log\" due to: Bug: tried to check/configure STData processing but have no pending metadata.

0 Karma

Path Finder

TailingProcessor-Ignoring file '/var/log/apache2/modsec_audit.log' due to: binary

In props.conf, I have add:

NOBINARYCHECK = true
CHARSET = AUTO

Thx

0 Karma

SplunkTrust
SplunkTrust
0 Karma

Path Finder

props.conf of the forwarder ?

0 Karma

Path Finder

index=_internal host="hostname"

I have as example:

3/29/14
6:28:09.847 PM

03-29-2014 18:28:09.847 +0200 ERROR TailingProcessor - Ignoring path="/var/log/apache2/modsecaudit.log" due to: Bug: tried to check/configure STData processing but have no pending metadata.
host = hostname source = /opt/splunkforwarder/var/log/splunk/splunkd.log sourcetype = splunkd
3/29/14
5:58:17.970 PM

03-29-2014 17:58:17.970 +0200 ERROR TailingProcessor - Ignoring path="/var/log/apache2/modsec
audit.log" due to: Bug: tried to check/configure STData processing but have no pending metadata.

0 Karma

SplunkTrust
SplunkTrust

Run a search on the indexer, something like this:

index=_internal host=yourforwarderhost modsec_audit.log
0 Karma

Path Finder

Can you be more precise ? What entry ? Where ?
Thx for your help even during the week end 🙂

0 Karma

SplunkTrust
SplunkTrust

No entries in the _internal log files from that host?

0 Karma

Path Finder

sudo ps auxxx |grep splunk*
root 1247 0.8 0.7 161860 32520 ? Sl 15:18 0:06 splunkd -p 8089 start
root 1251 0.0 0.0 49116 2884 ? Ss 15:18 0:00 [splunkd pid=1247] splunkd -p 8089 start [process-runner]

-rw-r--r-- 1 root adm 8528077 Mar 29 14:52 modsec_audit.log

Thx

0 Karma

SplunkTrust
SplunkTrust

Okay, is that file readable by the user running the forwarder?

0 Karma

Path Finder

[monitor:///opt/splunkforwarder/etc/splunk.version]
TCPROUTING = *
rcvbuf = 1572864
host = server
hostname
index = internal
sourcetype = splunk
version
[monitor:///opt/splunkforwarder/var/log/splunk]
rcvbuf = 1572864
host = server
hostname
index = internal
[monitor:///opt/splunkforwarder/var/log/splunk/splunkd.log]
_TCP
ROUTING = *
rcvbuf = 1572864
host = server
hostname
index = internal
[monitor:///var/log/apache2/modsec
audit.log]
rcvbuf = 1572864
crcSalt =
disabled = false
host = toto.domain.org
index = main
sourcetype = modsec
audit

With or without crcSalt, same pb.

0 Karma

SplunkTrust
SplunkTrust

That's utterly unreadable, but it seems to me as if it only lists Splunk's own internal log files - so it's not even trying to read your log.

Run this from the CLI of the forwarder:

/opt/splunkforwarder/bin/splunk cmd btool inputs list monitor
0 Karma

Path Finder

file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished reading /opt/splunkforwarder/var/log/splunk/licenseusage.log
file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished reading /opt/splunkforwarder/var/log/splunk/metrics.log file position 1144937 file size 1144937 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100.00 type finished
reading /opt/splunkforwarder/var/log/splunk/scheduler.log
file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished

0 Karma

Path Finder

A part of the answer, seems to be too big:
app canlist 1 canwrite 1 modifiable 0 owner system perms
read
* write
removable 0 sharing system eai:attributes
optionalFields
requiredFields
wildcardFields
inputs
/opt/splunkforwarder/var/log/splunk/audit.log
file position 50835 file size 50835 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100.00 type finished reading /opt/splunkforwarder/var/log/splunk/btool.log
file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished reading /opt/splunkforwarder/var/log/splunk/licenseaudit.log

0 Karma

SplunkTrust
SplunkTrust

...in an https-capable client, such as your browser.

0 Karma

Path Finder
0 Karma

SplunkTrust
SplunkTrust

As suggested by the script run on the server, go to https://127.0.0.1:8089/services/admin/inputstatus/TailingProcessor:FileStatus (replace 127.0.0.1 with the forwarder's host).

0 Karma

Path Finder

I am not a developer ... I do not understand your answer ... Sorry.

0 Karma

SplunkTrust
SplunkTrust

You can call that REST endpoint manually.

0 Karma