Getting Data In

modsecurity / Source doesn't show up

thierryit
Path Finder

Hi,

Running both Splunk server and Splunkforwarder on V6.0.2.
Both machine (web server and Splunk server) have their FW off.
After an "netstat -a" on both machine, I can see that there is a TCP connection established between my web server (port TCP 56xxx) and my Splunk server (port TCP 9997).
My inputs.conf is:

[monitor:///var/log/apache2/modsec_audit.log]
disabled = false
host = name_of_my_server
index = main
sourcetype = modsec_audit

On my Splunk server when going to: Search & Reporting/Search/Data Summary I only see one source (udp:514 -> my firewall) nothing else.
On hosts, I can see only my firewall .....

If I add in my inputs.conf one of my apache2 log, as example access.log, it will work like a charm ...
But not for my modsecurity log file .....

Any ideas ?

Thx

Tags (2)
0 Karma
1 Solution

thierryit
Path Finder

Hi,

WORKING !!!

I add to declare in "/opt/splunkforwarder/etc/system/default/props.conf"

[modsec_audit]
CHARSET = AUTO
NO_BINARY_CHECK = true

And in modsecurity.conf, I had to change this:
SecAuditLogType Concurrent (instead of Serial)

Thx martin for your help.

View solution in original post

0 Karma

thierryit
Path Finder

Hi,

WORKING !!!

I add to declare in "/opt/splunkforwarder/etc/system/default/props.conf"

[modsec_audit]
CHARSET = AUTO
NO_BINARY_CHECK = true

And in modsecurity.conf, I had to change this:
SecAuditLogType Concurrent (instead of Serial)

Thx martin for your help.

0 Karma

thierryit
Path Finder

TaillingProcessor - Ignoring path=\"/var/log/apache2/modsec_audit.log\" due to: Bug: tried to check/configure STData processing but have no pending metadata.

0 Karma

thierryit
Path Finder

TailingProcessor-Ignoring file '/var/log/apache2/modsec_audit.log' due to: binary

In props.conf, I have add:

NO_BINARY_CHECK = true
CHARSET = AUTO

Thx

0 Karma

martin_mueller
SplunkTrust
SplunkTrust
0 Karma

thierryit
Path Finder

props.conf of the forwarder ?

0 Karma

thierryit
Path Finder

index=_internal host="hostname"

I have as example:

3/29/14
6:28:09.847 PM

03-29-2014 18:28:09.847 +0200 ERROR TailingProcessor - Ignoring path="/var/log/apache2/modsec_audit.log" due to: Bug: tried to check/configure STData processing but have no pending metadata.
host = hostname source = /opt/splunkforwarder/var/log/splunk/splunkd.log sourcetype = splunkd
3/29/14
5:58:17.970 PM

03-29-2014 17:58:17.970 +0200 ERROR TailingProcessor - Ignoring path="/var/log/apache2/modsec_audit.log" due to: Bug: tried to check/configure STData processing but have no pending metadata.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Run a search on the indexer, something like this:

index=_internal host=yourforwarderhost modsec_audit.log
0 Karma

thierryit
Path Finder

Can you be more precise ? What entry ? Where ?
Thx for your help even during the week end 🙂

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

No entries in the _internal log files from that host?

0 Karma

thierryit
Path Finder

sudo ps auxxx |grep splunk*
root 1247 0.8 0.7 161860 32520 ? Sl 15:18 0:06 splunkd -p 8089 start
root 1251 0.0 0.0 49116 2884 ? Ss 15:18 0:00 [splunkd pid=1247] splunkd -p 8089 start [process-runner]

-rw-r--r-- 1 root adm 8528077 Mar 29 14:52 modsec_audit.log

Thx

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Okay, is that file readable by the user running the forwarder?

0 Karma

thierryit
Path Finder

[monitor:///opt/splunkforwarder/etc/splunk.version]
_TCP_ROUTING = *
_rcvbuf = 1572864
host = server_hostname
index = _internal
sourcetype = splunk_version
[monitor:///opt/splunkforwarder/var/log/splunk]
_rcvbuf = 1572864
host = server_hostname
index = _internal
[monitor:///opt/splunkforwarder/var/log/splunk/splunkd.log]
_TCP_ROUTING = *
_rcvbuf = 1572864
host = server_hostname
index = _internal
[monitor:///var/log/apache2/modsec_audit.log]
_rcvbuf = 1572864
crcSalt =
disabled = false
host = toto.domain.org
index = main
sourcetype = modsec_audit

With or without crcSalt, same pb.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

That's utterly unreadable, but it seems to me as if it only lists Splunk's own internal log files - so it's not even trying to read your log.

Run this from the CLI of the forwarder:

/opt/splunkforwarder/bin/splunk cmd btool inputs list monitor
0 Karma

thierryit
Path Finder

file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished reading /opt/splunkforwarder/var/log/splunk/licenseusage.log
file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished reading /opt/splunkforwarder/var/log/splunk/metrics.log file position 1144937 file size 1144937 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100.00 type finished
reading /opt/splunkforwarder/var/log/splunk/scheduler.log
file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished

0 Karma

thierryit
Path Finder

A part of the answer, seems to be too big:
app canlist 1 canwrite 1 modifiable 0 owner system perms
read
* write
removable 0 sharing system eai:attributes
optionalFields
requiredFields
wildcardFields
inputs
/opt/splunkforwarder/var/log/splunk/audit.log
file position 50835 file size 50835 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100.00 type finished reading /opt/splunkforwarder/var/log/splunk/btool.log
file position 0 file size 0 parent $SPLUNKHOME/var/log/splunk/splunkd.log percent 100 type finished reading /opt/splunkforwarder/var/log/splunk/licenseaudit.log

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

...in an https-capable client, such as your browser.

0 Karma

thierryit
Path Finder
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

As suggested by the script run on the server, go to https://127.0.0.1:8089/services/admin/inputstatus/TailingProcessor:FileStatus (replace 127.0.0.1 with the forwarder's host).

0 Karma

thierryit
Path Finder

I am not a developer ... I do not understand your answer ... Sorry.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You can call that REST endpoint manually.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...