Getting Data In

Splunk 6 - IIS nullQueue

rtafoya
Explorer

I'm trying to route certain IIS logs to the nullQueue but it doesn't seem to be working.
the IIS log entry looks like this:

2014-02-19 20:31:06 W3SVC1 Server1 10.10.10.1 GET /login.aspx - 80 - 10.10.1.1 HTTP/1.1 HTTP_monitor/10.0 - - websitelogin.com 200 0 0 5212 87 15

On my Indexer, I have setup /etc/system/local/props.conf like this:

[iis*]
TRANSFORMS-set = dropHTTPmonitor

Then I have /etc/system/local/transforms.conf like this:

[dropHTTPmonitor]
REGEX = (10\.10\.1\.1).*(HTTP_monitor\/10\.0)
DEST_KEY = queue
FORMAT = nullQueue

Restarted splunkd and splunk is still not dropping the events as I think it should. I tried simplifying the regex down to just (HTTP_monitor) and that didn't seem to work either.

Any ideas? (Splunk Enterprise 6.0.1 and universial forwarder 6.0.1-189883 (x64))

Tags (3)
0 Karma

cdupuis123
Path Finder

rtafoya you ever figure this out? I'm have the same type issue trying to nullQueue some load balancer noise out of the indexer. thanks!

0 Karma

dmaislin_splunk
Splunk Employee
Splunk Employee

Just know that it will only drop newly added events. The old events with that log line will still be there unless you add the role of Can_Delete and run a search to

* HTTP_monitor | delete
0 Karma

dmaislin_splunk
Splunk Employee
Splunk Employee

Does my suggestion above work?

0 Karma

rtafoya
Explorer

Thanks. Yes, I am aware that the already indexed events will not be dropped.

0 Karma

dmaislin_splunk
Splunk Employee
Splunk Employee

Just this?

[dropHTTPmonitor]
REGEX = HTTP_monitor
DEST_KEY = queue
FORMAT = nullQueue
0 Karma

rtafoya
Explorer

Unfortunately, I had to resort to setting sourcetype on the forwarder to "iisw3c" then setup props.conf and transforms.conf like I did IIS in splunk 5. I really wish i could get the nullQueue transform working with the Splunk 6 "INDEXED_EXTRACTIONS".

0 Karma

dmaislin_splunk
Splunk Employee
Splunk Employee

You have complete control over any given sourcetype so to make it easier I always start with not using IIS since that is one of the default source types for Splunk. Further, once you have control over your sourcetype then you can do any preferred field extractions the way you want and your nullqueue will be fine.

0 Karma

rtafoya
Explorer

How would I go about changing this in Splunk 6? the Default props.conf has the INDEXED_EXTRACTIONS setting. Do I just place a [iis] stanza in my local props.conf and set everything as i did with Splunk 5x iis extractions?

0 Karma

dmaislin_splunk
Splunk Employee
Splunk Employee

Oh, did not know you were getting your header info that way. Try the 5.x method using either DELIMS or just plain old EXTRACT or a REPORT with props and transforms and see if that works.

0 Karma

rtafoya
Explorer

I tried using the regex you suggested above, but that did not work.
Maybe this has to do with the new INDEXED_EXTRACTIONS settings for iis in Splunk 6.0?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...