I've seen a number of posts about this with varied responses. Here's what I'm trying to do: We have some web access logs that records load balancer hits every second. Those logs get sent (via Universal Forwarder) to the Splunk Indexer. I'd like to filter out those load balancer hits before they're indexed. Otherwise it's a waste of license. I have been having a hard time getting this to work. I've read other posts on here with similar questions, but none of the solutions are working for me. Is it true that this cannot be done with the Universal Forwarder? What exactly are the consequences of running the heavy forwarder? Thanks! ... View more

I'm trying to get a blacklist in my inputs.conf to work correctly, but it's just not happening. It seems so simple... I am monitoring /logs. Here's what I need: I need all files with a .bz2 and .gz suffix blacklisted I need all subfolders that start with esx-abcd- (and all files and folders under it) to be blacklisted. For example, /logs/esx-abcd-01/2011/09/14/syslog etc.... I tried this, and variants of it, but it didn't work: blacklist = (^esx-abcd-|\.bz2$|\.gz$) It seems like no matter what I try, it keeps indexing the directories under esx-abdc-*. I've even tried hard-coding in specific directory names (for testing purposes), yet it still goes ahead and indexes everything under it. Something obviously isn't right. Can someone help? Thanks! ... View more

I have Splunk crawling a /logs directory, which is where it receives most of its data. (/logs is populated via syslog-ng). In inputs.conf, I set host_ segment = 2 so that the hostname will be set to the second segment in the path. This has been working fine most of the time. Here is the inputs.conf stanza: [monitor:///logs] disabled = false sourcetype = syslog host_segment = 2 blacklist = \.(bz2|gz)$ But suddenly I'm noticing some strange hostnames on my indexer... "list_ primary_ nodes", "add_ aam_ node", "find_ active_ primary", "shut_ down_ vmap_ proce" etc... I noticed that they're all coming from a series of new servers that have been sending logs to Splunk: ESXi hosts. The log path is correct: /logs/HOSTNAME/2011/09/14/user Here is an event: 09/14/11 19:31:56 [shut_ down_ vmap_ proce] attempt to stop VMap_ HOSTNAME failed. So why is it not setting the hostname to HOSTNAME? Why is it setting it to this other hostname that it's capturing from this, albeit unusual-looking, syslog? Thanks! ... View more

I tried out the EQALIS Splunk for Network Operations app. Unfortunately, I don't think I can use it, but maybe someone can help. The problem is that incoming events from Cisco devices must have a sourcetype of "cisco_syslog". Well, all my syslog events come in to my Splunk server via syslog-ng and are put into a /logs directory which is crawled by monitor. Everything that is crawled in the syslog-ng /logs directory is given a sourcetype of "syslog". That includes servers, switches, etc... I have to force the sourcetype to be "syslog" in inputs.conf otherwise Splunk will give split the events into sourcetypes I don't want such "auth-too_small", "kern", etc... Or maybe I'm taking the wrong approach. I'd like to take advantage of the EQALIS app, but am not sure how to get the cisco_syslog sourcetype assigned to the applicable events when they're crawled by monitor. Does anyone have any suggestions? Thanks! ... View more

I have a requirement to have data older than one year removed from Splunk. By "older than year", I mean the event has to be older than one year, not necessarily when it was indexed. In my indexes.conf file, I set: [main] frozenTimePeriodInSecs = 31536000 31536000 seconds should be one year. And yet it's showing the earliest events (185,000 of them) as July 18, 2010 (today is August 15, 2011). It was my expectation that the earliest event would be August 15, 2010. Tomorrow's earliest event would be August 16, 2010, etc... How can I instruct Splunk to automatically purge events older than one year? Thanks! ... View more

I'm setting up a fresh new Splunk server and am re-indexing my data from scratch. Syslog data is being sent to my syslog-ng server/Splunk indexer via UDP 514. Rather than being sent directly to Splunk, I have the syslog data distributed to a file system/directory structure that I instruct Splunk to "monitor". (i.e. /logs/hostname/year/month/year/day/logfile) My expectation was that the host name would be set to the hostname set in the path of the file directory structure, and that everything coming in from the syslog would be set to sourcetype "syslog". Accordingly, here is my inputs.conf: [monitor:///logs] disabled=false sourcetype=syslog host_segment=2 blacklist=\.(bz2|gz)$ And 95% of my events are indexed correctly. Unfortunately, a few of my events aren't setting the host name correctly; it's using the non-FQDN as indicated in the syslog event itself for some older events (legacy reasons) rather than the name specified in the /logs/hostname segment. Also, most events are set to "syslog" as instructed in inputs.conf except for dhcp events which are being set to sourcetype "dhcpd". While technically accurate, it's not what I instructed Splunk to do in inputs.conf. I would have expected everything coming in from the /logs monitor to be set to sourcetype="syslog". Is there a reason Splunk is over-riding my settings? Thanks! ... View more

I noticed support for AIX 6.1 as of Splunk 4.2. Great! Then I noticed support for AIX 6.1 taken away with Splunk 4.2.1, or at least it is no longer listed. AIX 5.2 and 5.3 is still listed, but AIX 6.1 is not. Is this an oversight, or is there a problem with AIX 6.1 on Splunk 4.2.1? I didn't see anything in the release notes... Thanks! ... View more

One of our Splunk environments receives data from a FIFO pipe. That is, syslog-ng takes incoming syslog data and sends it straight to Splunk via the pipe. It also takes incoming syslog data and writes it to the file system in the format of /logs/hostname/year/month/day/logfile What I find odd is that if I sum up all the space used in the file system for the day, it consistency adds up to about 300-500 MB less than the amount of data that Splunk indexes for the day over the pipe. If Splunk is receiving the same data over the pipe that is being written to the file system, how is it that it's indexing 300-500 MB more data than it's receiving? I hope this question makes sense... We don't have a huge license, so 300-500MB/day can make a difference. I just want to understand where it's going. (On a side note, we're migrating to a new server, during which time we plan to lose the pipe and begin using monitor. Does that sound like a reasonable approach?) Thanks! ... View more