I have many indexes in my environment, which all have a maximum size set. However, we would like to instead keep data in an index according to date. For example: we want to keep all data in an index that is less than 30 days old. Is this possible? Thanks, Sean ... View more

The ability for Splunk to start where it left off is a great feature. However, sometimes that feature hurts us. Scenario: Indexer detects an error in an application that is located in Forwarder_123's logs. The Indexer kicks off a bash script that shuts down Splunk on Forwarder_123. (we shut it down because the application's error could exceed our license max in a matter of minutes) To fix the application, we restart the Application Server, so after the App Server restarts, it may be 30 minutes from the initial error was found. If we restart Splunk normally on Forwarder_123, all of the data that we do not want will be sent to the Indexer, which we do not want. So, is there a way to tell the Splunk forwarder to start reading log entries from the last timestamp, instead of from the last location it remembers reading from? I have tried the following command, but it does not appear to be what I want: /opt/splunk/server/splunk/bin/splunk clean eventdata -f The forwarder does NOT index any data. It simply sends the data straight to the Indexer. Thanks, Sean ... View more

The log entry is: 2011-04-20 01:04:12,026 [DEBUG] com.company.ldap.SpringLdapDao.java(?) - **username=ahall** returned no groups, which probably means the user needs to be assigned groups by security 2011-04-19 18:06:49,424 [DEBUG] com.company.ldap.SpringLdapDao.java(?) - **username=frozndrk** returned no groups, which probably means the user needs to be assigned groups by security I would expect to see a list of usernames with the simple values (e.g. ahall, frozndrk), but in reality, what I see for the username field is: ahall returned no groups frozndrk returned no groups In some odd way, it is also grabbing data to the comma. I only want the username, and not the junk after it. Why is this happening? Thanks, Sean ... View more

I have the following query: host=wps03 mc_getLDAPGroupsTimer | table time host username mc_getLDAPGroupsTimer | sort -mc_getLDAPGroupsTimer I am trying to get the date and time to display in the table, so I can see what happened when, but I obviously have the syntax incorrect. Or, maybe this is not possible for some logical reason. How can I display the date and time of an event in a table? Thanks, Sean ... View more

We have created a bunch of different indexes for all of our different systems. At some point, these systems will freak out, and produce 10-15G of data in a matter of minutes. We are still working to fix the freak out issue. Is disabling that index when it hits 2G in a day the best way to prevent a license violation? The only other option I have thought of is to send that forwarder's data to the null-queue. Is that a better option? This is desirable for a couple of reasons: 1) avoid filling up the splunk indexer drive 2) avoid license violations Thanks, Sean ... View more

I am trying to create a table (and then a report) of all exceptions/errors that occur for a given sourcetype. The primary problem (i suspect) is that I am not doing a count on a given field. The reason for this is that there is nothing common to extract that I can see so far because there are no key-value pairs when it comes to errors/exceptions. My query looks like: eventtype="all_web" (error OR exception) | chart count(events) as eventsBySourceTypeCnt by sourcetype | table sourcetype eventsBySourceTypeCnt I didn't think counting on "events" was going to work, but I had to start somewhere. Some of the data returned would by just the first portion of the query would be: commitCloseConnection - [18 Nov 2010 16:49:16,434] - ERROR [Default : 1617] PolarisDAO.java:190) - A java.lang.NullPointerException occurred - no detail available. [11/18/10 16:49:22:214 CST] 0000237b SystemErr R java.io.FileNotFoundException: /favicon.ico Any ideas what I can do here to count just the events? It would be nice to know how many NullPointerExceptions, Errors, or FileNotFoundExceptions there are per sourcetype, but I don't think I'm to that point yet. Thanks, Sean ... View more

From server1, I have access to the desired UNC path, and this same user is running splunk, so I know access is not an issue. \etc\apps\search\local\inputs.conf [monitor://\\SANCIFS_TDC_NETAPP01A.SAN.MyCompany.Com\CIFS_COGNOS$\Test\Logs] disabled = false host = sancifs_test index = default sourcetype = motio_test I have tried many different permutations of the forward and back slash for my monitor stanza, but nothing has worked so far. What am I doing wrong? Thanks, Sean ... View more