When I search with stats first(myfield) last(myfield) They return the opposite !!!! example : 10/10/2010 myfield=A 12/12/2012 myfield=B | stats first(myfield) last(myfield) returns first=B, last=A ... View more

I want to monitor configuration files, to keep track of the config everyday. I don't care that the file changed or not, I just want a copy with the daily version. How to setup this type of inputs ? ... View more

I upgraded from 4.2.* to 4.3 in windows, Since the service splunkweb fail to start, and here is the error : The instance's SvcRun() method failed File "C:\Program Files\Splunk\Python-2.7\Lib\site-packages\win32\win32serviceutil.py", line 785, in SvcRun self.SvcDoRun() File "C:\Program Files\Splunk\bin\SplunkWebService.py", line 37, in SvcDoRun subprocess.call(gMakeCertsCmd) File "C:\Program Files\Splunk\Python-2.7\Lib\subprocess.py", line 493, in call return Popen(*popenargs, **kwargs).wait() File "C:\Program Files\Splunk\Python-2.7\Lib\subprocess.py", line 679, in init errread, errwrite) File "C:\Program Files\Splunk\Python-2.7\Lib\subprocess.py", line 893, in execute_child startupinfo) : [Error 193] File "C:\Program Files\Splunk\Python-2.7\Lib\site-packages\win32\win32serviceutil.py", line 785, in SvcRun self.SvcDoRun() File "C:\Program Files\Splunk\bin\SplunkWebService.py", line 37, in SvcDoRun subprocess.call(gMakeCertsCmd) File "C:\Program Files\Splunk\Python-2.7\Lib\subprocess.py", line 493, in call return Popen(*popenargs, **kwargs).wait() File "C:\Program Files\Splunk\Python-2.7\Lib\subprocess.py", line 679, in __init _ errread, errwrite) File "C:\Program Files\Splunk\Python-2.7\Lib\subprocess.py", line 893, in _execute_child startupinfo) is not a valid Win32 application ... View more

Hi I have a license pool for X Gb per day, and I blow it every almost every single day. How to selectively reduce my indexing volume ? ... View more

I am configuring apps to deploy. Where is the best location for the configuration files, in myapp/local/ or myapp/default ? ... View more

I want to format nicely the fields or events at search time. by example : US phone : 11122223333 to (111) 222-3333 ipv4 : 1-2-3-4 to 1.2.3.4 credit card : 43431111222233334444 to 4343-1111-2222-3333-4444 ... View more

Hi I am maintaining my splunk config with chef ( or puppet). But after pushing my config, and restarting splunk, some files are modified (server.conf) This is causing a new deployment to occur in loop. Is it normal, how to avoid that ? ... View more

I saw that 4.2.4 is only supported on Mac OS 10.5 and 10.6. When will Lion be supported ? Will Splunk run in full 64bit ? ... View more

I'm getting the error message above while trying to login the interface. In IE, the tab also reads Login - Splunk UNKNOWN_VERSION(000) even though I have a valid license installed. This is my indexer/deployment-server/search-head/coffe-maker. ... View more

For my input [monitor::///home/mataharry/invasionplans/] index=statesecret whitelist=*.secret recurse=true In the manager > data inputs > files & directories I can see my input but the column "number of files" is 235 way larger than my actual files find /home/mataharry/invasionplans/ | grep ".secret" | wc -l returns only 10 Does it means that the columns if the total number of files in the location, not the number of files indexed ... View more

First I saw this banner all the time "received event for unconfigured/disabled index=XXXX " for the indexes _internal and also for _audit. I found out that they were disabled (manager > indexes and $SPLUNK_HOME/etc/system/local/indexes.conf). But when I enable them and restart, they are still disabled by splunk. Of course because _internal is disabled I had to look in the log files, here is the errors in splunkd.log 09-20-2011 23:06:05.513 +0000 ERROR IndexProcessor - One or more indexes could not be initialized and were automatically disabled, please see splunkd.log for more details 09-20-2011 23:06:05.512 +0000 ERROR IndexProcessor - caught exception for index=_internal during initialzation: 'Splunk has detected that a directory has been manually copied into its database, causing id conflicts [/opt/splunk/var/lib/splunk/_internaldb/db/hot_v1_1, /opt/splunk/var/lib/splunk/_internaldb/db/db_1307563383_1307490866_1].'.Disabling the index, please fix-up and run splunk enable index I checked and there are several buckets with the same id. What do I do now ? ... View more

Hi I am trying to have splunk monitoring a log file. But splunk indexed it once, and since is skipping it every time. My log file is recreated every day, then is filled during the day. The first lines are always the same. -> the crc md5 calculation on the first 256 characters is not enough to differentiate them. With the same name filename and location. -> using crcSalct= doesn't solve the problem. my file is /app/logs/superduper.log the header is like following with more than 256 characters before the actual timestamps and events ################## blah blah blah ################## ################## blah blah blah ################## ################## blah blah blah ################## ################## blah blah blah ################## .... my inputs is : [monitor:///app/logs/] crcSalt=< SOURCE > #remove the spaces they are to display on the webapge ... View more

My splunk instance is an indexer and deployment server on windows 2008 I have 2 problems : the WEB UI is sometimes very slow to load (splunkd is not responding fast to splunkweb) the deployment-clients are not correctly deployed on all clients (depends of the network where they are) I see in $SPLUNK_HOME/var/log/splunk/splunkd.log lots of those errors 06-14-2011 06:48:51.298 -0400 ERROR TcpInputProc - Error encountered for connection from src=10.131.83.10:3837. Winsock error 10053 I checked in MSDN, this error is a timeout of the socket : WSAECONNABORTED 10053 Software caused connection abort. An established connection was aborted by the software in your host computer, possibly due to a data transmission time-out or protocol error. ... View more

I want to move some events from an indexer to another, for a particular period of time. I saw that there are some importtool and exportool commands in $SPLUNK_HOME/bin how to use them ? ... View more

I want to check every hour iIf my forwarders are sending data constantly to my indexer, to setup an alert. I am using metrics.log to measure the thruput traffic but it returns only the last hosts sending data, the one not sending data are not even mentioned. How to compare that to the complete list of my hosts ? Can I use metadata, or summary searches, lookups ? ... View more

I have a search-head and several search-peer, I see sometimes this warning in the splunkd.log. DistributedBundleReplicationManager - bundle replication to 2 peer(s) took too long (12943ms), bundle file size=296150KB My lookup file is > 100 Mb and it seems that it's not replicated properly on all the search-peer. How to do ? ... View more

I am trying to make a search parameters which can group the different parameters in a single column and display as multiple columns, for example, for server1, there can be lot of incidents with different severity values, for example Server Name | Severity Server1 | 1 Server1 | 2 Server1 | 3 This has to be be represented in this format Server Name | Severity 1 | Severity 2 | Severity 3 Server1 | 10 | 5 | 4 Server 2 | 8 | 9 | 8 ... View more

I have a VM linux indexer/deployment server. with > 500 forwarders and deployment clients The server keeps crashing, or the splunkd process not responding. ... View more

Hi I am collecting events of states of devices, a device is supposed to be go from the state "false"=beginning of transaction to "true"=end of transactions. I would like to retrieve the duration of my transactions, to track the long/unfinished ones. My problem is that my logs doesn't always contains the end of transactions event. Here is an example of my logs. I added comments at the end of the events. you can see that the same device/store pair may starts several transactions without finishing them all. Thu Mar 31 10:56:26 PDT 2011 Device=1 Store=1 Status=false detail=transaction 1 OK start Thu Mar 31 11:56:26 PDT 2011 Device=1 Store=1 Status=true detail=transaction 1 OK end Thu Mar 31 12:00:26 PDT 2011 Device=1 Store=1 Status=false detail=transaction 2 broken start Thu Mar 31 12:30:26 PDT 2011 Device=1 Store=1 Status=false detail=transaction 3 OK start Thu Mar 31 12:40:26 PDT 2011 Device=1 Store=1 Status=true detail=transaction 3 OK end Thu Mar 31 13:56:26 PDT 2011 Device=1 Store=1 Status=false detail=transaction 4 broken start Thu Mar 31 14:56:26 PDT 2011 Device=1 Store=1 Status=false detail=transaction 5 broken start Thu Mar 31 15:56:26 PDT 2011 Device=1 Store=1 Status=false detail=transaction 6 OK start Thu Mar 31 16:56:26 PDT 2011 Device=1 Store=1 Status=true detail=transaction 6 OK end Thu Mar 31 17:04:26 PDT 2011 Device=1 Store=1 Status=false detail=transaction 7 broken start Thu Mar 31 17:10:26 PDT 2011 Device=1 Store=1 Status=false detail=transaction 8 start - unfinished yet I would like to retrieve : all the correct transactions with duration every time a new transaction restart, I want to consider the previous one as finished I also would like to have the duration of the last unfinished transactions till now. ... View more

I have an indexer on linux on a physical server, with 100+ forwarders, and local files indexing, it's also my deployment server, and my search-head, with all the users from my AD. I beefed-up the disk size, multiple cores, RAM... But I still see some complains at startup. How can I tune it ? here is my starting log in splunkd.log 03-03-2011 21:50:09.027 INFO ulimit - Limit: virtual address space size: unlimited 03-03-2011 21:50:09.027 INFO ulimit - Limit: data segment size: 1879048192 bytes [hard maximum: unlimited] 03-03-2011 21:50:09.027 INFO ulimit - Limit: resident memory size: 2147482624 bytes [hard maximum: unlimited] 03-03-2011 21:50:09.027 INFO ulimit - Limit: stack size: 33554432 bytes [hard maximum: 2147483646 bytes] 03-03-2011 21:50:09.027 INFO ulimit - Limit: core file size: 1073741312 bytes [hard maximum: unlimited] 03-03-2011 21:50:09.027 INFO ulimit - Limit: data file size: 2147483646 bytes 03-03-2011 21:50:09.027 ERROR ulimit - Splunk may not work due to low file size limit 03-03-2011 21:50:09.027 INFO ulimit - Limit: open files: 1024 03-03-2011 21:50:09.027 INFO ulimit - Limit: cpu time: unlimited 03-03-2011 21:50:09.029 INFO loader - Splunkd starting (build 95063). ... View more

I installed 4.2 splunk, and made it a forwarder (not lightweight or universal forwarder) Because I want to do some filtering on it before sending to the indexer. But I got license errors. I looked for the $SPLUNK_HOME/splunk-forwarder.license as I used to do on 4.1. But i only found a trial license !!! where is my forwarder license ? ... View more

I want to change the source filename for my data to remove the timestamp. from mypath\to\my\folder\userentrypoint17_20110309T143708_170500.log to mypath\to\my\folder\userentrypoint17.log the timestamp in the filename is not used, because the complete timestamp is precsent in each event. ... View more