Yes, the splunk continue to index upto 500MB per day even after expiry.. But answer for the message you is described in this Q & A http://answers.splunk.com/answers/44552/indexing-congestion-consistenly-happening.html ... View more

Did you check splunkd.log in $SPLUNK_HOME/var/log/splunk directory? please splunkd.log events here at the time of crash happened. If you are using linux, please issue the shell command 'dmesg' and see output if you find the information (put the splunk related info here) Did you check the CPU and Memory usage of the system? whats your system configuration? ... View more

whats your splunk topology and source paths? ... View more

when u type index=_internal, you should the hostname of uni. forwarder in host field. Looks like your forwarder communication with indexer not working. Did you enable 9997 port on indexer? Are there any firewall between indexer and forwarder? Configure the receiving port on Indexer (inputs.conf for receiving data on port say 9997) read details at http://docs.splunk.com/Documentation/Splunk/6.2.1/Forwarding/Enableareceiver ... View more

Hi Chadman Since you mentioned you are new to Splunk and as you are using Universal forwarder to send the data to indexer, read on http://docs.splunk.com/Documentation/Splunk/6.2.1/Forwarding/Setupforwardingandreceiving follow the below steps to send data from forwarder to indexer after installing forwarder and indexer. 1. Configure the receiving port on Indexer (inputs.conf for receiving data on port say 9997) read details at http://docs.splunk.com/Documentation/Splunk/6.2.1/Forwarding/Enableareceiver Configure the monitoring inputs on Uni. Forwarder (inputs.conf for monitoring the file) User conf example given above by lguinn. Also read more inputs config details & many examples at end of http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf Configure Uni. Forwarders to send the logs to Indexer (outputs.conf) User conf example given above by lguinn. Also read more outputs config details and many examples at end of http://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf ... View more

There is known issue and the defect id is (which is still exists in 6.2) SPL-82164 Migrating invalid data models from 6.0 to 6.1 fails. Also, pls confirm, are your data model were working before upgrade? from which version of Splunk you upgraded? whats the directory path your data model config placed? what is OS? ... View more

You can't pass tokens to saved searches. You have 2 options 1. make the search into inline search - OR - 2. have a postprocess search and filter the result (returned from saved search) in the process process search ..like "|search host=$host$ ... View more

Good. The event breaking is the root cause. . ... View more

How many lines of log events you see in one splunk event? ... View more

Can you please elaborate with example? would like to more on what is your search query looks like, the result u see and the result u expect.. ... View more

are you able to see forwarder's internal log events when issue index=_internal from search head? How are you ensuring forwarder is able to send the data to Indexer? Does the forwarder splunkd user has the read permission to the directory? Did you enable the receiving port on indexer e.g 9997 ?? ... View more

That's because adhoc searches are having higher priority over scheduled searches. How many CPU core in the search head and indexers in your configuration? You may have increase number of search heads or CPU core since it looks high number of searches are running concurrently. ... View more

Here you go... i tried the below command, the field newval will be 25 | eval Teste="10,15,60" | makemv delim="," Teste | eval val1=mvindex(Teste,0) | eval val2=mvindex(Teste,1) | eval newval=val1+val2 you can make changes as per your index.. Note: to get the last value (60 in the above example) you can use mvindex(Teste, -1) let me know if you need any help ... View more

You mentioned that you have used oneshot option while adding the input file. Hence, file is not monitored for the further updates. For continuous monitor the file: ./splunk add monitor [ -index < indexname> ] [ -sourcetype < name of source type> ] For adding file one time only : ./splunk add oneshot [ -index < indexname> ] [ -sourcetype < name of source type> ] Note : index name and sourcetype are optional. If you want to re-index the file, then you have 2 options 1. crcSalt = in the inputs.conf file 2. clear fish bucket ... View more

From cluster master you can push configuration to cluster peers only, not to the forwarders. If you want to configure the forwarder to send the data to particular index on cluster-peers, use one of the following methods a) use deployment server (app with inputs.conf file) OR b) create inputs.conf file in the forwarder to read the data from souce and restart OR c) use the following command to add the input files For continuous monitor the file: ./splunk add monitor -index [ -sourcetype ] For adding file one time only : ./splunk add oneshot -index [ -sourcetype ] If you want to send the data to cluster-peers (the index is created on cluster) from any forwarder, you no need to create index in the heavy forwarder since the data is finally indexed in cluster peers. Incase if you want to index the data in heavy forwarder too in addition to cluster, then you have to create the index in heavy forwarder manually or use deployment server or create from CLI command or manual put in indexes.conf file let me know if you need any more clarification. ... View more

Finally, I could achieve the desired result by scaling the spunk forwarders vertically i.e configured the multiple splunk forwarders on the same system with different management port & 20 parallel tcp connections to each JMS queue and able to read in JMS data in real time (about 0-2 sec latency). ... View more

Some problem with editor when saving the rex.. I saved the rex for you at http://regex101.com/r/xI6tN4/1 http://regex101.com/r/pR4kY5/1 http://regex101.com/r/pR4kY5/2 You can use Splunk field extraction utility... ... View more