Hello Splunkers, Today I have upgraded my Splunk environment from 6.0.1 to 6.6.1. Every dashboard and Splunk query is working fine except this. Can someone please correct why i am suddenly seeing this error after upgrade. index=ip_lux_metadata s_event="MSG_*" | eval q_precedence=if(s_event=="MSG_RECEIVE" AND like(s_comp_id,replace("*","[*]","%")) AND e_to=="SMC", 0, 1) | eval q_event=if(q_precedence==0, "MSG_SEND", s_event) | eval s_comp_id=if(e_to=="SMC", "SMC", s_comp_id) | search s_comp_id="*" | fillnull value="" e_id, e_ppg_id, e_crl_id, e_action | search e_id="*" e_ppg_id="*" e_crl_id="*" e_action="*" | eval q_proc_id = mvindex(s_proc_id, -1) | sort q_precedence | dedup s_proc_id | fillnull value="%DEFAULT_STATUS%" s_proc_outcome | eval q_time=_time | eval q_status_time=q_time | eval q_agent=coalesce(a_agent, s_comp_id) %START% | join type=outer s_proc_id [search index=ip_lux_metadata s_comp_id="*" s_proc_outcome="*" | eval q_status_time=_time | fields s_proc_id, s_proc_outcome, q_status_time] %END% | search s_proc_outcome="*" | eval q_time=_time | sort -q_time, s_comp_id, e_path | eval q_info=q_time + "," + q_status_time + "," + q_proc_id + "," + s_event + "," + s_proc_outcome | eval q_proc=q_time + "," + q_status_time + "," + q_proc_id + "," + s_proc_outcome | eval q_proc_outcome=q_proc_id + "," + s_proc_outcome | table q_info, q_proc, q_time, s_comp_id, q_event, e_action, q_proc_outcome, e_path, q_agent, e_ppg_id, e_id, e_crl_id | eval q_event=if(q_event == "MSG_RECEIVE", "Received", "Sent") | convert timeformat="%d/%m/%y %H:%M:%S.%3N" ctime(q_time) | rename q_time as Time, s_comp_id as Endpoint, q_event as "Received / sent", q_agent as Agent, e_action as Action, q_proc_outcome as Status, e_path as Path, e_ppg_id as "Propagation id", e_id as "Message id", e_crl_id as "Correlation id", q_info as " ", q_proc as " " ERROR IS Error in 'eval' command: The expression is malformed. The factor is missing. The search job has failed due to an error. You may be able view the job in the Job Inspector. ... View more

Hi, I need urgent assistance on upgrading Search head pooling. Mine is distributed environment(6.0.1) with below details Two indexers(Clustered) Two search heads(SHP) One Cluster master As per the Splunk docuemntation I need to upgrade in below sequence Licence Master ->Search head ->Cluster master ->Indexer For Search head pooling i have below doubt as mentioned in Splunk documents Test apps prior to the upgrade Before you upgrade a distributed environment, confirm that Splunk apps work on the version of Splunk Enterprise that you want to upgrade to. You must test apps if you want to upgrade a distributed environment with a search head pool, because search head pools use shared storage space for apps and configurations. When you upgrade, the migration utility warns of apps that need to be copied to shared storage for pooled search heads when you upgrade them. It does not copy them for you. *You must manually copy updated apps, including apps that ship with Splunk Enterprise (such as the Search app) - to shared storage during the upgrade process*. Failure to do so can cause problems with the user interface after you complete the upgrade. On a reference machine, install the full version of Splunk Enterprise that you currently run. Install the apps on this instance. Access the apps to confirm that they work as you expect. Upgrade the instance. Access the apps again to confirm that they still work. If the apps work as you expect, move them to the appropriate location during the upgrade of your distributed environment: If you use non-pooled search heads, move the apps to $SPLUNK_HOME/etc/apps on each search head during the search head upgrade process. If you use pooled search heads, move the apps to the shared storage location where the pooled search heads expect to find the apps. My Question is 1) I have already apps placed on NAS. How can i copy and paste from Search head again ? Is this makes sense ? PS:- I know Search head pooling is depreciated feature. We will upgrade to Search head clustering later as a different project. ... View more

Hi Guys, I have my searches disabled on Search heads as the default minimum free disk space is 5000MB. Problem is my splunk configuration. I have splunk installed on two different file system. One FS is for Search head pooling(NAS). Where and what changes should be make to avoid this error. 265G 905M 251G 1% **/APPLICATIONS/SPLUNKT** 9.9G 5.0G 5.0G 50% **/APPLICATIONS/SPLUNKT/Global_Storage** --- Search head pooling ... View more

Hello Splunkers.. I need urgent assistance in setting up Splunk ITSI. Our current Infrastructure is a distributed one running on Splunk version 6.0.1. Present Infrastructure where Splunk 6.0.1 is present:- Two indexers - RAM 16 GB, CPU 12 CORES Two search heads(SHP) - RAM 16 GB, CPU 12 CORES One Cluster master - RAM 16 GB, CPU 12 CORES We want to install Splunk ITSI and for that we have ordered completely new VM which will behave as a dedicated Search head for ITSI. Can someone please clarify my doubts:- 1) For 100-200 KPIs the VM I ordered has specs RAM 32 GB, CPU 16 CORES, Disc 500 GB Also i will upgrade present Indexers specs to RAM 32 GB, CPU 16 CORES. 2) Version upgrade. Can we run Splunk ITSI search head on version 7.1.x and what minimum version we need to upgrade for present Indexer, Search heads and CM. 3) We dont t want to load Search heads so that s why we have ordered new VM as dedicated search head. Is it good approach ? Thanks, Ramprakash ... View more

Hello Splunkers, I need to install Splunk forwarder on my AIX machine. Can someone please share step by step procedure? Also, I just ended up Untar the package. But I am not able to see Var directory. Below are the contents: /APPLICATIONS/SPLUNK/currentversion/splunkforwarder> ls -lrt total 168 -r--r--r-- 1 splunk saraluxt 510 Dec 13 2013 README-splunk.txt -r--r--r-- 1 splunk saraluxt 49092 Dec 13 2013 license-eula.txt -r--r--r-- 1 splunk saraluxt 57 Dec 13 2013 copyright.txt -rw-r--r-- 1 splunk saraluxt 0 Dec 13 2013 ftr -rw-r--r-- 1 splunk saraluxt 14624 Dec 13 2013 splunkforwarder-6.0.1-189883-AIX-powerpc-manifest drwxr-xr-x 2 splunk saraluxt 256 Dec 13 2013 include drwxr-xr-x 3 splunk saraluxt 256 Dec 13 2013 share drwxr-xr-x 3 splunk saraluxt 256 Dec 13 2013 openssl drwxr-xr-x 3 splunk saraluxt 4096 Dec 13 2013 lib drwxr-xr-x 10 splunk saraluxt 4096 Dec 13 2013 etc drwxr-xr-x 3 splunk saraluxt 4096 Dec 13 2013 bin ... View more