I don't know if this is the issue but line 16: word_cnt += count I think there are 2 different variables. 'cnt' and 'count'. At line 16 'count' is not defined. ... View more

I think the search is losing the field todayTotal when it do the stats command by status. I think the search should do the stats by status, todayTotal which will also keep the todayTotal field in the results so you can calculate the percent field. ...|stats count(status) as todayValue by status,todayTotal | eval percent = (todayValue/todayTotal)*100 ... View more

Sorry. I was trying to link to 'props.conf' and 'transforms.conf' which talks about generating fields for your events. This is the basis of creating the Alerts / Graphs / Knowledge Objects and the other interesting things that make Splunk search so powerful. http://docs.splunk.com/Documentation/Splunk/6.3.1/admin/Propsconf ... View more

I would guess that you are not running as the user that needs to start Splunk. I am not a Windows admin, but you may be able to start splunk from the Services Management Console. This should run as a local admin. If you want to start Splunk from the command line, you can try starting the command line as a local admin: •Click Start. •In the Start Search box, type cmd, and then press CTRL+SHIFT+ENTER. •If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue. ... View more

That is a great question. I don't think there is a one size fits all answer. I think you are on the right path by checking looking at the number of users that will be using the application. You may also want to consider the number of queries per user. Some queries may be more intense than others. I would not consider the search head as "just a web application". There is quite a bit of processing going on after the data is pulled from the indexes. There are things you can do to the indexes to make search time faster before or after indexing. There can be much more processing after the indexing is completed. I would take a look at the docs on link text and link text to see index time extraction and search time extraction. You should also think about all the applications and add-ons you will be using. Personally, I like to dedicate the search head. It makes admin / troubleshooting problems easier if one system is not performing. Many add-on apps can be added to the search head, without effecting the indexer. Additionally, the cost of adding an additional indexer (if needed later) should be minimal compared to the total cost of the Splunk configuration. If you are spending the time / money to distribute your Splunk cluster, you will probably want to dedicate your search head. ... View more

What about running a different command in that directory to test the shell. You should be able to run them directly from the CLI. I use user "splunk" and can run ps.sh: sudo su - splunk .../Splunk_TA_nix/bin/ps.sh splunk cmd .../Splunk_TA_nix/bin/ps.sh Both seem to provide the same output. You could also try running the script with a different shell, and see what that provides. I am not an expert in HP UX but it appears as though the shell provided does not like the "function" syntax. By default this TA uses /bin/sh as a shell. You could try running a different shell, but I know ksh has enough differences to cause some issues: /bin/ksh .../Splunk_TA_nix/bin/ps.sh /bin/csh .../Splunk_TA_nix/bin/ps.sh (These could also be in /usr/bin/*sh) I could be completely wrong here, but that is what it appears to be. I don't have an HP UX system to test. ... View more

Can you check the shell you are running: On mine it is: # ls -l /bin/sh lrwxrwxrwx. 1 root root 4 Apr 24 12:47 /bin/sh -> bash ... View more

They are both working, but your second eval is overwriting the OS value of your first. Change the second to: | eval OS=if(match(User_Agent,"windows"),"Windows",OS) ... View more

I think you can just use the "|" to another stats: index=my table | stats count(Host) by Account, Group | rename count(Host) AS Hosts | stats count(Group) by Account, Hosts | ... ... View more

If you can get to root on the Splunk server, you can use tcpdump or something to see if the packets are at least making it to the server. Most of the time, when someone says they send logs, and no logs are received, this will help. Often there is a firewall or something in the way. Sometimes the IP is being NAT, so it appears to be from a different server. For example, to see if you are receiving packets on port 9997: sudo tcpdump port 9997 Or to see all packets from the server 199.99.1.1 sudo tcpdump host 199.99.1.1 Of course you can combine those options to try to minimize the data. Another option is to use the *-w * to create a binary file and then you can pull that off the server and view / analyze the data with wireshark. ... View more

Here is a small modification to show the last 30 days in a chart. It works for a single pool, but could be modified for multiple pools. Good to see how things are changing, and it is efficient: index=_internal source=*license_usage.log type=RollOverSummary | timechart sum(b) as b max(poolsz) AS poolsz span=1d | eval PercentLicense = round((b/poolsz * 100),1) | fields - b poolsz Here is a small modification to show how much each index is using. It is good to run this over time "yesterday" (or some 24 hour time period) index=_internal source=*license_usage.log idx=* | stats sum(b) as b max(poolsz) as poolsz by pool, idx | eval percent = round(b/poolsz * 100,1) | eval GB=round(b/1024/1024/1024) | eval "License GB"=round(poolsz/1024/1024/1024) | fields - b poolsz | addcoltotals labelfield=idx label=Total GB percent ... View more