If you use the Add Inputs wizard to create remote inputs, it will edit the etc/apps/search/local/serverclass.conf I'm not sure why. I am also pretty sure that any changes made through the CLI will also be saved in etc/apps/search/local/serverclass.conf and not etc/system/local/serverclass.conf ... View more

I don't know what you mean by "normal search display." _raw represents the entire event. It is unusual to use _raw in the table command. Instead, include a list of the fields of interest. I can't tell you what fields that would be, since it depends entirely on your data. But for example: ... | table seq_num _time delta_time host status message ... View more

In an eval command, the single quotes are used to enclose field names that have special characters on the right side of the equals sign (=). ... View more

The foreach command iterates over a set of fields - not events. So if I had 6 fields with names that all started with Percent, I could use the foreach command to do the same set of actions for each field Percent*. You don't specify iteration in Splunk like in programming languages. In fact, you generally don't specify iteration at all. I suspect that what you want is quite different. You have a set of events and you want to calculate a few field for the various subsets of events. Try something like this: index="snow_incident" | head 1 | spath path=result{} output=x | fields - _raw | mvexpand x | spath input=x | eval Uptime = if(match(major_inc_temp,u_major_incident),"100","200") | eval major_inc_temp = u_major_incident | table major_inc_temp, Uptime, cmdb_ci The next question is: what do you want to accomplish? Are you trying to find a maximum of something? More info would really be helpful. ... View more

You can't put that in header of a report - unless the report is in a dashboard. ... View more

Yes you can, although this can add quite a bit of overhead to your search, depending on the number of results. And the search "index=*" is also an expensive search, unless you are in a small environment or your timerange is very short. index=* | delta _time p=1 as delta_time | eval delta_time = tostring(delta_time,"duration") | table _time, delta_time, _raw Notice that your delta time will always be negative, because Splunk shows the most recent events first. (Updated to correct typo in line 3 of the search) ... View more

Based on your timerange (6:51:00.000 to 7:51:41.000) - I would not expect to see the bins for 6:50 - 6:55 or 7:50 - 7:55 Weird that they show up. It might be worth opening a support ticket to find out if this is a known problem for your version of Splunk. Workaround: Only search for timeranges that you want to appear in the results. Then you won't need partial=f. In the timerange selector, you can choose "Advanced" and specify the exact timerange that you want. Or you can use earliest and latest as I did in one example. I apologize for not carefully checking the time range of the results in your answer. My first answer was from an environment where I couldn't really see the JPEG. ... View more

Yes, see the previous comment. In my answer, I used earliest and latest to make my search time range explict, since the timerange selector is a separate button on the search screen. You can use partial=t regardless of whether you use earliest or latest or the timerange selector. But the actual effect of the partial=t depends on the bin that the timechart command uses and the overall timerange of the search. ... View more

Yes, the bins in Splunk are fixed: the 5-minute bins break at :05, :10, :15, etc. For a span=1day, the bins break at midnight. You cannot change this "bin" behavior. There is nothing that Splunk can do about missing data in the file. Look carefully at the actual time range of your search - when you run the search, it appears in text above your results. See my last comment - yes, there is something weird about your results based on the timerange that you have highlighted... ... View more

The partial option in the timechart is not based on whether the data is "partial" - rather it is based on whether the time period for a result is partial. For example - a search of "the last 24 hours" actually searches the range "earliest=-24h@h latest=now". If you pipe the search results into "| timechart span=1h" - the results will automatically be "binned" into 1-hour buckets. The earliest bucket will cover an entire hour, but the latest will not (unless you run the search exactly at the beginning of the hour!) There is no way for Splunk to know whether the data that was collected between 01:00 and 02:00 is complete or not - perhaps nothing happened during that time period. But Splunk can definitely determine if a time span represents a full hour, or a full 5-minutes or whatever you have chosen. Note that there is always a time span for a bin/bucket - if you don't specify one, the timechart command will use a default. And the span=X does not override the partial setting - the two settings work together. If you want to test this, run a search on the internal index like this index=_internal earliest=-4h@h latest=now | timechart span=1h count Run the search a couple of times over a half hour. Note that the last data point is usually lower that then others - a lot lower at 10 minutes past the hour. Finally, run the search with the partial option: index=_internal earliest=-4h@h latest=now | timechart span=1h partial=f count Notice that the last bucket no longer appears. Finally, there is a more efficient way to do this - if you don't want a partial hour at the end of the search, why retrieve the data to begin with? index=_internal earliest=-4h@h latest=@h | timechart span=1h count The "latest" stops at the beginning of the hour. This will be more efficient than using the partial option and achieves the same result. ... View more

What was the search? Unless something was actually output to the summary index, it will be empty. You must use commands like sistats, sichart, sitimechart, collect to put data into the summary index. You might want to review the documentation on summary indexing here. ... View more

Where is the actual bottleneck on the heavy forwarder: network, memory, CPU? Forwarding to an indexer cluster should not be slower than forwarding to a single indexer, so I am not surprised that didn't help. There is no "collection interval" on the heavy forwarder; it should be able to "collect" the events asynchronously as they are sent over http/https. My guess is that you may be exceeding the bandwidth of a single event collector. Have you considered using 2 heavy forwarders and having the sender switch between them? If the resources on the heavy forwarder are not being taxed, then perhaps the sender trying to exceed its output bandwidth. ... View more

AFAIK, you cannot send data FROM an indexer that is using a free license. I do not believe that this is supported on any free version of Splunk. ... View more

It would be really helpful to include more information in your question. For example, where are the user names and email addresses? What format do you want to export them in? Does it matter where the file goes, or does it need to show up on your laptop? What are the field names? But I am going to assume that you want to create a CSV file based on the results of a search, and that you want the CSV file to be exported to your laptop. Try this: yoursearchhere | table usernames email | dedup usernames email | sort usernames When the search finishes, if it looks like what you want, click the Export button (just below the timerange picker, it is a small down-arrow). Choose CSV for the format and give the file a name. Click the green Export button and so on. ... View more

You could put a | sort fieldA fieldB etc at the end of your SPL. You will just need to figure out which fields to use for the sort to make the order come out the way you want... ... View more

Can you navigate to http://www.splunkbase.com from your laptop? The message that you are seeing normally means that there is a firewall blocking Splunk's access to the internet. You might be able to find more information about this error in one of Splunk's internal logs. On the Mac, you will probably find the internal logs under /Applications/splunk/var/log/splunk. The file splunkd.log is good for most errors, but for this I might scan most of the logs. BTW, all of the logs in this directory are indexed in Splunk: you can run searches again the _internal index to hunt for errors. If all else fails, you can go directly to Splunkbase via a browser and download the files to your Mac. Then in Splunk, choose the option to install the app from a file and point to the file(s) you downloaded... ... View more

Okay, I fixed the search (I hope) and added stuff to plot a line when the host is down and nothing when the host is up. It is pretty complicated, but this is the only way I could think of to do it: create the table much like you did, then generate a timeline, map the times onto the timeline and finally format the data so it can be displayed with a line chart visualization. Note that I trimmed out a lot of things that you can display in a table, but not on a line chart. So the search part is easier here: search query (check.status = 2 OR check.status=0) | rename check.status as check_status | stats count as occurrences min(timestamp) as minTime by id, client.name, client.address check_status | where occurrences > 2 | eval tag = id . ":" . client.name . ":" . client.address | fields - id, client.name, client.address | eval first_time=if(check_status==2, minTime, null()) | eval recov_time=if(check_status==0, minTime, null()) | stats first(first_time) as first_time first(recov_time) as recov_time by tag | streamstats count as host_number | eval _time = first_time | append [ gentimes start=-7 increment=1h | sort 500 -endtime | transpose 500 header_field=starttime include_empty=true | head 1 ] | foreach 1* [ eval curPeriod = 1 . <<MATCHSTR>> | eval <<FIELD>> = if(first_time <= curPeriod, host_num, null()) | eval <<FIELD>> = if('<<FIELD>>' ==host_num AND curPeriod <= recov_time , '<<FIELD>>' , null()) ] | fields - column first next host_num recov_time first_time occurrences check_status | table tag * | transpose 1000 header_field=tag | rename column as _time | eval _time = strftime(_time,"%x %X") I really feel like there should be an easier way, but darned if I can think of it. Maybe someone else will give a brilliant answer. If you want to do a lot of nice graphics, I recommend that you download and examine the Splunk 6.x Dashboard Examples app; it's great. There is also the Machine Learning Toolkit and even an app for a Gantt chart. (I've never used the Gantt chart, but it might work well for this.) ... View more

The problem is that a period (.) is not really valid in a field name, so it confuses the eval. Sorry, I forgot about that. So the lines should really be the way you did it. ... View more

If you want to plot a moving average, perhaps you can use the trendline command. I also like @somesoni2's answer too. your query to return field | sort _time | trendline sma30(responseTime) AS trend | table _time responseTime trend However this answer makes a different assumption about the data. This assumes that you collect responseTime information every 10 seconds. To get a 5-minute average at any point, you would need to average at the prior 30 responseTimes. This is what trendline does. ... View more

Try this yoursearchhere | chart count as num_source by host source | eval sourceValues = "" | foreach * [eval sourceValues = if(isnumeric('<<FIELD>>'),'<<FIELD>>' . ";" . sourceValues , sourceValues ] | makemv delim=";" sourceValues | where mvcount(mvdedup(sourceValues)) > 1 The challenge is this: there is no way to be sure of the names of the fields (columns) in the report, or how many fields there will be. What if in the future your search returned a different number of sources? I wanted to write a search that would be generally useful. The foreach command examines each column. It ignores nonnumeric columns (like host) and grabs the value for any numeric column (the various counts). It makes a string of the values separated by semicolons. The makemv command turns that string into a multi-valued field. The where command dedups the values in the multi-valued field - if they are all the same, then the mvcount should be one. ... View more

Also, why are you using single quotes in your search? ... View more

Perhaps this will be what you want mysearch .... | stats count(eval(decision=="ACCEPT")) as ACCEPTED count(eval(decision=="REJECT")) as REJECTED dc(requestID) as BATCH_RECORD_CNT by alp_merchantid alp_batchid alp_batch_start_time alp_batch_end_time |rename alp_merchantid as MERCHANTID, alp_batchid as BATCHID, olp_batch_amount as BATCH_AMOUNT, alp_batch_start_time as START_TIME, alp_batch_end_time as END_TIME ... View more

Why are you building a transaction? I can't tell if you are using it or not. Are you sure that ACCEPT and REJECT are capitalized in the data, and that the decision field actually exists? ... View more