Hi everyone,
below are several questions and each of them is very important for us. Hope for your help.
As written in docs these two limits have maximum values that shoudn't be exeeded:
[searchresults]
maxresultrows = <integer>
* Configures the maximum number of events are generated by search commands which
* This limit should not exceed **50000**. Setting this limit higher than 50000 causes instability.
[subsearch]
* This stanza controls subsearch results.
maxout = <integer>
* Maximum number of results to return from a subsearch.
* This value cannot be greater than or equal to **10500**.
* Defaults to 100.
Is it the same for any hardware configurations or at what hardware capacity its increasing will not harm the system?
At what extent can they be risen?
At the moment our standalone server has the following configuration:
Intel x86-64-bit chip architecture
Standard Linux or Windows 64-bit distribution
2 CPU, 4 core per CPU, 2.5-3Ghz per core
8GB RAM
4x300GB SAS hard disks at 10,000 rpm each in RAID 10
capable of 800 IO operations / second (IOPS)
standard 1Gb Ethernet NIC, optional 2nd NIC for a management network
and local/limits.conf file is set to
[searchresults]
maxresultrows = 300000
[subsearch]
maxout = 1000000
default/limits.conf has 50000 and 10000 respectively.
But when e.g. joining results they are still truncated if there are more than 50000 results..
Why local settings are not accepted?
And I also wonder why is there such low default limit on joining at all?
And another thing:
What specifics in limits configuration should have such heavy companies like eBay?
... View more
stacked100 option is good one in chart visualization. But seems it misses obvious thing - percentage labels in tooltips. In "Custom chart configuration reference" I found no param that can turn it on. The worse I need it in timecharts. Is it possible and how can I add percentage without evaluating it?
... View more
In fact this question is an app or enhancement request. It would be extremely useful to have more chart types, like it is shown here https://github.com/mbostock/d3/wiki/Gallery
They are so beautiful!:)
... View more
@sideview thanks for the answer, I had the same problems as the autor. Do layoutPanels that can be used along with search.html described somewhere?
... View more
Was there a progress in sorting in timechart (specifically in its columns)? I wish I could use it e.g. in a timechart with errors for sorting errors by their occurances in days.
... View more
Hello,
Can someone explain why there is no inbuilt functionality of deleting just some indexed data - from particular source or for particular date etc? Are there still not enough requests for this feature?
I know that it's better to create different indexes, but it's weird to make separate index for each search.
Importing to csv, editing and then reindexing as I've read is correlated with high risks.
Just hiding results from searches by "delete" won't move the data.
Cleaning entire index isn't a decision when lots of searches have already been indexed in it.
So can you please answer why is it so? And is it going to be solved?
Thanks
... View more
I've stumbeled upon my old question and realized that I should change the query as follows:
source="my_source", service_id="my_service_id" tx_status_id=COMPLETED | fillnull value=none offer_uid | where offer_uid=$offer_var$ | eval max_usage=case(offer_uid == "free_winter", 2, offer_uid == "free_summer", 15, offer_uid == "free_spring", 10) | eventstats count as usage by user_uid | stats count(eval(usage = "max_usage")) as used_offer_completely
... View more
@sideview thanks for suggestion. It seems that the key to my problem was as @jonuwz described. I haven't used eventstats command yet, maybe because it isn't still fully clear when it's better to use eventstats instead of stats.
... View more
@jonuwz, you're genius! I realized the problems you've stated in the beginning of your answer but didn't think of such uniting of offer stuff. Thank you very much!
... View more
Hello, we've faced with a problem of results trunkating while using join command. In fact limitations of max results for join are extremely dissapointing, we stuk with them here and there and changing params in limits.conf doesn't help anymore. One of our searches stopped working correctly due to this limits. Now I'm trying to remake a problem search to exclude using of join.
The search might find users that made their first monetary purchase after making a promo purchase. Each user can have multiple promo and monetary purchases. Promo purchsases can be of several promo offers.
How it was done before:
source="my_source"
| where isnotnull(offer_uid)
| stats min(_time) as promo_purchase_time by user_uid offer_uid
| join [search source="my_source" tx_amount>0
| stats min(_time) as monetary_purchase_time by user_uid ]
| where promo_purchase_time < monetary_purchase_time
| stats count as Customers by offer_uid
What I'm trying to make:
source="my_source"
| eval monetary_purchase_time=if(tx_amount>0, _time, a)
| eval offer_name=if(isnotnull(offer_uid), offer_uid, a)
| eval offer_purchase_time=if(isnotnull(offer_uid), _time, a)
| stats min(monetary_purchase_time) as min_monetary_purchase_time, values(offer_purchase_time) as offer_purchase_time by user_uid offer_name _time
I get this:
_time user_uid min_monetary_purchase_time offer_name offer_purchase_time
12/10/12 9:06:33.017 PM 727889 OFFER1 1355159193.017
12/10/12 9:11:08.225 PM 727889 OFFER1 1355159468.225
12/10/12 9:22:36.926 PM 727889 OFFER2 1355160156.926
12/14/12 9:42:17.085 PM 727889 OFFER2 1355506937.085
12/15/12 8:58:13.862 PM 727889 1355590693.862
I want to get plain table for each user_uid and his promo offers where can be made a comparision of his first monetary and promo purchase times.
When I change last stats command to this one:
| stats min(monetary_purchase_time) as min_monetary_purchase_time, min(offer_purchase_time) as min_offer_purchase_time by user_uid offer_name
..I get this:
user_uid min_monetary_purchase_time offer_name min_offer_purchase_time
727889 OFFER1 1355159193.017
727889 OFFER2 1355160156.926
But I want to get this:
user_uid min_monetary_purchase_time offer_name min_offer_purchase_time
727889 1355590693.862 OFFER1 1355159193.017
727889 1355590693.862 OFFER2 1355160156.926
with which I can then use a desired command:
| stats count(eval(min_monetary_purchase_time > offer_purchase_time)) as Customers by user_uid offer_uid
How can I fill empty fields of parameter min_monetary_purchase_time for each user with the value of first monetary purchase time?
I've tried variants with chart, downfill, but got incorrect values.
... View more
It's better to say "Thank you very much kallu!" late than never:) I've implemented your suggestion and it have saved lots of my time and nerves.
... View more
As @bmacias84 suggested
helped changing this limit:
[searchresults]
maxresultsrows =
By default we had 100000 and turned to 400000. Whether it's coincidence or not but we experience downgrading of system performance though.
... View more
@bmacias84, thanks, I've checked default and my user's limits with btool and found no differences. And what limits can conflict in my test case? If it can help I can send these limits lists, just need an email.
Also I run the same inputlookup commands with csv file from admin user - the same wrong sorting.
We're using Splunk 4.3.4 build 136012. This testing csv-file is 3Mb in size.
... View more
@sideview I'm facing with a similar problem maybe you can take a look and give advice? http://splunk-base.splunk.com/answers/62296/wrong-sorting
... View more
Hi everyone,
We met a problem with sorting data in a table. We sort users id-s, let's say there's 350000 of them, its format is 1 - 350000. The results are in a table wraped in paginator.
The problem is when trying to sort by user_id by default means of splunk web-interface (triangles near the param name), its last value in descending order is 300000 and then goes 300001, 300002, etc, instead of 350000, 349999, etc.
To check it by yourself, for example create csv file(test.csv) with one column of numbers 1-350000 (e.g. with a header "P"), add this file in lookups, and run the command:
| inputlookup test.csv | table P | sort 0 num(P) P
or just
| inputlookup test.csv | table P
and click triangles
P.S. I didn't manage to find any proper limits that can affect in limits.conf.
... View more
@Ayn @bmacias84 You're right, _time is a default metadata, but in our case this is data from indexed .csv file so _time value for all entries has the same value of its indexing time.
@bmacias84 sorry, I didn't catch what did you expect 'as(time)' should make. In fact 'chart sum(visitors) as visitors over ctime' is the way I'm doing the chart, but it's still unclear why timechart works in this case occasionaly.
... View more
As a timestamp I meen date&time in epoch time format like 123421341342.
Typically time information is presented in our raw data but not in this case.
... View more
Hi everyone!
I made a table that shows web sources from where visitors come to our service.
By clicking any row timechart of visitors for the selected source opens. But it opens not for each source.
The data that is used for these tables and charts doesn't have _time parameter but has year, month, day values. By concatenating these values and converting to timestamp I got _time and built timechart.
Here is working example with source "yandex":
index=visitors source=web | where source_from="yandex" | strcat year "." month "." day date | convert timeformat="%Y.%m.%d" mktime(date) as _time | timechart sum(visitors) as visitors
Not working example with source "google" though it has 5 times more occurances in our statistics then "yandex":
index=visitors source=web | where source_from="google" | strcat year "." month "." day date | convert timeformat="%Y.%m.%d" mktime(date) as _time | timechart sum(visitors) as visitors
The data seems to be similar for these sources, has no gaps in days.
I found that in the second variant splunk can't write result into _time. Trying to use eval _time=.. or strftime didn't helped. How can I write to _time to use timechart?
... View more
Yes, I am using this script. And it's exactly about running and writing to summary index your searches, like they were launched in the past with periodicity set in the (cron)schedule.
... View more
How can one make a cron schedule in Splunk run just in specific year? This can be easily done in 8-digit type of cron schedules but splunk uses just 5 digits.
The case is that I need to backfill summary index only for several months of current year, but our statistics has these months in 2012 and 2011 years so setting schedule like 0 0 * 7-9 * will index data in both years that will be redundant and also will take twice more time and space.
p.s. I can't set earliest time in search or limit time in other way as we need the search to count for all the time.
... View more
@Ayn Yes, we have 8 menus in navigation bar, each menu has about 8 links to the different views, and each view has nearly 20 metrics. Users that do not remember what exact metrics on every view, have to open each view to find what he needs. Labels for the views are rather descriptive but it's still not enough for a user's understanding what's inside exactly.
... View more
Can one make contents of all views that are used in application? It really makes sence to have such information on the main page when you have dozens of metrics at each view. I see it like autocollected links to each panel on a view. Any ideas?
... View more