About sideview

sideview · ‎05-08-2015

Yes. OK. Here is the section of the docs that gives high level links to docs pages about field extractions. http://docs.splunk.com/Splexicon:Fieldextraction What you've found here is a quick and dirty way, basically the "extract fields with search commands" link on that page. What you want to do is create a proper field extraction, either with the Interactive Field Extractor, or by putting a similar regex into the conf files. Both options have links on that page. I assumed you were already familiar with field extractions as they're something you need to understand quite early on. You also might want to go through the tutorial if you haven't already - http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchTutorial/WelcometotheSearchTutorial

sideview · ‎05-08-2015

I think what you want is really simple. It's very common for relatively new Splunk users looking for parallels to sql to dive too deeply into advanced things like the join command. However here, I think what you want is either "some text that I'm looking for" | table somefield or "some text that I'm looking for" | stats count by somefield UPDATE Looking at your sql (and it's been a while), it looks like maybe what you want is to get the matches for the "some text that I'm looking for", then get the values of somefield that appear in those events, and then get all events that have those values of somefield? If so then you want a normal subsearch, ie search syntax that looks like this: ["some text that I'm looking for" | table somefield] docs about subsearches: http://docs.splunk.com/Documentation/Splunk/6.2.3/Search/Aboutsubsearches

sideview · ‎05-08-2015

Can you tell us where your event boundaries are in Splunk? Are they multiline events such that the lines that have the somefield value are in the same event as the "some text I'm [not] looking for lines" ? Or are these all individual single-line events? If these are multiline events, then what you want is really simple. Either "some text that I'm looking for" | table somefield or "some text that I'm looking for" | stats count by somefield . And even if they are single line events then selfjoin and all that isn't what you need at all. sql metaphors only go so far in splunk before they lead you astray.

sideview · ‎05-08-2015

You don't want the join command here. I think you want to use append instead. The join command by it's nature will take rows from the outer search and match them up with rows from the inner search that have the same value for the joined field (here your field called "audit"). Since your inner search ends in stats count as req , it will have no value at all for audit, so that row will be discarded. What you want in this case, if you just want to add the results of your subsearch to the main search results, is append instead of join . index="uk" sourcetype="ukpro" serviceType=1 message="Received * bytes from IP*" | rename time as time1,message as Request | append [search index="uk" sourcetype="ukpro" serviceType=1 message="Deleted m_pReceivingSocket" | rename time as time2, message as Responce | stats count as req] | eval itime=strptime(time1,"%H:%M:%S.%3N") | eval ptime=strptime(time2,"%H:%M:%S.%3N") | eval TimeDiff=ptime -itime | table time2 time1 TimeDiff audit Request Responce | stats avg(TimeDiff) as AVG, max(TimeDiff) as MAX, min(TimeDiff) as Min,count(Responce)

sideview · ‎05-07-2015

Getting closer. OK use sed but do your shenanigans on a separate field called like displayURL. have the real url field listed in the Table module's "hiddenFields" param. Then you can use Table Embedding to display the "displayURL" field, but for all drilldown purposes and everything else use the real url field. Despite being "hidden", it'll be available in drilldown tokens etc.

sideview · ‎05-07-2015

Actually you might try a completely different approach, using Table Embedding to effectively truncate the urls if they're too long. basic idea - You use the search language to make another field, called say "truncatedUrl". If the url is less than N characters, it holds the url value. Otherwise it holds the first N-5 chars plus "...". Then you have <module name="Table"> <param name="hiddenFields">truncatedUrl</param> <module name="HTML" group="row.fields.url"> <param name="html">$truncatedUrl$</param> </module> </module> It's a good table-embedding trick. On a couple apps I've also built simple little "expand/collapse" links so the user can expand it to the un-truncated value if/when they want. That does admittedly require a few lines of JS.

sideview · ‎05-06-2015

I would get out of multivalue-land for this, even if you go back into multivalue fields at the end. For example: sourcetype="syslog-stats" | mvexpand syslog_dest_host | streamstats window=1 sum(syslog_dropped) as Dest_Syslog_Dropped, sum(syslog_processed) as Dest_Syslog_Processed by syslog_dest_host | table _time, host, syslog_dest_host, syslog_dropped, Dest_Syslog_Dropped, syslog_processed, Dest_Syslog_Processed If you need to fold it back up the way it was at the end, then do this: sourcetype="syslog-stats" | streamstats count as rowId | mvexpand syslog_dest_host | streamstats window=1 sum(syslog_dropped) as Dest_Syslog_Dropped, sum(syslog_processed) as Dest_Syslog_Processed by syslog_dest_host | table _time, host, syslog_dest_host, syslog_dropped, Dest_Syslog_Dropped, syslog_processed, Dest_Syslog_Processed | stats values(*) as * by rowId

sideview · ‎05-06-2015

I'm glad you figured it out before I made it back to answer. This is a bit of a hard road that you took. Kudos for sure! As you later figured out for yourself, the Tabs module is a decent alternative here, particularly if you can hit it with the CSS stick a bit. But another one that you might not have thought of, is to use Link modules instead of your HTML modules, and Gate modules. Gate is a little terrifying, but it's very powerful. You could have each Link contain a ValueSetter, and then a Gate. The Gate modules act as little teleporters and in spirit they allow a set of drilldown config to have more than one chain of upstream modules. Specifically, each of the 4 valueSetters would set the same differentiating token to a different value, and each of the 4 Gate's would have <param name="to">downstreamBlock</param> . then all the stuff "downstream" from these four link blocks wouldn't actually be downstream at all technically. Rather you'd have a fifth Gate module with <param name="id">downstreamBlock</param> at the top level of the hierarchy, but in a dashboard panel lower down on the page. There are examples of this sort of config in Sideview Utils - start with the Gate docs and examples. You might find a hidden testcase view as well to rummage around in.

sideview · ‎05-06-2015

URL's are particularly tricky because the are single enormous words, and normal wrapping isn't allowed to break words. Of course there are ways to do this. If you have a good browser spec I think you can just do <module name="Table"> <param name="cssClass">wordBreakingTable</param> and then in your application.css .wordBreakingTable td { word-wrap: break-word; } However depending on your browser support and specific requirements you might need to get a little tweakier than that. Here is some further reading. https://css-tricks.com/snippets/css/prevent-long-urls-from-breaking-out-of-container/

sideview · ‎05-06-2015

Sounds like either Page or error is undefined for a lot of your requests. I assumed you didn't want to count such events. My first stats command there will implicitly discard rows that don't have a value for either error or Page. If you want to include counts for things that don't have error or Page values, then add before the first stats, | fillnull error Page value="NULL"

sideview · ‎05-06-2015

I'm assuming your external file is a Splunk lookup, or that you can make it a Splunk lookup. And that your [a,b,c,d,e,f] field is called "foo", and say there's at least one other field in that lookup that is present on all rows, that we'll call "bar". And also let's assume that in the raw data your multivalue field is called mvField. <your searchterms> | fields mvField (and whatever few other fields you will ultimately need) | streamstats count as rowId | mvexpand mvField | lookup foo as mvField OUTPUT bar | where isnull(bar) | stats values(*) as * by rowId The idea is to use streamstats to paint a unique id on each row, mvexpand out your multivalue field, wash it through a lookup, then discard the rows that actually matched the lookup. Then finally we use stats values(*) to fold all the expanded rows back up, and end up with mvField being a multivalue field again.

sideview · ‎05-06-2015

New answer! OK. You need rows of error, columns of pages, and each cell to say "X/Y", for example "9/2131" where it says that 9 users experienced that error on that particular page, and 2131 total users visited that page. Here you go: | stats values(ipAddress) as addresses dc(ipAddress) as impact by error Page | streamstats dc(addresses) as totalUsersImpacted by Page | fields - addresses | eval impact=impact + "/" + totalUsersImpacted | xyseries error Page impact You need to do dc once in the regular stats command and then a second time with streamstats ... by error. This streamstats clause will do the same dc math but only per Error. To give the streamstats the raw materials we need, we need to pass along the values(ipAddress) although after streamstats we can safely discard them.

sideview · ‎05-06-2015

Ah, easy enough. | stats values(ipAddress) as addresses dc(ipAddress) as impact by error Page | streamstats dc(addresses) as totalUsersImpacted by Page | fields - addresses | eval impact=impact + "/" + totalUsersImpacted | xyseries error Page impact I'll post it as a separate answer since this one began pretty far away from what you needed.

sideview · ‎05-06-2015

Understood. You need rows of error, columns of pages, and each cell to say "9/2131" meaning that 9 users experienced that error on that particular page, and 2131 total users experienced that error across all pages. Then what you need will look like this. | stats values(ipAddress) as addresses dc(ipAddress) as impact by error Page | streamstats dc(addresses) as totalUsersImpacted by error | fields - addresses | eval impact=impact + "/" + totalUsersImpacted | xyseries error Page impact You need to do dc once in the regular stats command and then a second time with streamstats ... by error. This streamstats clause will do the same dc math but only per error. To give the streamstats the raw materials we need, we need to pass along the values(ipAddress) although after streamstats we can safely discard them.

sideview · ‎05-06-2015

If you have fields userId, page and error, it would look like this: <your search terms> | chart dc(userId) over error by page limit=100 The limit=100 is because by default the chart command will display 9 split by values and then roll all the rest up into a big one called "OTHER". limit=100 raises that threshold from 10 to 100.

sideview · ‎05-05-2015

I think what you're looking for is here - http://docs.splunk.com/Documentation/Splunk/6.2.2/SearchReference/SearchTimeModifiers or maybe here http://docs.splunk.com/Documentation/Splunk/6.2.2/Search/Specifytimemodifiersinyoursearch

sideview · ‎05-01-2015

Thanks for your answer. I understand makecontinuous better now and didn't realize it will only have a useful effect on incoming rows whose values are already bucketed, "1-5","6-10", etc.. However, digging deeper, makecontinuous has the same behavior as chart, and the key is that both of them lose their minds if the bucket size becomes less than 1. Here's a canned example of chart working perfectly, bucketing correctly, and putting in the intervening empty buckets -- | stats count | eval foo=mvrange(1,10) | mvexpand foo | eval foo=random()%10 | chart count over foo bins=10 | sort foo and here's a canned example of makecontinuous losing it's mind. | stats count | eval foo=mvrange(1,10) | mvexpand foo | eval foo=random()%10 | bin foo bins=200 | makecontinuous foo | chart count over foo | sort foo I suppose, in theory I could multiply the values by some very large power of 10, then use something truly evil like rex mode=sed to move the decimal point back over. !! Say it aint so. There has to be a better way. Unfortunately here I definitely have to support bucketing ranges into buckets of size<1. eg: many variables I need to bucket for my users in this reporting UI are floats in [0,1].

sideview · ‎05-01-2015

OK. A bit of a journey here. I am searching for a good reliable method of bucketing numeric field values into categorical ranges, with no a-priori knowledge of what the range actually is, and preferably where I can specify based on the number of bins rather than with explicit span. First step, bin command (aka bucket) The problem here is that bin doesn't represent the intervening buckets for which no data exists. Seen from the perspective of the individual search commands, the behavior makes sense. But it's unacceptable for use in charts. <search terms> | bin foo bins=10 | stats count by foo Next stop: chart command. It can actually do this... sometimes. timechart can have a "bins" or "span" command not just in the beginning to control _time, but also after the split by field, to control on-the-fly bucketing of a numeric split-by. Chart command is even nuttier, allowing on the fly numeric-to-categorical bucketing for both the group-by and the split-by field. Here it is in action: <search terms> | chart bins=10 count over foo And it works! Yay. The intervening buckets of 1-2, 2-3, 3-4 all show up in this particular case. Except... in most cases this feature completely loses its mind. Here is a good illustration. It's quite interesting behavior but unfortunately answers will only let me post two images so I have tried to find a representatively nutty one for you: <search terms> | chart bins=50 count over foo So... Is there any explanation for what makes chart command unreliable here, and is there a way to beat it with a stick to make it reliably bucket things? Or has anyone found a good flexible way to add in the empty buckets that bin/bucket omits? Bad solutions: - In other questions I've seen makecontinuous put forward as a solution, but much effort seems to suggest that makecontinuous does not do anything useful, at least not in 6.2. =/ If you can make makecontinuous solve this though, please tell me how. - Various tricks that allow for | append [| stats count | eval foo=mvrange(0,50) ] | stats sum(count) as count by foo to glue on a dummy row for each value being bucketed. While these work for particular cases, it's super clunky and I believe all such tricks require a-priori knowledge that I don't have.

sideview · ‎04-27-2015

Great. Yes those have been propagating a bit. If you have a CustomBehavior module with a "requriesDispatch" param on it set to true, you're really doing something pretty advanced! Or you copy and pasted it from somewhere and you should remove it. 😃 Happy to help.

sideview · ‎04-27-2015

It sounds like you want to filter out any A="alpha" values that happen to come after the first A="beta" value. I strongly suspect that you're going into "multivalue land" a bit too early, so in this answer I'm going to stay out in the plain old rows and do the filtering there where it's easier. Wind back what you have, and remove the stats list(A) as A_list, list(B) as B_list, list(_time) as time_list part. Instead add this `| eval isBeta=if(A=="beta",1,0) | streamstats sum(isBeta) as betaCount | where A!="alpha" OR betaCount==0 | fields - isBeta betaCount` Onto the end of that you can add back the | stats list(A) as A_list, list(B) as B_list, list(_time) as time_list if you want, and the unwanted alpha rows will have been removed. You can peel back the piped commands one by one to see how they work. OR, you can also study this completely fabricated resultset here. Paste the following search verbatim into your Splunk search bar and you'll get a result set of 8 rows, where the 7th row turns out to be an "alpha" that we want to filter out. | stats count | fields - count | eval A=split("alpha,alpha,beta,c,d,e,alpha,f",",") | mvexpand A And applying the same solution to the end of our "fake" search language gives: | stats count | fields - count | eval A=split("alpha,alpha,beta,c,d,e,alpha,f",",") | mvexpand A | eval isBeta=if(A=="beta",1,0) | streamstats sum(isBeta) as betaCount | where A!="alpha" OR betaCount==0 | fields - isBeta betaCount where as you can see the alpha is gone.

sideview · ‎04-27-2015

The answer will concern the overall module hierarchy, and the placement of one or more "dispatching" modules in a place that is basically too high up in the hierarchy. If you post it or email it I can spot it quite quickly. In the end Iit will be something like a JobProgressIndicator that is placed somewhere where it is not downstream from any Search module. Essentially when there is no other search defined at a point in the hierarchy, there is an implicit "" search over all time. If a dispatching module is therefore present somewhere where nothing is overriding that "" search, the "*" search will indeed happen. Take a look at the "Introduction to the Advanced XML" page in the Sideview Utils docs, they will tell you about "dispatching" modules and there is a section on that page specifically about how, when, where and why searches get kicked off. If you can't find the misplaced dispatching module or how to rearrange your view, you can email it to me at nick [at] sideviewapps.com and these things are very easy to spot if you know how.

sideview · ‎04-21-2015

It will end up only counting times when the requests per minute is less than 10 but also greater than 0. Basically if you think about time periods in which there are zero events, for these _time values there will be no rows at all coming into your stats command. See my answer for more details.

sideview · ‎04-21-2015

OK. Everything before the last line in your question | stats count(concurrency) by concurrency is from our previous question over at http://answers.splunk.com/answers/227393/how-to-use-the-concurrency-command-to-timechart-th.html so this is really more concerned with looking at a "frequency distribution of concurrency", not the nuts and bolts of how that concurrency is calculated. Using | stats count by concurrency like you are here will produce results that look like what you want, but it'll be an extremely misleading visualization. The reason is that the rows coming into the stats command are all either the start time or the end time of a call. There is no representation for the time in between these points. To see why this is a problem, let's look at a specific situations. Let's analyze a time period from 1pm to 2pm. Say we have 5 calls that all start right at 1pm and that are each 15 minutes long. Thinking about this, we want to ultimately see some frequency distribution with a lot of concurrency=0, and a fair bit of concurrency=5. However let's pipe such a set into | stats count by concurrency . It will give us a distribution but it turns out that concurrency=0 will have a value of 1, then every value of concurrency from 1 to 4 will have a value of 2, then concurrency=5 will have a value of 1. This doesn't seem to match our expectations very well. Then if we consider 5 other calls that start at 2pm but that are each only 1 minute long. The stats count by concurrency output here will be exactly the same as it was for the first calls. If we consider a time period that covers both sets of calls, it'll be the same chart with all y-axis values doubled. o_O Instead I think it makes more sense to analyze some discrete unit of time like minutes or seconds, and look at the distribution of concurrency values for each sourceip across all the time periods. Here in the search below I'm telling timechart to use not 1 second but rather 15 minutes as our bucket size but it's up to you. The search language looks a lot like what we did in the last question except at the end we have an extra untable and stats command. ... all the stuff before up to | eval concurrency = if(increment==-1, post_concurrency+1, post_concurrency) | timechart span=15min max(concurrency) as max_concurrency last(post_concurrency) as last_concurrency by sourceip limit=20 | filldown last_concurrency* | foreach "max_concurrency: *" [eval <<MATCHSTR>>=coalesce('max_concurrency: <<MATCHSTR>>','last_concurrency: <<MATCHSTR>>')] | fields - last_concurrency* max_concurrency* | untable _time sourceip concurrency | chart count over concurrency by sourceip That search above will analyze all the 15 minute periods for each sourceip, and give you a frequency distribution for each sourceip value, and graph them all as separate lines on the same frequency distribution chart. If instead you want to just see a single overarching frequency distribution of "per-sourceip concurrency", replace that last chart command with our old friend, {drum fill} | stats count by concurrency .

sideview · ‎04-21-2015

questions. Do all events contain a value for both A and B? Are there any events which do not have either A or B but only time? If the only events coming in have both A and B, it's probably easier to filter out the specific unwanted events with various streamstats/eventstats/eval tricks before they get rolled into your multivalue fields.

sideview · ‎04-20-2015

There are some parts to your question that make it seem like a trick question. index=_internal group=per_sourcetype_thruput | timechart count by series | untable _time series count The part where you say And I expect there to be multiple lines in stats like this: time count ip [time] null null null 12 xx.xx.xx.xx There is a row with a time value and neither count NOR ip value, and then there is another row with no time value, and an ip and a count. So it's probably best if you give a little more detail, flesh out what you want the final output to look like, or get more specific about what you're ultimately trying to do. Here is a search that will give you all of the possible combinations of ip value, and thirty second interval. it filters to only show the 30 second intervals that had more than 10 events with the given ip. | bin _time span=30s | stats count by ip _time | where count>10 Here is another example where the filtering leaves ip's that have a total of 10 rows anywhere in the set, ie regardless of the time value. | bin _time span=30s | stats count by ip _time | eventstats sum(count) as ipCount by ip | where ipCount>10 Here is another example where you get the ip's that appeared more than 10 times. Every row here also has a "time" field. the values on all rows are hte same, and they hold the time value of the single latest event in the set across all IP's. | stats first(_time) as time count by ip | where count>10 | eventstats min(time) as time Short answer: disambiguation needed! UPDATE. Throwing out one more search for you, based on some clarifications you made in other comments. This search will give you the counts and IP's, for the IP's that had more than 10 occurrences in that timerange, and if there are no such IP's whatsoever, it will give you a single row with the latest time of the overall data. that row will have an ip value of "NULL", and again that row will vanish completely if there are any real IP values in the output. | fillnull ip value="NULL" | stats first(_time) as time count by ip | where count>10 OR ip="NULL" | sort 0 + ip | streamstats count | where ip!="NULL" OR count="1"

Posts	3349
Solutions	892
Karma Given	2122
Karma Received	2938
Member Since	‎11-07-2009

Online Status	Offline
Date Last Visited	3 weeks ago

How to deal with alerts that can't be enabled in t...

Why is my python script suddenly writing "Authoriz...

Looking to rewrite a join across several indexes w...

To what degree does chaining (or nesting) automati...

Why do I get "error parsing XSL Stylesheet" when I...

Finding and reporting on callers getting bounced a...

Callmanager CDR - whether we can do reporting on t...

Callmanager CDR - reporting and alerting around sp...

Strange relationship between fieldformat command a...

calculating the total time an extension has spent ...

Re: Return all rows in which field value included ...

Re: Return all rows in which field value included ...

Re: Return all rows in which field value included ...

Re: stats count not work in sub query?

Re: Sideview Table - How to wordwrap at certain wi...

Re: Sideview Table - How to wordwrap at certain wi...

Re: streamstats sum() by not functioning as expec...

Re: complex search logic with SideView HTML

Re: Sideview Table - How to wordwrap at certain wi...

Re: Stats for unique users encountering an error b...

Re: Compare multi value field with an external fil...

Re: Stats for unique users encountering an error b...

Re: Stats for unique users encountering an error b...

Re: Stats for unique users encountering an error b...

Re: Stats for unique users encountering an error b...

Re: Limit Search by timeframe

Re: Reliable way to specify span to bucket numeric...

Reliable way to specify span to bucket numeric val...

Re: Why is a default * search kicking off in Sidev...

Re: How to delete a Value from a MultiValue field ...

Re: Why is a default * search kicking off in Sidev...

Re: How to edit my search to return one count for ...

Re: How to calculate concurrency distribution

Re: How to delete a Value from a MultiValue field ...

Re: Getting multiple stats for a single search

Join the Conversation