@MuS, done. Thanks! ... View more

Face slap :-). Thanks, @steveyz. I've edited my answer and replaced NULL with null() (and tested that this still works). ... View more

Yes, for the original poster's specific use case, based on the information provided here, I agree. However, while I came here looking for an answer to the same one-liner question, "How to omit from a timechart series that include only zeroes?", my use case is slightly different. In my case, each event represents a database request. Each event contains, among many other fields, nine "suspend clock" fields, representing the different ways in which a database request might be suspended. Requests can be suspended in a combination of these ways. I want to visualize these suspend time "components" together, on a single chart. But I want (as per the question) to omit any series whose values are all zero. So, I can't simply omit events with a zero value in one of these fields. That was another reason for me to add my answer (although I was hesitant to do so, not wishing to offend or break etiquette): it occurred to me that other users might also have the same (one-liner) question, but with a use case closer to mine. ... View more

I'm still grateful for the pointer to map , but decided to accept the answer from @bchung - not because of this original answer, but because of the example script he later added in a comment. (I'll take tips on how to better manage acceptance of answers in such a situation: I want to do the right thing; I don't want to offend anyone.) ... View more

(Picks up dropped mic.) Thank you, @bchung. You just fast-forwarded my reality ;-). I knew I'd have to look at developing custom search commands in Python sooner or later, but you've handed to me on a plate an example that I can understand, use immediately, and customize. Thank you, thank you. I've accepted your answer based on this latest comment (I acknowledge this might be dodgy etiquette, but I am not going to ask you to submit this as a separate answer). ... View more

I had the same question, saw the accepted answer, and decided to develop my own solution. Some reasons not to add my answer: The question is several years old An answer has already been accepted The accepted answer was by someone with high karma points I'm a Splunk newbie, so I might well be missing something here Reasons to do it: My answer performs processing after timechart , which might be more performant than the accepted answer (Lisa, your thoughts on this?) Even if my answer is wrong, I hope I'll learn something from subsequent comments, probably from @lguinn 🙂 My answer (with line breaks for readability): ... | timechart ... by queue_name | foreach * [eval nonzero_<<FIELD>> = if('<<FIELD>>'==0 OR "<<FIELD>>"==_time, null(), '<<FIELD>>')] | fields _time nonzero_* | rename nonzero_* as * Here's the "trick": if you assign a null value by calling the function null() , the eval command does not create a field. So, the foreach command only creates a nonzero_* field if the original field has at least one nonzero value. The subsequent fields command only keeps the _time field and any nonzero_* fields. The rename command reverts to the original field names. ... View more

Thanks for the tip about the map command, and for the link. I've read and reread your answer to that question carefully, and recaffeinated ;-), but I cannot see how to adjust that approach for my use case. In the meantime, I've developed a solution for my use case that avoids this question. For details, see my comment on the answer by @richgalloway. (I'll admit, this demotivates me from spending more time right now looking into map .) Still, I suspect that map is as close an answer as I'm going to get (to this specific question about dynamically building commands) without delving into developing a custom search command in Python. If this is still the best answer after the weekend, I'll accept it. Thanks again! ... View more

Hi Bill, Thanks very much for the detailed answer. However: you should do the lookup for conn_type before timechart I disagree. For details, see my comment on the answer by @richgalloway. ... View more

Thanks! And, yes, you're absolutely correct. However, the following point from my question remains: I could replace all of the original values with readable values before creating the timechart [...] but I'd prefer to use a technique that doesn't involve processing every input event for the timechart. That seems like too much processing. The Splunk documentation makes the same point: Optimizing your lookup search If you are using the lookup command in the same pipeline as a transforming command, and it is possible to retain the field you will lookup on after the transforming command, do the lookup after the transforming command. This morning, I came up with the following solution: sourcetype=my_log_type | timechart count by conn_type | untable _time conn_type value | lookup connection_types.csv conn_type output description | xyseries _time description value This uses untable and xyseries with lookup instead of renaming fields. I now have a solution for my use case, so my original question about dynamically building commands is, to me, academic (until I hit a use case that needs it). Thanks again for your input. Your answer prompted me to think harder about how to do the lookup after timechart . ... View more

@somesoni2, The following search - your answer, slightly tweaked - produces the desired result (on my system): sourcetype="steverimar" | eval temp=Name."#".Month."#".Year | fields temp Data* | untable temp data Value | rex field=temp "(?<Name>.*)#(?<Month>.*)#(?<Year>.*)" | eval Month=Month+tonumber(substr(data, 5))-1 | table Name Month Year Value To exclude the unwanted rows, I replaced your original: fields - Name Month Year with: fields temp Data* To increment the Month values, I inserted: eval Month=Month+tonumber(substr(data, 5))-1 One thing I liked about your answer is that it didn't rely on the field names being Data1 , Data2 ... But my tweaks rely on those names. Can you offer alternative tweaks to fix the nits without introducing this dependency? ... View more

Nice! I learned some new tricks from your search string, thanks. I like your answer better than mine in several ways; for one, it's more concise. On my system, with the "test" event I described in my answer, the following search: sourcetype="steverimar" | eval temp=Name."#".Month."#".Year | fields - Name Month Year | untable temp data Value | rex field=temp "(?<Name>.*)#(?<Month>.*)#(?<Year>.*)" | table Name Month Year Value produces the following results (as displayed on the Statistics tab): Name Month Year Value Steve 2 2015 1 Steve 2 2015 2 Steve 2 2015 3 Steve 2 2015 4 Steve 2 2015 5 Steve 2 2015 localhost:8088 Steve 2 2015 test Steve 2 2015 1 Steve 2 2015 {"":_"",_"":_,_"":_,_"":_,_"":_,_"":_,_"":_,_"":_} Steve 2 2015 http:GXH Steve 2 2015 steverimar Steve 2 2015 MYPC (I wish this forum supported table formatting. Does it? I've tried GitHub-flavored Markdown table formatting: nope. And HTML table, tr, and td tags get "sanitized" away.) Some nits: The results include not just the desired Data1 , Data2 ... values, but also the values of the following fields: host , index , linecount , punct , source , sourcetype , splunk_server . (Or is this my problem; Splunk behavior that I can change on my system?) The Month column has the same value (2) in every row, whereas, according to the question, the value should increment by 1 after each row, starting from the value of the original Month field: 2, 3, 4, 5, 6. ... View more

I would welcome alternatives to: eval Value=spath(_raw, data_column_name) That is: A field (in this example, data_column_name ) contains the name of another field. You want to get the value of that other field, and use it to set the value of a third field (in this example, Value ). ... View more

I second the request from @somesoni2: Would you be able to provide the actual field names from your current result? The following solution is based on the specific field names that you cited: Data1 , Data2 ... This solution is deliberately designed to work for a variable number of Data fields. I started with the following input data, in JSON format: { "Name": "Steve", "Month": 2, "Year": 2015, "Data1": 1, "Data2": 2, "Data3": 3, "Data4": 4, "Data5": 5 } For testing purposes, I deliberately specified unique values for each Data field. The following search string (with line breaks inserted for readability) generates the output that you specified (I ingested the input data as sourcetype "steverimar"): sourcetype=steverimar | eval data_column_name="" | foreach Data* [eval data_column_name=data_column_name + " <<FIELD>>"] | makemv data_column_name | mvexpand data_column_name | eval Value=spath(_raw, data_column_name) | eval month_offset=tonumber(substr(data_column_name, 5)) - 1 | eval Month=Month+month_offset | table Name, Month, Year, Value Line-by-line explanation: sourcetype=steverimar Get the matching events. eval data_column_name="" Create a new field named data_column_name , and set its value to an empty string. foreach ... For each field whose name starts with "Data", append the name of that field to the value of the data_column_name field. makemv ... Convert data_column_name into a multivalue field. mvexpand ... Expand the values of data_column_name into separate events, one event for each value. eval Value ... Create a new field named Value , and set its value to the value of the field whose name matches the value of the data_column_name field. For example, if the value of the data_column_name field is "Data1", then set the value of the Value field to the value of the Data1 field ("1"). eval month_offset ... Create a new field name month_offset , and set its value to the numeric suffix of the value of the data_column_name field, minus 1. eval Month ... Add the month_offset to the value of the Month field. (You could merge this step with the previous step. I kept them separate here for clarity.) table ... Transform the search results (for display in the Statistics tab), limited to the fields that we want in our final results. The technique used in steps 2 to 6 is described in the Splunk docs topic "Build a chart of multiple data series". Here's a screenshot of the results in the Statistics tab of Splunk Web (Splunk Enterprise 6.4): It occurs to me that this solution might not be exactly what you need, because your actual column names might not be Data1 , Data2 , etc., but this solution matches the column names that you specified in your question. I have not made the search string "smart" enough to increment the year if the month number is greater than 12. That would be easy to do. ... View more

Thanks for the answer. No, I hadn't tried adding another static check box. Yes, that works. I looked at the "Check Box Input" example in the Splunk 6.x Dashboard Examples app, and it seems that this behavior - "Search is waiting for input" when no check boxes are selected - is by design. I can imagine this behavior was the subject of some debate between the Splunk designers. I have mixed feelings about it. In my specific case, I want to offer the user a choice between two states: Exclude blacklisted transaction codes (the default state) Include them For this case - without considering Splunk-specific implementation issues - a single check box seemed to me the most appropriate choice of UI control. However, in Splunk, as we've seen, if no check box in a group is selected, the corresponding token remains unset, resulting in a "Search is waiting for input" message, which is not the behavior I want. At least, not in this situation; I'm open to learning about situations where this behavior is desirable. Thanks again for your answer. It's prompted me to rethink how to represent such a choice to the user in a way that is straightforward to implement in Splunk. Rather than introducing a second check box, which, logically, offers the user a choice between not two but four states, I have, for now, replaced the single check box with two radio buttons: Blacklisted transaction codes: [ ] Include [X] Exclude Simple XML: <input type="radio" token="blacklist" searchWhenChanged="true"> <label>Blacklisted transaction codes:</label> <choice value="">Include</choice> <choice value="NOT [|inputlookup tran_blacklist.csv]">Exclude</choice> <default>NOT [|inputlookup tran_blacklist.csv]</default> </input> I'd still prefer to use a single check box, but this is an acceptable workaround for me. Finally, regarding the "Check Box Input" example in the Splunk 6.x Dashboard Examples app... I think that the presence of the "ANY" check box in this group is problematic (I can elaborate, if asked). And if you select multiple check boxes and inspect the resulting search string, you can see that the "OR" delimiter lacks leading and trailing spaces: (sourcetype="*"ORsourcetype="splunk_web_access") I'm currently using a free Splunk license on my PC (I'm discussing this situation with local management), so I can't submit a case for that. What, if anything, should I do about that? Report it here in the guise of a question, perhaps? (Or buy a license. Yes, fair call.) ... View more

Hi @ppablo I viewed Martin's profile before downvoting. I knew I was downvoting an answer from someone with a lot of karma points. However, I hadn't seen that proper etiquette link before; thanks for that. I see people expressing differing opinions there, but the consensus seems to be, as you say, to reserve downvoting for the cases you cite. I will follow that practice in future. I agree unequivocally with the following comment on that page: one thing i'd like to say in particular is that i want to emphasize that downvotes should never given, nor taken, personally At the time, it did not occur to me that downvoting this answer might be interpreted as "taking a negative approach in how I engage with other folks", but after reading the comments on that proper etiquette page, and based on the reaction from you and Martin, I can see that I was both naive and ignorant. Again, I apologize. I sincerely did not mean to offend. ... View more

I've belatedly seen the question "How do I get the time span (span=X) in a search to automatically adjust depending on the time picker value chosen?", which I think answers my original question (I'll try it tomorrow), but now I'm mired in the follow-on issues, mentioned in my previous comment, of aggregating over time in the base search, and then charting selected values in the post-process searches. ... View more

Thanks for the quick response. Yes, you're absolutely right: timechart dynamically selects the span, and all panels in my dashboard show the same span. Works great. What I'm concerned about is when I change those (currently self-contained, standalone) search strings for each timechart to use a base search, and then I specify a time range that encompasses a few million events, thereby hitting the limits described in that Splunk doc topic I cited. That's when I think I'll need to worry about explicitly setting a bin span and aggregating in the base search, so that the base search isn't attempting to pass millions of search results to the post-process searches. And since the aggregation will be done in the base search, I wonder whether to continue using timechart , but that introduces other issues. For example, here's a base search (note: with a fixed bin span; not what I want to end up with): sourcetype=my_log_type | bin _time span=5s | stats avg(response) as avg_response, avg(cpu) as avg_cpu by transaction_code, _time (in practice, the stats function would process more fields; I've deliberately kept this example short, averaging only two fields.) and here are two corresponding post-process search strings. A timechart : timechart avg(avg_cpu) by transaction_code and an xyseries : xyseries _time, transaction_code, avg_response Using xyseries avoids "double aggregation" (averaging averages, producing incorrect results). However, xyseries doesn't "span": it marks every discrete time value on the X-axis with the complete time stamp value (optionally truncated); but if those values don't fit along the X-axis, you'll get no time values displayed on the X-axis at all. Whereas timechart does a good job of marking the time on the X-axis at readable intervals. I'd appreciate advice on all of this. The more I look at that (bogus) avg(avg_cpu) in my example timechart (above), and then consider the cons of using xyseries instead, the more I think I'm making some fundamental error in my approach to managing that (understandable) "bottleneck" limitation of passing a huge number of search results from the base search to the post-process searches. ... View more

I apologize. I had no such expectation, but I can see how it might look to you. I showed someone at work some comments I had added to existing questions on Splunk Answers, and they asked me why I did not upvote or "me too" the questions. I decided they had a point. That prompted me to return to this question, that I had recently commented on, and downvote it for the reason I gave. In hindsight, I should have performed a single action - a downvote with a comment - rather than commenting, and then downvoting almost immediately after. I appended that "(Before downvoting...)" explanation because I wasn't sure how or where my reason for downvoting would appear (I now realize it appears in the comments thread). Again, I apologize for my poor etiquette. I did read the question carefully, and I did spot that "For example..." in the details of the question. But my reason for downvoting your answer remains. My answer addresses exactly that, the table will update. Yes, I agree, it does. If you have a different question, feel free to open a new question. My question is "Can timechart zoom alter time range picker?"; the same as this question: that is, the single line that Splunk Answers characterizes as the question ("Type your question here...") as opposed to the multi-line details ("Fill in the details..."). I'm unsure how to proceed. Opening a new "Can timechart zoom alter time range picker?" question feels wrong to me, and asking the same question using different words also feels wrong. I'd be grateful for your thoughts. ... View more

Thanks for this answer. I just ran into the same issue (in Splunk Enterprise 6.4). I am sending JSON-format logs to Splunk via the HTTP Event Collector (EC). Some field values are explicitly set to null : some_performance_indicator_field: null One reason for setting a field value to null , rather than omitting the field, is to indicate that the version of the performance monitoring system that generated this data does report that performance indicator (from the underlying transaction processing system being monitored), but that, in this instance, there was no value to report. As opposed to a (typically earlier) system that does not report that indicator, and so that field does not occur in its output. A null value tells you something about the performance indicator, whereas, if the field is absent, the performance indicator might have had any value; it just wasn't reported. So (with apologies to readers who are well aware of this), there can be a significant semantic difference in JSON between a field being set to the explicit value null and that field being omitted. It appears (and my own ad hoc testing confirms this answer) that Splunk interprets the JSON null field value as the string value "null" . That behavior is consistent with the following definition in the Splunk documentation topic "The search processing language syntax": Null field A null field is not present on a particular result or event. (Although I would prefer it if the documentation explicitly covered the case of the JSON null value being interpreted as the string value "null" , and any similar issues with other input data formats. Perhaps it does; just not in that topic.) I presume (I haven't tested this) that this means that, for the purposes of searching in Splunk, the following JSON KV pair: myfield: null is identical to: myfield: "null" ... View more

Thanks for persisting with this, much appreciated. Strictly speaking, no: what you're suggesting is not what this question is about. This question is about something similar to Grafana's "shared crosshair", which highlights - by which I mean shows a marker, such as a vertical line - at the same point in time in other time-based charts as the one that you're currently "mousing over" (forgetting about touch devices, for now), without actually panning or zooming. However, you hit a nerve with your "Pan and Zoom"-based answer, because I'm very interested in an answer to that (separate but related) question, too. Regarding that question: yep, exactly as you say: I have a dashboard with a time range picker and various panels. Some of the panels display timecharts. When I zoom ("marquee select" a time range in) a timechart, I want the entire dashboard - the panels and the time range picker - to reflect that zoomed time range. I'll come back to you later. That's very kind of you, thank you! ... View more

Thanks for the clarification. You can adjust all charts to the selected timeframe with the pan and zoom function. Are you referring to using <set token="..."> in the <selection> of a timechart to set tokens, and then have other charts refer to those tokens as their time range? If so, I have issues with that approach, as described in my comments on the question "Can timechart zoom alter time range picker?". The related question "How to "match zoom" from any chart on a dashboard" is attempting to achieve more or less the same thing, except that I want zooming to also update the time range picker. If not, what technique are you referring to? I would be happy to learn that my preconceptions based on using Kibana (Elastic Stack) are blinding me to how to easily achieve this syncing in Splunk. ... View more