Splunk Search

Timechart values not working

gnovak
Builder

I've been trying to chart some data and every way I try, it just doesn't work.

I'm able to create a table of my data fine. I use the search:

sourcetype="ec2_web" "[EVENTS]" | rex field=_raw "\d:\s+(?<event>[\w+\s]+?)\s+(?<Status>\d)\s+(?<Avg>\d+.\d+)" max_match=100 | rex field=source "/(?<script>[^/]+)$" | search script="ec2_ireland_script4.txt" | table _time,script,event,Avg

A sample of how the data looks is attached.

alt text

If I try to put this in a timechart, I get a blank chart when using values.

sourcetype="ec2_web" "[EVENTS]" | rex field=_raw "\d:\s+(?<event>[\w+\s]+?)\s+(?<Status>\d)\s+(?<Avg>\d+.\d+)" max_match=100 | rex field=source "/(?<script>[^/]+)$" | search script="ec2_ireland_script4.txt" | timechart values(Avg) by event | fields - NULL OTHER

Do I have to convert these values to chart them correctly? Why would values not work in this situation?

Tags (1)
1 Solution

sideview
SplunkTrust
SplunkTrust

Sorry - in my earlier answer I misunderstood your end goal.

It sounds like you just want to graph the 'Avg' field, but since it's already an average you don't want to do any more aggregation - max, min or avg. you want instead to just graph the values themselves.

To do this you can actually use the xyseries command.

sourcetype="ec2_web" "[EVENTS]" | rex field=_raw "\d:\s+(?<event>[\w+\s]+?)\s+(?<Status>\d)\s+(?<Avg>\d+.\d+)" max_match=100 | rex field=source "/(?<script>[^/]+)$" | search script="ec2_ireland_script4.txt" | xyseries _time event Avg

View solution in original post

0 Karma

paulmarticsi
Explorer

Sometimes there's invisible whitespace in the numbers:
eval Avg=replace(Avg," ","")
Had this same issue - numbers appear as text so don't calculate or plot, especially when extracted with REX at search.

0 Karma

sideview
SplunkTrust
SplunkTrust

Sorry - in my earlier answer I misunderstood your end goal.

It sounds like you just want to graph the 'Avg' field, but since it's already an average you don't want to do any more aggregation - max, min or avg. you want instead to just graph the values themselves.

To do this you can actually use the xyseries command.

sourcetype="ec2_web" "[EVENTS]" | rex field=_raw "\d:\s+(?<event>[\w+\s]+?)\s+(?<Status>\d)\s+(?<Avg>\d+.\d+)" max_match=100 | rex field=source "/(?<script>[^/]+)$" | search script="ec2_ireland_script4.txt" | xyseries _time event Avg

0 Karma

gnovak
Builder

Ok we changed the way the log was displaying the data, I did add the SHOULD_LINEMERGE after it still was being cheeky, and then tried your search and it works great. thanks for all the advice and assistance. Tricky but finally enjoying the chart!

0 Karma

sideview
SplunkTrust
SplunkTrust

I see now what you guys were talking about with mvzip and mvexpand. But I wouldn't bother with that - this data shouldn't be multiline so just reindex with SHOULD_LINEMERGE="False" and I think it'll show up. I don't think xyseries can deal with multivalue fields and maybe it just quietly fails.

0 Karma

gnovak
Builder

If I change the order of the fields after xyseries i start to see at least a legend but I haven't been able to get the data to show up

0 Karma

gnovak
Builder

well the data is extracted into fields so i don't see how it really matters. It's just odd

0 Karma

sideview
SplunkTrust
SplunkTrust

Weird. they removed it from the menu. You can see it if you just manually replace "flashtimeline" with "charting" in the URL. but it doesn't matter. I just ran an xyseries search in the flashtimeline view and it should run fine there.

index=_internal source=*metrics.log group=per_sourcetype_thruput | xyseries _time series kb

And ABSOLUTELY. If you're indexing these events as giant multiline events, then who knows what's happening but that's probably the reason. Fix that and reindex the data and it should make a lot more sense.

0 Karma

gnovak
Builder

Where is this? I'm in the search app, I run the search, I don't see any option for advanced charting? I also tried adding | fields - NULL OTHER to the end of the search but that didn't do anything. So you understand I'm running this independent of an advanced xml dashboard config. I'm running this in the search app.

0 Karma

sideview
SplunkTrust
SplunkTrust

run it in the advanced charting view, not in flashtimeline. Also is it possible that "everything being blank" is just because the first page is all null values? can you throw in a | search event=* Avg=* to restrict your incoming events to only the events that are going to be meaningful?

0 Karma

gnovak
Builder

I don't even get a chart or those chart options. On a side note i'm going to do an experiment to have the logs separate each event into its own line and try charting that way. I'm not sure if this is being strange because of the grouping of this data as one event.

0 Karma

gnovak
Builder

If I use
sourcetype="ec2_web" "[EVENTS]" | rex field=_raw "\d:\s+(?[\w+\s]+?)\s+(?\d)\s+(?\d+.\d+)" max_match=100 | rex field=source "/(?

0 Karma

sideview
SplunkTrust
SplunkTrust

So you can see _time values in the table, and you can see values of the event field across the columns of the table, and there are Avg values populating the table, but nothing shows up.

What kind of chart type are you using? When you create a report this way with no aggregation there are lots of null values in the data, and when there are lots of null values, if you are using "line" chart, with "nullValueMode" left at it's default of "gaps" and "showMarkers" left at its default of "False", then the chart will literally display nothing... Change the above settings or the chart type.

0 Karma

gnovak
Builder

The numbers are displayed in milliseconds. Perhaps I should try changing milliseconds to seconds?

0 Karma

gnovak
Builder

It's strange but this didn't work either. I'm wondering if it has something to do with how this data is presented. I'm going to try a few more things. Whatever help you have given is greatly appreciated though. It's strange but when using xy series, nothing shows up. It's just all blank.

0 Karma

gnovak
Builder

Hmm...everything came up blank. I'll see about tweaking it perhaps

0 Karma

linu1988
Champion

eval t=mvzip(event,AVG)|eval z=split(t,",")|eval event=mvindex(z,0)|eval avg=mvindex(z,1)|table _time,event,avg

i am not sure where you need to apply mvexpand as i dont have the same records with me

0 Karma

gnovak
Builder

I'm reading up on the command now to try it out. I'm not quite sure how this works. Tried a few times and so far didn't get the results I expected. It says The mvexpand command only works on one multivalued field.

0 Karma

linu1988
Champion

Could you try mvzip(event,Avg),Split to two different fields,mvexpand then create a table on _time,event,Avg? See if gives you any chart on splunk search app..

0 Karma

gnovak
Builder

well a bar graph of the values over time by event is what I'm aiming for.

0 Karma

Ayn
Legend

Yes, timechart won't understand a list of values - it looks for one numerical value, not a list of them. (How would it chart them?)

What do you want the chart to look like?

0 Karma
Get Updates on the Splunk Community!

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...

DevSecOps: Why You Should Care and How To Get Started

 WATCH NOW In this Tech Talk we will talk about what people mean by DevSecOps and deep dive into the different ...