topic Re: Bug? splunk advanced searching/views does not display correctly in Splunk Search

Bug? splunk advanced searching/views does not display correctly

nina15 — Thu, 12 Jan 2012 04:45:03 GMT

Hi...
Its been a while I have problems with searching in Google maps or geoip which the thread was going on here: geoip search results not correct

if u follow up the thread, u see it came to a point that we all realized there exists some sort of limit that does not let geoip or Google maps to display more than ten thousand...
today suddenly, I realized its not only geoip/Google maps, but it actually is any kind of advanced searches. for instance if you search for all the data in normal search using "*", and if you have huge number of indexed data, (i.e. billions of data), u'll probably see all in the search but if you change the view to "Advanced Charting View" then you'll only see partially few thousands of those results...
Im not sure whether this is a bug or if there is some sort of limitation in any file... but that definitely causes major problems.
Does anyone have any idea how to solve this issue..?

Re: Bug? splunk advanced searching/views does not display correctly

Drainy — Thu, 12 Jan 2012 09:05:57 GMT

Looking at the module references the is a limit within the XML that you can define although you want to be careful how you adjust this as it can have an impact on the performance. Hopefully with 4.3 this won't be as large an impact thanks to the HTML5 visualisations but I expect the calculations in the background also have an equally large impact;

            <module name="FlashChart">
                <param name="width">100%</param>
                <param name="height">400px</param>
                <param name="maxResultCount">10000</param>
            </module>

Also here, http://splunk-base.splunk.com/answers/10349/chart-only-showing-1000-events , Nick has some ideas on how to expand the number of results in other ways (Nick is pretty much the expert on all things XML related)

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Fri, 13 Jan 2012 02:50:52 GMT

thanks for your response...

ok, to access the module I have to go to Manage Views, right..??
there was no such thing as maxResultCount for charting view..
i saw the width, height, even maxpages, but no max result count...

also, if that was the case, how come when I search in normal search, it gives me 5 billions of data but when I just add geoip commands to the same exact search windows, suddenly only shows 19,000 events!!?!!

to be more specific, when I search for SourceIP="" I get billions of event results, but when I search for SourceIP="" | geoip SourceIP I only get 19,000..

so for the case of advanced charting also when I search for SourceIP="*" I only get 15000 while the normal search as I said were few billions...

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Fri, 13 Jan 2012 04:17:56 GMT

from what I understand, that post is only about charting, time ranges and XML...
my problem I think resides in lower level of splunk, is not the matter of how it is represented, but the problem is that the results are not fetched at all... the poster of that thread had a problem that the results were shown in the table, but not on the chart, only.
mine is not displayed anywhere when I search in Advanced charting or when I use geoip in the normal search...

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Mon, 16 Jan 2012 05:24:03 GMT

still no answers/opinions...???

Re: Bug? splunk advanced searching/views does not display correctly

dmaislin_splunk — Wed, 18 Jan 2012 02:21:19 GMT

Nina,

Can you open a support ticket? Let me know the ticket is and I will escalate for you.

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Thu, 19 Jan 2012 01:33:52 GMT

thanks so much dmaislin for responding... i really was feeling im loosing it...
anyways, i've already opened a support ticket. the number is CASE [73624].
thanks alot for ur support 🙂

Re: Bug? splunk advanced searching/views does not display correctly

hexx — Sat, 21 Jan 2012 03:12:05 GMT

What search are you running that seems to limit its output to 10,000 rows in the advanced charting view? I am fairly certain that this limit is imposed by the default rendering of the advanced charting view, but if you were to run your search through a reporting command such as stats or timechart, you would get your full set of rows.

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Sat, 21 Jan 2012 06:37:02 GMT

I know I tried many sorts of searches that should have shown thousands of results... but on using stats or timechart I have to try first and get back to u... but if there is any limits anyways, wouldnt it prevent from all kinds of searches...?

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Sat, 21 Jan 2012 06:38:23 GMT

besides, if thats the case, why geoip behaves the same way... its very unlikely its a coincidence...

Re: Bug? splunk advanced searching/views does not display correctly

hexx — Sat, 21 Jan 2012 07:48:38 GMT

The display of result rows will be limited on a per-command basis and is typically configurable in limits.conf with maxresultsrows for stanzas such as [searchresults] or [stats]. Now, even if the display of results rows is truncated, reporting commands such as stats will still show accurate aggregates, which take all input events/results into account.

For more information and details, I would recommend to read this Splunk Answer as well as this one.

Update: Adding the information below to clarify the purpose of the advanced charting view.

The important thing to understand is that the advanced charting view is designed to process the results (not the raw events!) produced by a reporting command. This is why you will see the exact same search which would show millions of events in the flashtimeline return an approximate maximum of 10,000 results in the advanced charting view. Now, if you take that same search and pipe it to | stats count, you will see that the search powering the advanced charting view will indeed process all expected events into results before letting you decide how to render those into a chart.

To sum it up : Do not use the advanced charting view to render and view events, it is not its purpose. Instead, use it to experiment with different visualization methods to apply on the results of your reporting search.

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Thu, 26 Jan 2012 10:04:09 GMT

thanks for your response hexx,

as I'd stated in my description the view causes the problem... which is in line with your say that using other search commands (stats, timechart, etc) gives more results... I explained in my other post (the link is available above), the problem seems to raise when it has to display more than that number of results, not counts and stats... (I've already explained this in detail in my other post)

and again, in other post I did mention that I tried all possible parameters in limits.conf which includes the ones you are saying, and yet it did not have any effects neither on advanced charts nor geoip!!!

Re: Bug? splunk advanced searching/views does not display correctly

Drainy — Thu, 26 Jan 2012 10:16:02 GMT

Could you run the command
./splunk cmd btool limits list --debug
and pastebin the results with a link here please?

Re: Bug? splunk advanced searching/views does not display correctly

hexx — Thu, 26 Jan 2012 18:17:41 GMT

@nina15 : Just to be sure, could you tell us exactly what search string you are feeding into the advanced charting view? Could you tell us how many events the search reports to have found? Ideally, I'd like to see a screenshot of the search job inspector output.

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Fri, 27 Jan 2012 10:08:36 GMT

Draineh,

here the limits.conf results in pastebin

Hexx,

I'll get that and post it here ASAP. thanks

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Mon, 30 Jan 2012 02:24:52 GMT

Alright... since this problem behaves exactly the same with any kind of searching I do, I start from a very simple search first...

I am applying field extraction using DELIMS, hence I have a field called SIP which stands for sourceip...

so now, what I want to show u is results for SIP=* for a normal search, then results for same search on advanced charting view, then SIP=* | geoip SIP to also have geoip behaviour...
as you can see, both geoip and advanced chart only retrieve 10000 results !

these are snapshots for the normal search where you can see is up to 7 millions and I had to actually stop it since there was already enough results and it was taking much time... but the point is, the data that exists is way more than 10000...

then here it is on advanced charting:

and last but not least, geoip!

as you can see, both advanced charting and geoip have only 10000 results! the matching event is different. which indicates 2 problems actually:

both of them stop only on counting 10000 results
accuracy of these 2 as well is now a question mark! because if they both only fetch 10000 results, given that my search in both was the same, their matching events as well should be exactly the same!!

Re: Bug? splunk advanced searching/views does not display correctly

hexx — Tue, 31 Jan 2012 05:26:24 GMT

Thank you, now I know what's going on. Please review my amended answer.

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Tue, 31 Jan 2012 06:37:08 GMT

thank you for the update.
if the user wishes to maximize that for any reasons of their own, how would that go..?
besides, would you explain as well geoip's behavior as geoip command entered in flashtimeline also stops after 10000 results...

Re: Bug? splunk advanced searching/views does not display correctly

hexx — Tue, 31 Jan 2012 07:02:09 GMT

Looking at the MaxMind geoip app, I'm not sure that you are invoking the geoip command correctly. As far as I can tell, geoip.py is an external command used by the geoip lookup, not intended to be a search command. It should therefore be invoked with the lookup command. What if instead of :

... | geoip SourceIP

...you run :

... | lookup geoip SourceIP

Re: Bug? splunk advanced searching/views does not display correctly

nina15 — Mon, 28 Sep 2020 10:22:31 GMT

thanks hexx for detailed information and references you provide here.. but this is getting even weirder... 😄

although the command is used by all the users without the lookup command, I tried your way and received error:

[EventsViewer module] Error in 'lookup' command: The lookup table 'geoip' does not exist.

and trying the same command in Google Maps gives this error:

Rendering...
Error : Traceback: 
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/maps/appserver/modules/GoogleMaps/GoogleMaps.py", line 53, in generateResults
    for result in getattr(job, entity_name)[offset:end]:
  File "/opt/splunk/lib/python2.6/site-packages/splunk/search/__init__.py", line 1280, in __getitem__
    self.job.pushValidation()
  File "/opt/splunk/lib/python2.6/site-packages/splunk/search/__init__.py", line 610, in pushValidation
    raise splunk.SearchException, fatality
SearchException: Error in 'lookup' command: The lookup table 'geoip' does not exist.

besides, the SPP about page, located at .../app/maps/about is a help document with this search as example:

Perform a geolocation lookup for
values of the clientip field in
access_combined events:
sourcetype=access_combined | geoip
clientip

Same as the previous example, but also
perform DNS lookups in case when the
value of the clientip field is a
hostname and not an IP:
sourcetype=access_combined | geoip
clientip resolve_hostnames=true

Same as the first example, but using
the geo lookup instead of the command
sourcetype=access_combined | lookup
geo ip as clientip

etc... etc... etc...

I even tried this:

SIP="*" | lookup geo SIP

and got the same error...