All Apps and Add-ons

Why is the Splunk App for Stream putting all data into _internal, and can I change the target index?

New Member

It looks like the Splunk App for Stream puts everything into _internal. In general, we don't like to do that. In this case, do we really need to? It seems like too much volume to put in _internal.

My plan is to capture summary data on most network connections for hundreds of product servers. I can't find anything about index choices in the docs. Does anyone have any recommendations on which index(es) to use? This data will be part of our security app we're building, and access will be restricted as well.

I have some data on a test server, just with the tcp and udp streams enabled as they come by default. The only messages I see in the events are: "decodePacket: skip bogus packet with length less than IP header length". Any idea what that means?

thank you!

Scott

Tags (2)
0 Karma

Engager

Hello yes you can.

I use this technique to put my streams in different indexes and manage the permissions.

You need to declare the index on your Splunk stream app indexes.conf. to do that follow this steps:

1) On your stream server : SPLUNKHOME/etc/apps/splunkapp_stream/local
2) edit or create the indexes.conf file
3) declare your indexes (already created on your indexers) like this
[yourindex]
coldpath = $SPLUNKDB/yourindex/colddb
enableDataIntegrityControl = 0
enableTsdixReduction = 0
homePath = $SPLUNK
DB/yourindex/db
MaxTotalDataSizeMB = 512000 #(in my case)
thawedPath = $SPLUNK_DB/yourindex/thaweddb

4) restart your splunkd service
5) now when you configure your streams in the stream app you can see your indexes in the dropdown menu

Hope it helps

0 Karma

Splunk Employee
Splunk Employee

This is due to tcp segmentation offload which generates a 0 length IP Header so the stream forwarder skips the data.

A Fix is due out soon, try disabling tcp offload

Regards

Splunk Employee
Splunk Employee

Stream only puts internal logs and statistics into the "_internal" index (this doesn't count towards your license volume). By default, it will put all events derived from network traffic into the "main" index. You can change the index used for each stream within the UI (at the top of the page, after you click on a particular stream in the list), or the default index for all streams by setting the following parameter in the streamfwd section of your inputs.conf file:

index=somethingelse

Note that the priority for index selection is:

  • The config for a specific stream
  • index defined in inputs.conf
  • main
0 Karma

New Member

Thank you both, that helps explain it. Does anyone know what to check why we can't see any data? All we see are the error messages to _internal about "decodePacket: skip bogus packet with length less than IP header length".

Clearly something is not working. This is win2012r2 hosted in Azure, so has a hyperv network card. splunkd is running as localsystem.

Thank you for any insight or steps to try.

0 Karma

Splunk Employee
Splunk Employee

By default only Stream internal information goes to _internal.

The real event data (by default) goes to the main/default index. You can also configure on a per Stream basis and destination index if you like. And finally, if you want to set a global destination, you can modify SplunkTAstream/local/inputs.conf and add an "index = foo" attribute.

-jsie