topic Why did Splunk use up our ingest bandwidth? in Installation

Why did Splunk use up our ingest bandwidth?

adepasquale — Wed, 17 Jun 2020 00:26:00 GMT

Last night a received an alert that we went over our bandwidth Cap. Upon closer inspection, I saw a new "host" in the 30 day license chart... the host was SPLUNK. I ran this command (same as the 30 day license command) only i limited it to just Splunk

index=_internal 
    [ `set_local_host`] source=*license_usage.log* type="Usage" h="SPLUNK"
| eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h) 
| eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s) 
| eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx) 
| bin _time span=1d 
| stats sum(b) as b by _time, pool, s, st, h, idx 
| timechart span=1d sum(b) AS volumeB by h fixedrange=false 
| join type=outer _time 
    [ search index=_internal 
        [ `set_local_host`] source=*license_usage.log* type="RolloverSummary" earliest=-30d@d 
    | eval _time=_time - 43200 
    | bin _time span=1d 
    | stats latest(stacksz) AS "stack size" by _time] 
| fields - _timediff 
| foreach * 
    [ eval <<FIELD>>=round('<<FIELD>>'/1024/1024/1024, 3)]

This report shows that Splunk NEVER logs anything towards our ingest limit... until yesterday where it indexed .25 GB of data. What would cause this?

Re: Why did Splunk use up our ingest bandwidth?

dflodstrom — Thu, 12 Jan 2017 18:52:47 GMT

Are you able to share the source or sourcetype(s) of these events? Is it possible that someone added something like the TA for unix which contains inputs?

Re: Why did Splunk use up our ingest bandwidth?

adepasquale — Thu, 12 Jan 2017 18:57:39 GMT

Sure none of this data is sensitive so just let me know what commands you'd like to see run. What is interesting is if i just search for the host "SPLUNK" i get a lot of audit data.. how would a limit that search to just stuff that counts against our ingest limit.

also, I'm really the only one who uses this machine and i don't see any add-ons recently installed.

Re: Why did Splunk use up our ingest bandwidth?

adepasquale — Thu, 12 Jan 2017 19:01:21 GMT

Never mind.... just saw that someone added data to splunk using the drag and drop add data feature that day!

Re: Why did Splunk use up our ingest bandwidth?

dflodstrom — Thu, 12 Jan 2017 19:02:23 GMT

Data logged to internal indexes (_internal, _audit, _introspection) should not count against your license. Try this (start the search with the pipe) "| tstats count WHERE index=* host= by source"