Moved question to Splunk Adminstration, "Getting Data In", where you will get better answers than in "Search".   ... View more

Much of what you are doing does not make much sense. The initial dedup will give you exactly one record for each host and package, so all the following logic that presumes there is anything to count, or any MV in  the records, is not needed. The latest() aggregate function  for stats will pull you the last value, so you don't even have to dedup. So, this gets you the equivalent output of what you wrote. tag=Windows_Update package=* host=* | fields host package eventtype | stats latest(_time) as _time latest(eventtype) as eventtype by host package | eval status=case(eventtype=="Update_Successful", "Successful at ("._time.")", eventtype=="Update_Failed", "Failed at ("._time.")", true(),"NA") | table host package status   Now, if you wanted a history, including the last value, then you could do something like this: tag=Windows_Update package=* host=* (eventtype=="Update_Successful" OR eventtype=="Update_Failed") | fields host package eventtype | eval status=case(eventtype=="Update_Successful", "Successful at ("._time.")", eventtype=="Update_Failed", "Failed at ("._time.")") | sort 0 host package - _time | stats latest(status) as Last_Status list(status) as Status_History by host package     ... View more

Subsearches are limited to 50K records, so this append isn't  working for you. | append [|inputlookup ram_error.csv | where not ID IN ($mytoken$) ] . There is a slightly different format you can use... | eval rectype="newrec" | inputlookup ram_error.csv append=true | where (rectype="newrec") OR ( not ID IN ($mytoken$) ) | fields - rectype     ... View more

Simple to understand and simple to support are good decisions. Well done. ... View more

Try this index=pan app=* | stats count values(app) as app by src | where NOT (app="ssl")   ... View more

Okay, look at what happens when you do these commands   | makeresults | eval myfield1=mvappend("a","b","c") | eval myfield2=mvjoin(myfield1,"!!!!") | eval myfield3=makemv(myfield2,"!!!!")  and then this command | mvexpand myfield3 ... View more

You're quite welcome.  We love to help. ... View more

kmeans has an option for setting a range of ks to attempt.     | kmeans k=3-12   Just feed it different data a few times and see what it does for you.  Here's the reference. https://docs.splunk.com/Documentation/Splunk/8.0.4/SearchReference/Kmeans ... View more

I'm skeptical that predict would be the right way to do that.  It seems like the right thing to do would be, each night off peak, to calculate the next day's boundaries once  for each 5, 10 or 15 minute increment, and output those times and limits to a lookup table. Then, you'd just have to calculate the current errors and read the lookup table to get the limits for whatever _time and site you are running and test the compliance.    ... View more

A summary index can contain literally any number of columns.  Just output the record with one column for each item you want to report.   So, if an event had values for functions a, c r and t, and the Overall function was 1, then it might look like    (time) total_function=23, overall=1, a=12, c=7, r=0, t=15     or, if I misunderstood your meaning, maybe it might be    (time) total_function=23 overall="1;3" detail="a;c;r;t"   or   (time) total_function=23 overall="1;3" detail="a=12;c=7;r=0;t=15"     The next record does not have to have all the same fields.   ... View more

Try this:   index=My_log | stats count by Account_ID Cell_Number | lookup accountId.csv Account_ID output Account_ID as foundme | where Account_ID = foundme | table Account_ID Cell_Number     Notes: 1) You have to keep all the fields you need in the stats command somehow, or they will not exist afterwards. 2) When you output the lookup results, you need to give it a new name or you won't know whether it was found or not.   ... View more

Here's one way: index=pan app=* | stats count by src app | where app!="ssl"   Here's another: index=pan app!="ssl" | stats count by src   ... View more

I see this as a nontrivial version of Splunk soup.  I'd proceed like this... index=A OR index=REL or index=B OR index=C | fields index parent child name status id | rename COMMENT as "double the REL records, levaing the others single" | eval myfan=mvrange(0,if(index="REL",2,1)) | mvexpand myfan | rename COMMENT as "set up match keys and data fields" | eval A_id=case(index="A",id, index="REL" AND myfan=0,parent) | eval B_id=case(index="B",id, index="REL" AND myfan=0,child, index="REL" AND myfan=1,parent) | eval C_id=case(index="C",id, index="REL" AND myfan=1,child) | eval A_name=case(index="A",name) | eval A_status=case(index="A",status) | eval B_name_status=case(index="B",name."!!!!".status) | eval C_name_status=case(index="C",name."!!!!".status) At this point records look like this index=A id name status A_id A_name A_status index=B id name status B_id B_name_status index=C id name status C_id C_name_status index=REL myfan=0 parent child A_id B_id index=REL myfan=1 parent child B_id C_id   Then I'd proceed like this... | rename COMMENT as "reduce to required fields with one of these two" | fields - id name status parent child | fields index myfan A_id A_name A_status B_id B_name_status C_id C_name_status | rename COMMENT as "roll data from REL myfan 0 to A, then myfan=1 to A, tehn drop REL" | eventstats values(eval(case(myfan=0,B_id)) as B_id by A_id | eventstats values(eval(case(myfan=1,C_id)) as C_id by B_id | where index!="REL" | rename COMMENT as "now we have only A, B, C records, and the A records have all relevant keys." | rename COMMENT as "Roll B record to A then drop B" | eventstats values(B_name_status) as B_name_status by B_id | where index!="B" | rename COMMENT as "Roll C record to A then drop C" | eventstats values(C_name_status) as C_name_status by C_id | where index!="C" | rename COMMENT as "Above could be a stats" | rename COMMENT as "Add placeholders to handle potential NULLS" | eval B_name_status=coalesce(B_name_status,"N/A!!!!N/A") | eval C_name_status=coalesce(C_name_status,"N/A!!!!N/A") | rename COMMENT as "split up the records, then the fields" | mvexpand C_name_status | mvexpand B_name_status | eval B_name=mvindex(split(B_name_status,"!!!!"),0) | eval B_status=mvindex(split(B_name_status,"!!!!"),1) | eval C_name=mvindex(split(C_name_status,"!!!!"),0) | eval C_status=mvindex(split(C_name_status,"!!!!"),1) | rename COMMENT as "drop unneeded fields" | table A_name A_status B_name B_status C_name C_status That's all air code, so you'd have to shake it down with a small subset of the records before running the whole data set.       ... View more

That SPL should not limit the lookup to 10K, and we have lookups that are in the millions.   Try this :  | inputlookup ram_error.csv | stats count  If that number is over 10K, then the problem is not with the part you've told us, but with the search you're using to bring the data back.  Show us that, and we can help you fix it.  ... View more

Okay, you need to figure out which records are dropped.  Here's how I would work this. 1) run each job for the exact same time range. 2) use |loadjob to load the output data from each job and cut it down to a few key fields. 3) use diff to compare the two files and find which records were added/deleted. 4) cut the file down to those records, look at just those records and see what's going on.   It may be that they are duplicate records that are being dropped somehow, or it may be that something in the search that you havent shown us has a reason to act differently  ... View more

Switch the view to look at the details (_raw).  That field has spaces on either side of the word " all ".  How is it defined in the JSON?  It's probably an error in the extraction routine, possibly caused by an error in the JSON itself. ... View more

To be clear -    1) Searches work from the farthest inside  braces, and work out, with the exception of the braces that follow an appendpipe (we'll come back to that.) 2) The subsearch in braces will return some set of information, and what set of information is defined by the verb that preceded it, if any. 3) If the braces are before the first pipe in the outside search, then the braces will return data as if the inside search had been run, then piped to the format command.  thus, if the end of the search in braces looked like this...       [ search index=foo ... more search terms ... | table host ]       then it returns as if the search had been run like this       index=foo ... more search terms ... | table host | format       and the output field called search contains something like this       ( host="value1" ) OR ( host="value2" ) OR .... ( host="value99" ) )       Before the first pipe, the value of field search is what will be dropped into the outer search. In your case, the inner search is just taking the data out of the lookup table to create the above stuff.   The next step in your code, it runs that against the index as a subsearch.  This is the place where @woodcock 's alarms went off.  There are limits on how much data a subsearch will return, so you may get results that are wonky, depending on how long you run the search over.  But let's assume it works. Your data comes back from that subsearch, and then it gets run through on the right side of a join. Now, joins in Splunk are always, at their most basic level, left joins.  Every record on the left side gets matched to the first matching record on the right side.  If you have NOT told it to match every record on the right, then only the first gets matched. Then, if you've told it you only want an inner join, it will throw away unmatched records from the left side. (These are some of the reasons we suggest people avoid join when they can.)  Finally, what you are matching to on the far left is the same lookup table records you had at the far inside.   On the other hand, if you use woodcock's version, then it works this way  All the records from any host are read. For each record, the lookup table is accessed in memory for a match.  If it is found, then the host value from the lookup table is copied  to a new field.  Next, if the new field is not found, the record is dropped as not being wanted. Now, if it was possible that there would be no records for any particular host, and you wanted to make sure they were there, then you could add an append at the end like this   | inputlookup bsl_project_host.csv append=true     And, one final thing.  after appendpipe, instead of the braces being processed first, they are processed when the data reaches that point in the search.  the entire set of records up until that point is put through the logic inside the braces, and then the output of that (subject to subsearch limitations) is added onto the end of the current set of records.      ... View more

  | addtotals fieldname="Total Count" | addcoltotals labelfield=date_mday label="All Days"   The addtotals command will add up the totals horizontally, the addcoltotals will add them vertically. I've updated the code above to include these.   ... View more

That's a very good start, @bowesmana.  Two additions...  1) As a practice, we always include in the pseudocode a fields command to limit the junk and speed the search. If beginners learn that strategy early on, it will save them centuries of machine time.  When doing values(*) as *, it's especially important. 2) streamstats is finicky with time_window, so if we're doing anything complicated, then we usually include a sort 0 to explicitly validate the event order right before the streamstats.     base search | fields _time user ... the exact fields that you want to know about ... | sort 0 _time user | streamstats time_window=101ms values(*) as * by user     I gave it 1 extra ms, since I can never remember whether streamstats is inclusive or exclusive, and with ms it might matter.    ... View more

@gcusello  You probably also want to give him the code for the "30 second" part of his request..  Here's one cut at that:   | index=wineventlog (host=PC1 EventCode=4648) OR (host=PC2 EventCode=4624) | fields _time EventCode host | sort 0 _time EventCode host | streamstats time_window=31s values(EventCode) AS eventcodes dc(EventCode) AS bothpresent list(host) as hosts | where bothpresent=2   Or, if you needed the _raw from the two events...   | index=wineventlog (host=PC1 EventCode=4648) OR (host=PC2 EventCode=4624) | fields _time EventCode host | sort 0 _time EventCode host | streamstats time_window=31s values(EventCode) AS eventcodes dc(EventCode) AS bothpresent list(host) as hosts values(_raw) as Raw | where bothpresent=2   Also, @dsdeepak , you might want to correlate some other field, such as the user field on both machines, into the search.  If your hypothetical attacker would have the same Windows user field on both machines, then you need a little more code.  All this search will do is detect when there is ANY 4648 on one machine and 4624 on the other, not necessarily connected to each other.      ... View more

  Try something like this: your search that gets individual ratings in this form | fields _time host "% _total Prcessor times" | bin _time span=1m | stats min("% _total Prcessor times") as min% by _time host | eval MyWarning = case(min% > 0.8,1) | streamstats time_window=301s sum(MyWarning) as Warning5m by host | where Warning5m > 4  Modify the MyWarning eval for whatever format your data actually returns in.  80 or 0.80 or whatever. ... View more

There are a number of things I'd check.  Python is finicky about indentation, so I'd probably write a python script with a cut-back version of the SPL to make sure what the values are immediately before the stats command, if none of the following fix it. Here are some ideas -  1) Run the stats together on one line. 2) Put quotes around GET 3) Use tonumber to force RequestTime to be a number, in case for some reason it is being evaluated as a string. 4) Make sure that all lines are at the same indentation. 5) Add commas between each clause in the stats line.     search index=cdvr host=* AND source="/var/log/nginx/access.log" AND sourcetype="gemini-ecdn-nginx-access" | rex field=_raw ".*?\t.*?\t.*?\t.*?\t(?<Method>\w+)\s/(?<URI>.+?)\sHTTP.+?\t.*?\t(?<Status>.+?)\t.*?\t.*?\t.*?\t.*?\s.*?\t.*?\t(?<host_header>.+?)\t" | rex field=URI "(?<RecordingID>.*)\.(?<resource>.*)?\?.*" | dedup RecordingID | search Method="GET" resource="m3u8" | stats count(eval(tonumber(RequestTime)<2.00)) as PlaybackNumSuccessful count(eval(RecordingID)) as PlaybackNumTotal | eval PlaybackNumFailed=(PlaybackNumTotal-PlaybackNumSuccessful) | eval SuccessPer = (PlaybackNumSuccessful/PlaybackNumTotal)*100 | eval PlaybackLatencyLessThan2SecSuccessRate=round(SuccessPer, 3)."%" | fields PlaybackNumTotal PlaybackNumFailed PlaybackLatencyLessThan2SecSuccessRate       There's one further thing to try ...     | stats sum(eval(case(tonumber(RequestTime)<2.00,1, true(),0))) as PlaybackNumSuccessful, sum(eval(case(tonumber(RequestTime)>=2.00,1,true(),0))) as PlaybackNumLong, sum(eval(case(isnull(RequestTime),1, true(),0))) as PlaybackNumNull, count(eval(RecordingID)) as PlaybackNumTotal     That will give you  information about whether the RequestTime field is being interpreted incorrectly or not recognized at all.   ... View more