About samjenk_2

samjenk_2 · ‎11-19-2021

Thanks for the ticket references, @nunoaragao. I just heard from our Splunk administrator that Splunk has gotten back to him on this matter. With the caveat that your Splunk deployment (and the deployment of others) may be different from ours, they told us that the IOWait thresholds were simply set too aggressively, and that we can either disable the health report like others are doing, or tune the IOWait thresholds in health.conf, as described here: https://docs.splunk.com/Documentation/Splunk/8.2.3/DMC/Configurefeaturemonitoring Disappointingly, it looks like the example health.conf file in Splunk's online documentation [1] doesn't include a description of the iowait feature stanza, but it's reasonably documented in the file $SPLUNK/etc/system/default/health.conf. There are several different metrics tracked in this feature, and in our file, they all have to do with CPU utilization (I would have thought disc I/O). Personally, I'm inclined to monitor CPU utilization with other tools to work out what an typical load is like for these metrics, and adjust the thresholds accordingly. Or maybe disable the IOWait feature, but leave the rest of the health features enabled. [1] https://docs.splunk.com/Documentation/Splunk/8.2.3/Admin/Healthconf#health.conf.example

samjenk_2 · ‎11-10-2021

Thanks to you all for reporting this. We experienced the same issue in our upgrade from 8.0.3 to 8.2.2.1. Our Splunk support contact confirmed they're aware of it, and for now we're hoping for a expeditious resolution rather than disabling reporting on it. Question for @nunoaragao, is there a public page where we can track the status of SPL-213405? I was hoping it was a public bug report, but not being able to find it with Google, I gather that's the ticket number for the case you opened with support?

samjenk_2 · ‎07-09-2019

Were you ever able to improve the speed of this search by replacing transaction with a more efficient function, like stats or streamstats ? If so, could you share? Thanks!

samjenk_2 · ‎07-09-2019

I'm pursuing a related problem in trying to build transactions between 4688 (process created) and 4689 (process terminated) events, so we can know precisely how long processes run. I'm looking into correlating these data and saving them to a summary index so they can be further correlated against other Windows logs using Process_ID, Process_Name, time, and host. Once you have results, do you plan on writing them out as distinct events (one event per process tree) in this manner? Especially for long-lived processes, I would think you'll need a way to maintain state on what child processes have been created.

samjenk_2 · ‎06-11-2019

Doesn't this run up against the limit imposed on the number of results that are returned from a subsearch? By default, limits.conf sets that to 10,000, but it can only be increased to 10,500. As written, won't this search miss data if there are more than 10,000 events of EventCode 5156?

samjenk_2 · ‎11-15-2017

I'm having a similar issue on a search head running 6.6.4, but not on 6.4.7. Can you report your version of Splunk you're using? When searching over large data sets (2.2 million events) and using a | timechart count by <fieldname> clause, 6.4.7 returns a table and chart of the top ten values of <fieldname> plus the rest aggregated as OTHER . This is the expected behavior. Running the same search, Splunk 6.6.4 proceeds normally, graphing the top 10 + OTHER , until just before the search finalizes. When the search finalizes, all of the columns are aggregated into OTHER in both the Statistics and Visualization tabs. This is not expected, and is repeatable on searches returning more than 2.2 million events. Both of these search heads are searching over the same data. When timechart is invoked on 6.6.4 with useother=f , the search returns no results. Fore smaller data sets (fewer than 2.2 million events), 6.6.4 behaves as 6.4.7 does.

samjenk_2 · ‎08-23-2016

Thanks again to maciep and sundareshr for their input. I've gotten a little closer on this issue, this time revisiting the notion of breaking my query into parts. I can get the behavior I want by doing the following: (1) {query for first set} | rename src_ip as first_ip | table sid,first_ip | outputlookup mylookup.csv (2) {query for second set} [| inputlookup mylookup.csv | fields sid] | lookup mylookup.csv sid OUTPUT first_ip This approach is a variation on the one I tried in my first post, but with the subsearch based on the lookup table itself, rather than running (1) as a subsearch. This allows me to make first_ip a field for every event in the second set matching on sid , with which I can then treat as any other field with stats , timechart , and the like. It'd still be nice to be able to do this in one search, though.

samjenk_2 · ‎08-18-2016

Thanks for the suggestion, sundareshr. The query you suggest is about 3 times slower than maciep's, but I like how it rolls up all src_ip values for a given index, sort of like the list() function.

samjenk_2 · ‎08-18-2016

Thank you, maciep, for the suggestion. This search does display the information I want as a chart, so thanks for that. I'm still chasing how to do this with a lookup table so that the first_ip field would be available for piping to stats and other summary commands. Have you been able to run queries on dynamically-generated lookups with 'outputlookup' before?

samjenk_2 · ‎08-10-2016

About my Environment Everything here is run using Splunk 6.4.2. The Problem I need to correlate session IDs and IP addresses between two sets of data. It involves: Finding the session IDs (sid) and source IPs (src_ip) from the first set of data. Finding those same session IDS (sid) in the second set of data, even if they don't match the src_ip from the first set of data. Yielding events from the second dataset with the fields sid, src_ip and first_ip, where sid is the same between both data sets, src_ip is unique to the second data set, and first_ip is the value of the sid's src_ip from the first data set. I've come very close to a resolution using the following pipeline: (a) pull the sid from the first set as a subsearch on the second set (b) create a sid/first_ip pairing in a lookup table "mylookup.csv" (c) perform a lookup on mylookup.csv to match sid to first_ip Here's the search I've tried: (01) {query for second set} [search {query for first set} | rename src_ip as first_ip | table sid,first_ip | outputlookup mylookup.csv | fields sid] | lookup mylookup.csv sid OUTPUT first_ip Search (01) runs fine without the "lookup" clause, in that it returns all of the events from the second data set with the same sid as those in the first. When I run the search as written, though, I get the error: (02) Error in 'lookup' command: The lookup table 'mylookup.csv' does not exist or is not available. What's strange is that I know the lookup must exist, because after running search (01), I can retrieve the table's contents using the following command [1]: (03) | inputlookup mylookup.csv Things I've Tried I tried using mylookup.csv to lookup sid as another field [2], like this: (05) {query} | lookup mylookup.csv sid AS my_sid. And that returned the same does not exists/not available error. I've even tried first running a search that creates mylookup.csv, then running a search to perform a lookup on mylookup.csv, like this: (04a) {query for first set} | rename src_ip as first_ip | table sid,first_ip | outputlookup mylookup.csv (04b) {query for second set} [ search {query for first set} | fields sid ] | lookup mylookup.csv sid OUTPUT first_ip Search (04a) completes, but I still get the same error at (02) when I run the search at (04b). I've checked the "Exploring Splunk" book, my Splunk training material, and answers.splunk.com and haven't found anything else explictly talking about using the lookup table created by outputlookup, just how to create the lookup table. Questions (A) Is there a canonical way of referencing lookups that you've created using outputlookup that I'm missing? Do I need to create a lookup definition for the lookup table I create, or is mylookup.csv sufficient? (B) Is there a better way to perform the kind of correlation I want? I haven't tried the KV Store yet, as I'd like to know that I can use the output of outputlookup first. Thanks! [1] https://answers.splunk.com/answers/144139/how-do-i-search-a-csv-file-created-via-outputlookup.html [2] https://answers.splunk.com/answers/54165/lookup-use-without-lookup-definition.html

Posts	10
Solutions	0
Karma Given	0
Karma Received	3
Member Since	‎08-10-2016

Online Status	Offline
Date Last Visited	‎07-28-2022 05:38 PM

Can't run "lookup" on a lookup table created by "o...

Re: Cannot Disable Health Report Features in 8.2.2

Re: Cannot Disable Health Report Features in 8.2.2

Re: I am trying to write a query that displays how...

Re: Correlating Windows event 4688 logs on New_Pro...

Re: Comparing 2 Windows Event logs together

Re: Splunk removes all items from chart apart from...

Re: Can't run "lookup" on a lookup table created b...

Re: Can't run "lookup" on a lookup table created b...

Re: Can't run "lookup" on a lookup table created b...

Can't run "lookup" on a lookup table created by "o...