it's been a while since I had access to this data (so I can't put a search together for you), but I believe the _introspection index is where you may want to start.  Maybe this other post will be helpful:  What is included in HEC introspection data?  ... View more

speaking of which....in case there are some rogues parens in the message, maybe this is even a bit safer - instead of matching everything... \((?<rule_name>[^\(]+?)(?:-00)?\)$   ... View more

I think you should be able to set it in your search element during the finalized (or done or whatever) phase.   In the splunk docs, these options can be found under  "search event handlers" or something like that.  So pseudo'ish dashboard xml might look like this.    <search> <query> . that . long . search </query> <finalized> <set token="skill_list">$result.nums$</set> </finalized> </search> . . . <row> <panel> <table> <search> <query>index=whatever AND skill IN $skill_list$ .... </query> </search> </table> </panel> </row>   ... View more

not sure if you need to actually use where for your scenario, but if the fields are already extracted, then a subsearch might be better/easier?   index=<your_index> [|inputlookup <your_lookup> | eval search=where_condition]   ... View more

Nothing stands out as wrong with that search.  I'd suggest reviewing the job inspector, maybe the keywords or remotesearch fields (Job -> Inspect Job, expand search properties) - that should give you an idea if the subsearch is working as you expect. I think a lookup is fine approach.  If you have the ability, you could install the Lookup Editor app, which provides an excel-like experience for modifying lookups.  Or you could use the outputlookup command overwrite the lookup.  Typically that would involve using inputlookup to get the events, using where/append/etc to modify the results, then using outputlookup to write it back. ... View more

I just tried with 8.1.3 and wasn't able to reproduce.  Also didn't see anything in the release notes about that. Can you reproduce with a new lookup as a quick test?   can you lookup by ip instead of cidr, just to make sure the lookup works in general?  could there be anything annoying like whitespace in the cidr field?  may be worth diving into props/transforms to ensure nothing got moved/modified/overwritten during the upgrade? ... View more

I'm guessing you are'nt getting responses here because this isn't like a simple 3-step process.  And if somebody did try to answer, this thread would likely become more like a Professional Services (PS) engagement than a forum answer. I won't be able to help other than pointing you to the document below.  If it seems overwhelming, it may be worth engaging PS (if that's an options for you) Splunk Add-on for AWS  ... View more

Not entirely sure if i understand what you're after, but from what I can gather, I think I would go the opposite away - get the field values, then map those into a search of index2.  Something like below maybe?  The regex could change depending on your definition of "substring". Also if you just wanted the events that match in general (and not needing the actual substrings), a simple subsearch may be better, not sure. index=index1 | stats count by field1 | map search="search index=\"index2\" | rex max_match=0 \"(?i)(?<substrings>\w*$field1$\w*)\"" | where isnotnull(substrings) | table substrings, _raw   ... View more

does that happen if you just end the search after the timechart?  did you add the table and fields commands to troubleshoot? ... View more

ok...so I guess you didn't get that working then?  Here is an example of how maybe you could use tstats to get the index, source and sourcetypes.....just not sure if it will be too much for the map function.  But also including the where clause that you need - same as before, just have to escape the quotes.  But I'm still not sure what you tried that didn't work..... Not sure how many combinations of index/source/sourcetype you have, so you may need to bump up the maxsearches as well.  And if it is too much, I think you should know how to go back just using the index. | tstats count where index IN (indexA,indexB,indexC) by index source, sourcetype | map maxsearches=100 search="search index=$index$ source=$source$ sourcetype=$sourcetype$ alan | fieldsummary | where match(values,\"(?i)alan\") | eval index=\"$index$\",source=\"$source$\",sourcetype=\"$sourcetype$\""   ... View more

friday firedrills over here.....can you just toss the where after the map command?  or it might helpful if you can share what you tried that failed.  ... View more

well i did forget to the put the "alan" keyword back in the search - did you put it back in there?  If not, try adding it in the search in the map function.  And also i guess the where clause as well.  all of that needs to back into that search..... ... View more

maybe someone else out here will have a better answer, but i can see maybe getting the index by using the map command....but if you don't know the sources/sourcetypes/etc, then that could get trickier.  But here is maybe the next step for your path. This will create an event for each index and then use the map function to the same search over each, adding a field for index to the results.  I'm sure there is a way with tstats or eventcount to build the indexes/sourcetype you want and just expanding this.  Although, not sure how expensive this becomes either.   | stats count | eval index=split("indexA,indexB,indexC",",") | mvexpand index | map search="search index=$index$ | fieldsummary | eval index=\"$index$\""   ... View more

Ah ok.  I'm not sure how expensive it is, but i'm wondering if fieldsummary might be helpful here?  Something like this maybe?  I'm sure the match would need to be a bit more complicated.... index IN (indexA,indexB,indexC) alan | fieldsummary | where match(values,"(?i)alan")       ... View more

Obviously, you could just add the keyword "alan" to your search to get the results you may want....but those may contain results you're not interested in. i don't know if you would consider it scalable, but typically the approach i think is to normalize the data as you onboard it.  That could be with the Common Information Model app or just on your own.   For example, in the scenario above, for each of the sourcetypes, create a new field called username - either alias existing fields to username or extract/eval/etc username in the case that it's just in the raw data.  Once you have a username field across all sourcetypes, then just add username="alan" to your search. If you find yourself onboarding a new sourcetype that contains a user that you want to search in a similar way, ensure a username field exists or is created. ... View more

you may be able to just swap the order in the coalesce statement - so that the monitoring id will be used if it exists, then id would be checked for next. ... View more

thanks for following up...these types of improvement notes can often help others when they land here with a similar problem. ... View more

This one is a bit tricky I think. Breaking the events up based on the INIT line should be easy enough. Tying those back to the description is tougher. In Splunk, we have to create an event first from the data streaming in. It's not like we're parsing a file in whole, we're processing it as it streams by. And so we can't grab the header, store it and then tag that data to other events....at least no way that i know how. BUT, we can do some things. For example, we can set the entire header aside as its own event and give it a different sourcetype. And then create events for the job actions. Later on when we search the data, we should be able to tie together those two events based on the host and source - since they will match. So at that point, we can sort of glue the description back in. Not sure if it's the best method to do this, but maybe it's one. This quick example assumes the sourcetype set when you ingest the logs is app:job:log and creates one called app:job:header....both of which are easily changeable of course. Indexer/Parse Config (On your indexers) props.conf [app:job:log] SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+)(?=\[[^\]]+\]\s*'INIT') TIME_PREFIX = ^\[ MAX_TIMESTAMP_LOOKUPAHEAD = 20 TRANSFORMS-header_sourcetype = set_app_job_header_sourcetype transforms.conf [set_app_job_header_sourcetype] REGEX = ^\[[^\]]+\]\s*#{5} DEST_KEY = MetaData:Sourcetype FORMAT = sourcetype::app:job:header Search Config (on your search heads) props.conf [app:job:header] REPORT-header_fields = app_job_header_fields transforms.conf [app_job_header_fields] REGEX = \]\s+#\s+([^:]+):\s*([^\r\n]+) FORMAT = $1::$2 And then assuming all that works, a simple sample search might be like index=app sourcetype=app:job:* | eventstats values(Description) as Description by host source | where sourcetype="app:job:log" ... View more

you seem to be tabling out a lot of fields. Can you let us know what the fields/headers would be for the ideal "one row" of data you're looking to get to? ... View more

depending how your search works currently and/or what the current multi-row results look like, you could maybe use chart of xyseries. So if you're currently using stats to format the data already, maybe use chart instead, e.g ... | chart latest(result) over circuit_id by request_id Or if you already have a table with fields of circuit_id, request_id and result, then maybe use xyseries to reformat it to a chart layout. ... | xyseries circuit_id, request_id, result Without seeing the search...not sure if those will help, but worth mentioning i think. ... View more