If you edit your earlier answer to correct the syntax, I'll be able to mark it as the solution... ... View more

Oh, I see... But should not it be $result.Level$ -- that is, singular "result", not plural "results"? Thanks! ... View more

Sigh... Yeah, this was an "interesting idea" four years ago... ... View more

But it is not a "personal decision". The "scripted input" means, UF will not be monitoring the files by itself -- it will rely on the periodically spawn script to look for them, and process anything discovered. Or am I mistaken? And, if it is invoked periodically -- rather than once for each core -- then the script must process all cores discovered, in one go... And keep track. Yes, I know, how to keep track of files, but UF already has this tracking implemented -- for the ordinary logs (Splunk's bread and butter), it keeps track not only of each file, but of the position within each file. I'd much rather rely on that mechanism, than reimplement it... The scripted input solution will use the custom script to both, detect core-dumps and process them. I'd much prefer to have to implement only the latter part -- the processing, not the detection... So, can something like the below be added as an input:   [monitor:///my/application/directory/core.*] disabled=0 source_type=coredump process_with=/my/scripts/core2json   It is the last line of the hypothetical input, that I'm inquiring about... Can I ask UF to invoke a program instead of attempting to parse the file on its own? The UF will not need to know, what the script did. It just need to keep track of for which files the script has been invoked already -- and whether the invocation has been successful (exited with code 0). (For some reason, I cannot even find the full reference of the inputs.conf syntax 😞 Only examples -- but not the full list of available verbs...) ... View more

8 years later, the link is broken 😞 ... View more

But this starter_script.sh will still be invoked periodically -- and expected to process all discovered core-dumps in one invocation, rather than once for each core, will it not? Which means, I'll have to keep track of which cores have been processed already -- something, I'd rather leave to the UF itself... ... View more

Could I trouble you for an example of such an input? The only documentation for "scripted input", that I see, uses the script to poll a database -- rather than to process a file matching the specified pattern... Other examples all invoke the specified script periodically -- which means, it would have to process all of the discovered core-dumps in my case. I'd prefer "atomic" operation -- with core-dump detection being done by the UF itself, and the script invoked once for each detected file... Is that possible? Thank you! ... View more

Thanks! Would it make sense to turn the text-blob output by the gdb into JSON? Will UF then notice, that it is JSON -- or do mark the "input" as such somehow? So that, for example, each function listed on the stack becomes its own member of the "stack" array? ... View more

Here is the sample: 12/21/21 7:43:43.000 PM {    logger: connectivity.runner.Runner    message: Managed service successfully started    severity: INFO    thread: main    time: 1640133823.948 }   host = rt22a10031pv00 source = PROD sourcetype = log4j2 12/21/21 7:42:02.000 PM {    logger: connectivity.core.shutdown.GracefulShutdownStrategy    message: Application is closing. Running commands will end with partial results.    severity: WARN    thread: Thread-6    time: 1640133722.085 }   host =rt22a10031pv00 source = PROD sourcetype = log4j2 ... View more

@isoutamo wrote: have you tried this? Sorry, I don't see, how it would help me -- my problem is not, that only a substring is matched... Our log-entries are structured (JSON) and, consequently, the search-string contains quotes of its own. ... View more

@gr0undzer0, no, that's not, what I meant... The downtimes are not scheduled (well, not precisely scheduled), but the application always logs something like "Ok, I'm shutting down", when it is being shut down, and "Started successfully", when it finishes starting back up later. I'd like my alert to ignore any and all messages logged in between those two. I know, messages can be grouped -- with transaction -- and there are examples for charting how long something took by substracting the start- from the end-timestamp. ... View more

@vsingla1 wrote: After grouping the fields into one list, how do I make this list comma separated? This is, what I have somewhere already -- the field Mnemonic (singular), specific to every event, is grouped into Mnemonics (plural), which is then passed to multi-value join: | eventstats values(Mnemonic) as Mnemonics | eval Mnemonics=mvjoin(Mnemonics, ",")   ... View more

Just wanted to add, that those, who want all of their fields to be grouped, can use the asterisk -- instead of painstakingly enumerating them all (and then re-enumerating, when the field-set changes). This works for all regular fields -- but not for the special ones (like _time), those still must be listed explicitly: | stats values(*), values(_time), values(_raw) by eventID   ... View more

This is, what I ended up using -- thanks to @gcusello for the stats ... BY host idea: a search for normal events | fields host, _time | stats max(_time) AS most_recent by host | where most_recent < relative_time(now(), "-5h") | eval most_recent = strftime(most_recent, "%F %T") The above performs whatever search you typically use, then looks for hosts, that haven't reported any search-satisfying matches within the specified time (5 hours in the above example). The search time-range is set by the usual time-picker, which should, obviously, include the alert time. (The relative_time call can, probably, be expressed nicer, but this works.) ... View more

Thanks, but I don't have the access necessary to install new apps... ... View more

Thanks for the ideas, but why do I need to create a lookup? The hosts are already known to Splunk -- all those, that have reported in the last, say, 30 days, but have not reported in the last 5 hours. ... View more

Ok, there are two options here, which can also be combined: Set the includeMDC parameter to true and hope (or provide for), the MDC in your case contains all of the fields you need — they will be in the properties sub-dictionary of every logged event. Set the messageFormat parameter to json -- and format your message to be in proper JSON itself: value="{"cat": "meow", "message": "%m"}"/> The message field of the submitted event will then itself be a dictionary. In the above example, that sub-dictionary will contain two fields: cat and message . These can be searched for on Splunk-server as message.cat="meow" . I'd still like to be able to add additional fields next to the message and the severity , though -- not inside a sub-dictionary... ... View more