Hi @richgalloway , Can we use general regex to extract all the fields having key value pair. I have installed a custom app on another SH, where we typically install all apps to distribute the load, but it is not reflecting (the configurations are not working) on another SH in ES. What could be the reason? In the custom app, the object owner is not showing any user after installation. How can we change it under all configurations in the UI of SH, we doesn't have backend access. Thanks

First, thank you for giving a clear illustration of input, desired output, and the logic linking the two.  Let me confirm: Are you skipping Joe because IP address is not 1.1.1.1? Assuming this is correct, you are looking for something like   <some index search> transaction IN (Logged, DeleteTable) | stats list(transaction) as transaction min(_time) as logon_time max(_time) as delete_time values(userIP) as userIP by User | where mvindex(transaction, 0) == "Logged" AND mvindex(transaction, -1) LIKE "DeleteTable" AND delete_time < relative_time(logon_time, "+10min") AND userIP == "1.1.1.1" | fieldformat logon_time = strftime(logon_time, "%F %T") | fieldformat delete_time = strftime(delete_time, "%F %T")   Output from your sample data is User transaction logon_time delete_time userIP Alex Logged DeleteTable 2023-11-05 12:05:00 2023-11-05 12:10:00 1.1.1.1 Mike Logged DeleteTable 2023-11-05 12:02:00 2023-11-05 12:06:00 1.1.1.1 This is an emulation you can play with and compare with real data   | makeresults | eval _raw="# Time User Transaction 1 12:01 David Login from 1.1.1.1 2 12:01 Joe Login from 2.2.2.2 3 12:02 Mike Login from 1.1.1.1 4 12:03 David Something else 5 12:05 Alex Login from 1.1.1.1 6 12:06 Mike Something else 7 12:09 Joe Delete table 8 12:10 Alex Delete table 9 12:06 Mike Delete table 10 12:20 David Delete table" | multikv forceheader=1 | eval transaction = case(Transaction LIKE "Login from %", "Logged", Transaction == "Delete table", "DeleteTable", true(), "SomethingElse") | rex field=Transaction "Login from (?<userIP>.+)" | fields - _* linecount Transaction | eval _time = strptime(Time, "%H:%M") | search transaction IN (Logged, DeleteTable) ``` the above emulates <some index search> transaction IN (Logged, DeleteTable) ```  

As already stated, there is no deduplication at ingest or index time. Forwarders that load balance across multiple indexers send to one indexer until some criterium is met (time, volume, or indexer is unavailable) and then sends to the next.  It will send the same event twice only if indexer acknowledgment is in effect and an ACK is not received.  When that happens, one or more events may be duplicated and that must be handled at search time.

interesting, i know there is a search time dedup command, is there no index time command?  Also, when a UF is configured to output to 2 HFs which are configured for autolb, does it actually send the event to both HFs, or does it send to 1st available or whichever responds?  If it sends to both, how does the de-duplication happen?

Hi, not answering a question since you didn't ask one, but it seems like you're asking if you can set-up a user if you cannot reach the GUI on either the CM or DS in a distributed cluster? If so, you might start here:  https://community.splunk.com/t5/Security/How-to-create-a-user-from-the-command-line-and-require-password/m-p/410937 DOCS: https://docs.splunk.com/Documentation/Splunk/9.1.1/Security/ConfigureuserswiththeCLI  

1) eventstats adds the aggregated value (sum in this instance) to each event, stats replaces the events with the aggregated statisitcs 2) No, this is not normally possible - addtotals adds an extra event (row) to the pipeline at the end. The way the pipeline is displayed happens after the total row has been added and there is no way to predict how big the first page of the display is going to be ahead of time.

Hello, I tried your suggestion and it worked successfully. I accepted this as solution. I appreciate your help. Thank you. 1)    If I printed out "total" field, it can give total 140 in each row.    How did eventstats know how to calculate the total of 140, when "stats" command has scoreSum of 20/40/80?          If I played around and used "stats" or "eventstats group by vuln", it didn't work.  Please suggest          | eventstats sum(scoreSum) as total     vuln scoreSum total scoreSum_pct vulnA 20 140 14.3% vulnB 40 140 28.6% vulnC 80 140 57.1% Total_scoreSum 140 420 100% 2)   If there are hundreds of row, there will be multi pages. The "Total_scoreSum" field will appear at the end of the row.        Is there a way to display it on the first page, but at the bottom, not at the top (using sort)? | addcoltotals labelfield =vuln label=Total_scoreSum scoreSum scoreSum_pct Thank you so much

Why did you not include all of that in the OP?  You could have had a solution many hours sooner. index=myindex earliest=-24h ``` Count each hash value ``` | eventstats count by SHA256HashData ``` Find the hash value with the lowest count ``` | eventstats min(count) as minCount ``` Keep the hashes with the lowest count ``` | where count=minCount | collect index=summary