I thought about this one for a while and the elegant solution eluded me (I had some horrible ideas with table/transpose and the running foreach... yuck). But then I had an epiphany 🤣 I was focusing too much on events in their natural order whereas the key to do it properly is sorting. Firstly, you do | sort user + _time So you have batches of events concerning the same user sorted by time. Now we need to find a way to get the time of the previous login. Luckily, Splunk has a nice feature called autoregress which copies a value of a field from previous event(s). So we copy the previous login time (and user to filter out moments in which we switch from one user to another). | autoregress _time as oldtime p=1 | autoregress user as olduser Now we have all the data needed to find our culprits | where user=olduser AND _time-oldtime>90*86400 And voila, you have your logins after long period of inactivity. As a bonus you also have the time of previous login (although it could use some formating 😉 ... View more

Port number has nothing to do with Wireshark showing TLS negotiation or not (at least in the typical configuration unless you did something bad to it 😉 ). Even if it wasn't decoding the TLS setup itself you should at least be able to spot some of the certificate data (like DN or CA name) within the packets being exchanged. If you don't it's probable that no TLS negotiation is taking place. ... View more

Correct me if I'm wrong but this way you'll miss sequences of events like this: - login at day 2 (transaction starts here) - login at day 14 (transaction ends here) - login at day 123 (new transaction ends here) ... ... View more

You're 100% right - spath does manipulation on a higher level so with bigger amounts of dataraw regex manipulation might indeed be faster. But spath is faster to write and might just be quick enough. ... View more

Just because you can do something from a technical point of view doesn't mean it's allowed licensewise.   ... View more

https://docs.splunk.com/Documentation/Splunk/8.2.1/Admin/TypesofSplunklicenses The Enterprise Trial license gives access to all Splunk Enterprise features. The Enterprise Trial license is for standalone, single-instance installations of Splunk Enterprise only. The Enterprise Trial license cannot be stacked with other licenses. The Enterprise Trial license expires 60 days after you install the Splunk Enterprise instance. The Enterprise Trial license allows you to index 500 MB of data per day. If you exceed that limit you receive a license warning. The Enterprise Trial license prevents searching if there are a number of license warnings. To learn about license warnings and violation, see What happens during a license violation? You might want to contact your sales representative for Sales Trial license but it's not obvious if it will do.   ... View more

There is something wrong with data alignment in the data prepare section (before first fields command) - got mixed up in copy-pasting. You could try splitting them manually with a regex after multikv instead of relying on splunk's tabulation.   | makeresults | eval _raw=" 121 E3002189 2021-08-27 12:01:34 249 E1000874 2021-08-27 12:05:21 121 E2000178 2021-08-27 12:27:09" | multikv noheader=t | rex "^(?<User_Id>\S*)\s(?<Error_Code>\S*)\s(?<Error_Time>.*)" | append [ | makeresults | eval _raw="User_Id Location Login_Time Logout_Time 121 P155 2021-08-27 11:54:56 2021-08-27 12:14:19 121 U432 2021-08-27 12:22:16 2021-08-27 12:34:52 249 M127 2021-08-27 12:01:32 2021-08-27 12:35:45 249 J362 2021-08-27 12:38:25 2021-08-27 12:50:11" | multikv noheader=f | rex "^(?<User_Id>.*)\s(?<Location>.*)\s(?<Login_Time>\S+\s\S+)\s(?<Logout_Time>\S+\s\S+)" ]| fields - _raw _time linecount| eval login_period_loc=Location."%".Login_Time."%".Logout_Time| stats values(Error_Time) as errortimes values(login_period_loc) as loginperiodlocs by User_Id | mvexpand loginperiodlocs | mvexpand errortimes | rename errortimes as Error_Time | eval spl=split(loginperiodlocs,"%") | eval Loc=mvindex(spl,0) | eval Start=mvindex(spl,1) | eval End=mvindex(spl,2) | fields - loginperiodlocs spl | where Error_Time>=Start AND Error_Time <= End ... View more

If in doubt, append and count 😁 I have a monstrosity for you: | makeresults | eval _raw="User_Id Error_code Error_Time 121 E3002189 2021-08-27 12:01:34 249 E1000874 2021-08-27 12:05:21 121 E2000178 2021-08-27 12:27:09" | multikv noheader=f | append [ | makeresults | eval _raw="User_Id Location Login_Time Logout_Time 121 P155 2021-08-27 11:54:56 2021-08-27 12:14:19 121 U432 2021-08-27 12:22:16 2021-08-27 12:34:52 249 M127 2021-08-27 12:01:32 2021-08-27 12:35:45 249 J362 2021-08-27 12:38:25 2021-08-27 12:50:11" | multikv noheader=f ] | fields - _raw _time linecount| eval login_period_loc=Location."%".Login_Time."%".Logout_Time| stats values(Error_Time) as errortimes values(login_period_loc) as loginperiodlocs by User_Id | mvexpand loginperiodlocs | mvexpand errortimes | rename errortimes as Error_Time | eval spl=split(loginperiodlocs,"%") | eval Loc=mvindex(spl,0) | eval Start=mvindex(spl,1) | eval End=mvindex(spl,2) | fields - loginperiodlocs spl | where Error_Time>=Start AND Error_Time <= End ... View more

One thing - in case of small ranges it might be negligible but if you want a random number from a big range (like between 0 and 1.5*2^30) performing a modulo on random() will not give you uniform distribution. ... View more

You need two different searches - one to sum count over hosts (and then do addtotals to get total sum) and another one to sum over statuses. That's the simplest solution I think ... View more

Sure. First extract the raw json part into a field, then use spath to extract a value from json. Like that: | rex "(?<jsondata>{.*})" |  spath input=jsondata path="defaultMsg" | spath input=jsondata path="userName"   ... View more

Why not use spath to extract data from the json part? ... View more

Well... I'd say that this measurement wouldn't be that useful. Even if you added timestamps at subsequent forwarders you'd not be able to tell whether you're getting stuck at parsing, output from one forwarder, input into other one. I think it's more useful to monitor the forwarder throuhgput and see if it's not falling below typical values. One other thing - the typical _indextime-_time calculations only make sense if all your components have properly configured time source and produce properly timestamped data. I use it in general not as a measurement of latency in the whole event processing path but rather as a measurement of the source system's clock quality. It's quite typical for windows events to be several secons or even minutes "behind realtime" in indexing events simply because the event acquisition process for windows (especially with WEF) works in batches. Having said that, you probably could use ingest-time eval to add time()-based value to an event (didn't try it myself though!) but remember that you'd need to have indexed field(s) to store values for this time so it gets a little complicated here because you'd need to either have to create multiple fields in order to hold values for several tiers or have to manipulate multivalue fields (which I'm not sure is possible at ingest time and even if it was it'd be horribly inconvenient to use for further processing for stats) ... View more

Oh man, I found a very ugly solution (can't pass a multivalue result fields from subsearch in any other way; if anyone knows a workaround I'd be grateful to know). Let's assume I get my users from a winevents index. To create my list of all the times with users I'd do something like that: | gentimes start=1 end=1000 | fields starttime | eval id=1 | join id [ search index=winevents | stats values(user) as users | fields users | mvexpand users | transpose 0 | eval id=1 ] | fields - id column | foreach row* [eval users=mvappend(users,'<<FIELD>>')] | fields starttime users | mvexpand users Of course at the end you'd have to append it with this additional count field equalling zero and sum with your original search. Yes, I know this solution is very ugly - it contains mv combining/splitting as well as transposition and is horribly ugly overall. But it seems to work 😉 ... View more

If your search is indeed literarily as you wrote it: | stats values(user) as user count by host Then you're getting two different stats values per host - one is a multivalue single field containing all the users for this host, the second one is a sum of all events for this host. That's probably not what you wanted. You might list the values and count it afterwards: | stats values(user) as users by host | eval usercount=mvcount(users) But it's not very pretty Or you might use dc(user) | stats values(user) as users dc(user) as usercount by host ... View more

Why don't you just extract the whole json and use spath? For example | rex field=_raw "(?<json>{.*})" | spath input=json path="unique_appcodes{}." output=some_field Of course if you need to process each entry from unique_appcodes separately further down the stream you'd need to mvexpand ... View more

If you use stats or tstats, you're searching against data which may lead to ommited indexes if you haven't received any events for the time period specified. Yes, you can stretch the timeframe but it still won't help for indexes for which you never received any events. So the easy solution for listing all defined indexes would be to use rest | rest /services/data/indexes | rex field=id ".*/(?<index>[^/]+)$" | fields index   ... View more

Ahhh, so you'd probably have to try other approach like using subsearch | stats values or something similar. ... View more

OK. It wasn't clear from the original posting what you're doing with the users (there were uniqe time values so I assumed the users were non-overlapping). If you mean that for every user you can have a separate sequence of dates, instead of | eval user="root" you can do | users=mvappend("user1","user2","user3"...) | mvexpand users (of course you'd add | eval fail_num=0 after that) Then at the end you'd stats not by _time but by _time and user. Is this more what you need?   ... View more

So, in general, it would be best to remove unneeded fields as soon as possible, right? (of course it's always a trade-off between performance now and - for example - flexibility to modify your search later) ... View more