topic Re: ES Use case - How to detect an ongoing activity using Transaction ? in Splunk Search

ES Use case - How to detect an ongoing activity using Transaction ?

neerajs_81 — Tue, 14 Jun 2022 06:26:16 GMT

Hello, Is there a way to use transaction command to let us know if an activity/attack is ongoing ?
Scenario : Create a search that detects ongoing DDOS activity

I have the following search that will detect DOS activity events and track them using transaction. I see there is a maxspan option available but there is no minspan . Even if i schedule this to run every 1h, the maxspan will show those results that are less than 1h too. Since there is no minspan option, how to make it detect an ongoing activity ? Hope i am clear

My search:

Maybe my above approach is wrong. How else can we accomplish this?

Re: ES Use case - How to detect an ongoing activity using Transaction ?

ITWhisperer — Tue, 14 Jun 2022 06:33:23 GMT

Can you not use the eventcount returned with each transaction event to determine if an attack is ongoing?

Re: ES Use case - How to detect an ongoing activity using Transaction ?

neerajs_81 — Tue, 14 Jun 2022 07:08:27 GMT

Ok. Let me check. Should i check for something like isnull(EventCount) or isNOTnull(EventCount) to determine transaction is on going ?

Re: ES Use case - How to detect an ongoing activity using Transaction ?

ITWhisperer — Tue, 14 Jun 2022 07:40:36 GMT

The transaction command processes the events in the pipeline. What I am suggesting is that the number of events in the transaction might tell you whether there is an attack (within the transaction). To see if it is "ongoing" you could look for the latest timestamp in the transaction and compare it to the current time?

Re: ES Use case - How to detect an ongoing activity using Transaction ?

neerajs_81 — Wed, 15 Jun 2022 07:31:12 GMT

i am trying to do what you suggested, look at the latest timestamp ( as in the last timestamp) in the "transaction" and compare with current time but its not working out. Can you pls advise where i am going wrong in the below search ?

The transaction results have multiple events within each one, and there is a field called datetime which is multi value field and it has values of timestamps of all different events in that transaction. I am using mvindex to capture the "last" value from this datetime array. That will give me the last as in latest timestamp of that activity.

I checked under "Interesting fields" in Splunk , the DT field values are correctly showing up. But the "LastSeenEventTime" is not getting created. Any suggestions why ? I am converting DT into epoch time and saving that a LastSeenEventTime . Then i am comparing with now() field to achieve the use case.

| eval DT =mvindex(datetime,-1) | eval LastSeenEventTime = strptime(DT, "%m-%d-%Y %H:%M:%S") | table eventcount logtype, eventID, status, eventType, severity | where DT = now()

Re: ES Use case - How to detect an ongoing activity using Transaction ?

ITWhisperer — Wed, 15 Jun 2022 08:03:47 GMT

LastSeenEventTime and DT do not appear in your table command so are not available beyond this point. Could this be your issue?

Re: ES Use case - How to detect an ongoing activity using Transaction ?

neerajs_81 — Wed, 15 Jun 2022 09:16:26 GMT

No, not really. Even if i include them in table command, the table view returns <empty>
Couple of questions:
1. Is this the right way to compare/check against the current time - | where DT = now()

2. As per my screenshot you will see the datetime field has values in this format: 2022-06-15 16:15:21+08:00

So if i am doing a | eval LastSeenEventTime = strptime(DT, "%Y-%m-%d %H:%M:%S") , is this correct? The +08:00 is not accounted for in the time format.

Re: ES Use case - How to detect an ongoing activity using Transaction ?

ITWhisperer — Wed, 15 Jun 2022 10:05:33 GMT

1. Comparing to now() is unlikely to get a hit. The timestamps you are comparing are from the events, which will have been logged, then ingested and indexed, all of which takes time, so they are unlikely to be instant i.e. they won't match now(). You would probably be better considering the difference between now() and DT and see if it is close (by whatever you consider to be close, taking into account the lag time between the event being logged and it being available in the index).

2. For the time format to take timezone into account, you need to add it to parsing string

| eval LastSeenEventTime = strptime(DT, "%Y-%m-%d %H:%M:%S %:z")