ES Use case - How to detect an ongoing activity using Transaction ?

[neerajs_81]

Hello,   Is there a way to use transaction command to let us know if an activity/attack is ongoing ?
Scenario :  Create a search that detects ongoing DDOS activity

I have the following search that will detect DOS activity events and track them using transaction.  I see there is a maxspan option available but there is no minspan .   Even if i schedule this to run every 1h, the maxspan will show those results that are less than 1h too.  Since there is no minspan option, how to make it detect an ongoing activity ?  Hope i am clear

My search:

index=arbor ...
| transaction eventID startswith=starting endswith=end maxspan=1h
| eval starttime = _time 
| eval duration = "Ongoing"
| convert ctime(starttime)
| table starttime, duration, condition

Maybe my above approach is wrong. How else can we accomplish this?

 

[ITWhisperer]

Can you not use the eventcount returned with each transaction event to determine if an attack is ongoing?

[neerajs_81]

Ok. Let me check.  Should i check for something like  isnull(EventCount) or isNOTnull(EventCount)  to determine transaction is on going ?

[ITWhisperer]

The transaction command processes the events in the pipeline. What I am suggesting is that the number of events in the transaction might tell you whether there is an attack (within the transaction). To see if it is "ongoing" you could look for the latest timestamp in the transaction and compare it to the current time?

[neerajs_81]

i am trying to do what you suggested, look at the latest timestamp ( as in the last timestamp) in the "transaction" and compare with current time but its not working out. Can you pls advise where i am going wrong in the below search ?

The transaction results have multiple events within each one, and there is a field called datetime which is multi value field and it has values of timestamps of all different events in that transaction.  I am using mvindex to capture the "last" value from this datetime array.  That will give me the last as in latest timestamp of that activity.  

I checked under "Interesting fields" in Splunk ,  the DT field values are correctly showing up.  But the "LastSeenEventTime" is not getting created.  Any suggestions why ?  I am converting DT into epoch time and saving that a LastSeenEventTime . Then i am comparing with now() field to achieve the use case.  

| eval DT =mvindex(datetime,-1)
| eval LastSeenEventTime = strptime(DT, "%m-%d-%Y %H:%M:%S")
| table eventcount logtype, eventID, status, eventType, severity
| where DT = now()

[ITWhisperer]

LastSeenEventTime and DT do not appear in your table command so are not available beyond this point. Could this be your issue?

[neerajs_81]

No, not really. Even if i include them in table command,  the table view returns <empty>
Couple of questions:
1. Is this the right way to compare/check against the current time  -   | where DT = now()

 2. As per my screenshot you will see the datetime field has values in this format:  2022-06-15 16:15:21+08:00

So if i am doing a | eval LastSeenEventTime = strptime(DT, "%Y-%m-%d %H:%M:%S") , is this correct?  The +08:00 is not accounted for in the time format.

[ITWhisperer]

1. Comparing to now() is unlikely to get a hit. The timestamps you are comparing are from the events, which will have been logged, then ingested and indexed, all of which takes time, so they are unlikely to be instant i.e. they won't match now(). You would probably be better considering the difference between now() and DT and see if it is close (by whatever you consider to be close, taking into account the lag time between the event being logged and it being available in the index).

2. For the time format to take timezone into account, you need to add it to parsing string

| eval LastSeenEventTime = strptime(DT, "%Y-%m-%d %H:%M:%S %:z")