Detect events with sequential values

[echalex]

Hi,

I'm trying to discover if a public URL is being misused by analsying the access logs. Basically, the URL contains a base string and an ID number (integer). We would like to see if someone is trying to download more data than we would like by guessing the ID number. Something like this should trigger an alert:

/content/documents/10241
/content/documents/10242
/content/documents/10243
/content/documents/10244

We know the user and the ID is extracted as a field called documentId

[Ayn · Accepted solution]

You could do this with streamstats.

... | streamstats global=f window=2 latest(documentId) as nextdocid, earliest(documentid) as thisdocid by user | where nextdocid-thisdocid=1

[echalex]

Hi, yes that is an interesting idea. Thanks for that. The problem I see with that is it would only check for a sequence of two documentIds, which would give me what I think of as a false positive. I would like to detect it when someone is trying at least five or so URLs in a short period. Probably even more.

[Ayn]

You could increase the window and use values() instead. Something like this:

... | streamstats global=f window=5 list(documentId) as docids by user | where mvindex(docids,0)-mvindex(docids,1)=1 AND mvindex(docids,1)-mvindex(docids,2)=1 AND ...

[echalex]

That actually seems like a good option. Thanks!