topic Re: Alert for filename with previous date in Alerting

Alert for filename with previous date

vm_molson — Wed, 06 Jan 2021 16:05:50 GMT

I would like to write an alert that runs Tuesday through Saturday, and looks for files that have been dropped off on our FTP server with the previous day in the filename and fires when four of those files have been found.

Example, on 1/6/2021, I want my alert to tell me when 4 files have been delivered matching a filename of xxx_20210105_yyy (where xxx_ and _yyy can be anything). Those files are typically delivered in the morning hours, but possibly have come in as early as 11:30 PM the night before.

What is the best way to do this?

Re: Alert for filename with previous date

scelikok — Thu, 07 Jan 2021 17:48:30 GMT

Hi @vm_molson,

You can use below query to filter events that matches your criteria, schedule the alert like every 5 minutes and set alert condition as result count more than 3.

| your search that has "filename" field contains ftp filenames | eval yesterday=now()-86400 | eval yesterday=strftime(yesterday,"%Y%m%d") | where LIKE (filename, "%_".'yesterday'."_%")

If this reply helps you an upvote is appreciated.

Re: Alert for filename with previous date

vm_molson — Mon, 11 Jan 2021 19:56:47 GMT

Thank you! Will try this out

Re: Alert for filename with previous date

vm_molson — Thu, 14 Jan 2021 20:33:52 GMT

Hey @scelikok

... thank you for the tip! It seems to work pretty well, but 3 out of the 4 filenames are like:

blahblahblah_DATE_blahblahblah

.. and only one of the filenames is like

blahblahblah.DATE.blahblahblah

The exact syntax you suggested works to get the file with the periods in the filename, but not with the underscores. I've tried various permutations and I know I am missing something.. but any thoughts about how I might get the others as well?

Re: Alert for filename with previous date

scelikok — Thu, 14 Jan 2021 21:00:24 GMT

Hi @vm_molson,

In fact it should have been work with only underscores but you can try below two options;

Only blahblahblah_DATE_blahblahblah OR blahblahblah.DATE.blahblahblah

| eval yesterday=now()-86400 | eval yesterday=strftime(yesterday,"%Y%m%d") | where match(filename, "[_\.]".'yesterday'."[_\.]")

Any separator before and after DATE

| eval yesterday=now()-86400 | eval yesterday=strftime(yesterday,"%Y%m%d") | where match(filename, "\w+.".'yesterday'.".\w+")

If this reply helps you an upvote is appreciated.

Re: Alert for filename with previous date

vm_molson — Wed, 27 Jan 2021 16:56:26 GMT

Appreciate you sticking with me, @scelikok ! I actually am still not able to make it work. Let me specify the exact file names I should be getting... maybe I missed something in my explanation? Maybe it is that there is no character directly after the date on three of them? I had assumed a space would count as any character, but I'm not as up on my RegEx as I should be.

The log entries look like:

...created /Companyname_POS_20210126 - ...

...created /Companyname_TRN_20210126 - ...

...created /Companyname_TRD_20210126 - ...

...created /COMPANYIDTAXLOT.20210126.230053 - ...

Re: Alert for filename with previous date

scelikok — Wed, 27 Jan 2021 19:24:22 GMT

Hi @vm_molson,

For sure the problem is because of space. In regex, all options should be defined specifically in order to lower false positive matches. Based on your sample events below should work;

| eval yesterday=now()-86400 | eval yesterday=strftime(yesterday,"%Y%m%d") | where match(filename, "[_\.]".'yesterday'."[_\.\s]")

Or if you sure there is no other date in the file you can use below;

| eval yesterday=now()-86400 | eval yesterday=strftime(yesterday,"%Y%m%d") | where match(filename, ".+".'yesterday'.".+")