topic Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour? in Splunk Search

How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

yogeshv23 — Wed, 24 Jun 2015 02:57:21 GMT

I wanted to know how to write a search that will trigger an alert when it meets the following conditions: During a period of 1 hour, if the same account number is seen in the transaction log with transactions originating from different State codes and the total transaction amount exceeds $10000, then Splunk has to trigger an alert. Can I please get sample search that will do the following?

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

woodcock — Wed, 24 Jun 2015 04:45:50 GMT

Do you have field extractions done? Show us a sample event.

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

lguinn2 — Fri, 26 Jun 2015 07:17:30 GMT

Try this

yoursearchhere
| stats sum(amount) as total values(state_codes) as states by account_number
| where mvcount(states) > 1 AND total > 10000

Save this search as an alert. You can run this as a scheduled alert, once per hour - or as a real-time alert over the past hour. Set the trigger condition to be "number of results > 1".

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

yogeshv23 — Wed, 08 Jul 2015 06:21:04 GMT

Hi,
Below are a couple of lines of the events. I will be doing the field extractions as well.
Kindly help.

2015-06-14 12:24:45,944 [10] DEBUG transaction.service.Transaction - [Transaction Id : 5e7fe57b-5aff-4575-a189-a6344aac2838, Status Code : Success, Latitude : 37.7845763, Longitude : -122.403748, ZIPCode: 94103, State : CA, Account Details : [ AccountNumber : 1111111310902197, AccountName : John Doe 310902197 , Ammount : 2000, Store : Macys ]]
2015-06-14 12:26:47,949 [10] DEBUG transaction.service.Transaction - [Transaction Id : 5e7fe57b-5aff-4575-a189-a6344aac2838, Status Code : Success, Latitude : 40.7033127, Longitude : -73.979681, ZIPCode: 10003, State : NY, Account Details : [ AccountNumber : 1111111310902197, AccountName : John Doe 310902197 , Ammount : 3000, Store : Nordstrom ]]

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

yogeshv23 — Wed, 08 Jul 2015 06:23:00 GMT

Thanks for your search query, I have provided a couple of lines of transaction logs above, please take a look and let me know if you have any suggestion or updates

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

woodcock — Wed, 08 Jul 2015 15:48:06 GMT

This should do it:

... | rex ",\s*State\s*:\s*(?<State>[^,]+).*?AccountNumber\s*:\s*(?<AccountNumber>\d+).*?Ammount\s*:\s*(?<Amount>\d+)" | bucket _time span=1h | stats sum(Amount) as total values(State) as States dc(State) AS numStates BY _time AccountNumber | where numStates > 1 AND Amount > 10000

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

yogeshv23 — Thu, 09 Jul 2015 02:00:29 GMT

Thank you! I will give it a try.

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

yogeshv23 — Thu, 09 Jul 2015 03:53:45 GMT

Thank you so much. It worked!!!

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

yogeshv23 — Thu, 09 Jul 2015 03:57:17 GMT

Thank you so much, this helped me narrow down my search query!

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

yogeshv23 — Sun, 12 Jul 2015 23:59:42 GMT

@woodcock,
I have set up a realtime alert using the query provided but am running into a few issues with alerting. I would like to get an alert when the condition is met for a particular account during a rolling window of time.
Example: if accountnumber 12345 has transactions logged within an hour in 3 different states which totals to greater than $10000 then it needs to just alert once (ideally) and then wait for the condition to be met again before triggering another alert.
How should I set up such an alert? Please guide me.

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

woodcock — Mon, 13 Jul 2015 01:34:06 GMT

Do not use Real-Time; use a reasonable window scheduled at a reasonable period (something like every 10 minutes for the last hour). There are many reasons why and we can discuss them but trust me, this is a bad idea and will not do what you are hoping it will. When you schedule the alert, there are built-in throttling configurations that should meet your need; have you looked at (tried) them? If none will work, then you can output your current conditions with outputlookup and pull them back in for every search with inputlookup and use dynamic-lookup to write your own throttling conditions within your search.

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

yogeshv23 — Mon, 13 Jul 2015 14:22:08 GMT

Thank you. When i try to schedule the alert, the only options I get are for scheduling every hr, day or week.
I was not able to figure out a way to schedule an alert like you have mentioned.

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

woodcock — Mon, 13 Jul 2015 15:50:01 GMT

Choose the cron syntax and use:

*/10 * * * *

Re: How alert if the same account number is seen with transactions from different State codes and the total transaction amount exceeds $10,000 within one hour?

yogeshv23 — Mon, 20 Jul 2015 03:00:47 GMT

Thank you! Worked fine and saved my time from trying to make the real time alert to work.