topic Re: Kafka stream alert in Alerting

Kafka stream alert

Pravinsugi — Sun, 12 Nov 2023 16:29:21 GMT

I have 2 string which need to be searched in splunk both string having different index and different source type.one string is "published sourcing plan " and another string is "published transfer order" .I need to get "published transfer order" log from the splunk.if it's not available after 5 min of getting "published sourcing plan "log in the splunk.i need to count it or need to retrieve some details like salesorderid from "published sourcing order" log .how to prepare sea rch query in splunk.incase none of the log available in the splunk for "transfer order published",I need to capture the things

Re: Kafka stream alert

tscroggins — Sun, 12 Nov 2023 16:58:53 GMT

Hi @Pravinsugi,

Can you provide sample events with sensitive information redacted?

Do you have two event types?

"published sourcing plan"
"published transfer order"

Or do you have four event types?

"published sourcing plan"
"published transfer order"
"published sourcing order"
"transfer order published"

Is the salesorderid field extracted from all event types or only from "published sourcing order?"

Re: Kafka stream alert

Pravinsugi — Sun, 12 Nov 2023 17:13:12 GMT

Yes all event .but some are json format

Re: Kafka stream alert

tscroggins — Sun, 12 Nov 2023 17:15:16 GMT

Can you provide sample events?

Re: Kafka stream alert

Pravinsugi — Sun, 12 Nov 2023 17:38:59 GMT

For example

1) One index and one source type and search string is "hello" "how" "where".here each search string will give common log as "id" and "name"

2) once done with all 3 search string (hello,how,where).within in next 5 min ,one log should present in splunk.

3)that log contain "completed" as string which also have "id" and "name".

4) incase after 5 min,"completed" string is not available in the splunk log,I want to retrieve"id" and "name" from my "hello",how,where string search result .pls help me with search query.

Re: Kafka stream alert

Pravinsugi — Sun, 12 Nov 2023 18:08:08 GMT

Could you please anyone help me on this?

Re: Kafka stream alert

tscroggins — Mon, 13 Nov 2023 00:38:34 GMT

Without actual sample events, this may not match your use case exactly, but it's a starting point.

I've used the following events to test:

2023-11-12 00:00:00 id=1 name=a hello how where
2023-11-12 00:01:00 id=2 name=b hello how where
2023-11-12 00:03:00 id=1 name=a completed
2023-11-12 00:10:00 id=3 name=c hello how where
2023-11-12 00:10:00 id=4 name=d hello how where
2023-11-12 00:14:00 id=3 name=c completed
2023-11-12 00:16:00 id=4 name=d completed

Save the following as an alert, and schedule it to run every minute:

((index=index1 sourcetype=sourcetype1) OR (index=index2 sourcetype=sourcetype2)) earliest=-6m@m latest=@m
| addinfo
| where _time<relative_time(info_max_time, "-5m@m") OR match(_raw, "completed")
| transaction keepevicted=t id name startswith="hello AND how AND where" endswith="completed"
| where (closed_txn==0 AND NOT match(_raw, "completed")) OR duration>300
| table _time id name

You can test the search using hard-coded, 6-minute timespans instead of earliest=-6m@m latest=@m. There will be a >1 minute delay before alerts are triggered, but the 6-minute time range allows us to cover the start time +/- 30 seconds of a sliding 5-minute window.

Using the sample data, alerts would be triggered at the following times:

Execution time: ~2023-11-12 00:07:00 - no completed event
=>

_time	id	name
2023-11-12 00:01:00	2	b

Execution time: ~2023-11-12 00:16:00 - completed event late (>5 minutes)
=>

_time	id	name
2023-11-12 00:10:00	4	d

Re: Kafka stream alert

Pravinsugi — Mon, 13 Nov 2023 02:21:32 GMT

Please help me on below things:

Requirements:

Once 3 events meets, immediately next event should published.if event is not published after 5 min ,need alert.

Example :

We have one customerno.for the customer number ,I have to search whether 3 events meets logs available in the splunk log or not

Ex: index= 1 source type ="abc" "s1 event received" and "s2 event received" and "s3 event received"

When I search above query ,I will be getting like

S1 received for 12345 customer

S2 received for 12345 customer

S3 received for 12345 customer

If for one customer,all 3 event are met,next i want to search "created" message available in the splunk for same customer (12345)

Here "created" message index and source type is different

If "created" message not available for 12345 customer no after 5 min once all 3 events meets,I need alert.pls help on this query.

Re: Kafka stream alert

Pravinsugi — Mon, 13 Nov 2023 03:17:01 GMT

Please help me on below things:

Requirements:

Once 3 events meets, immediately next event should published.if event is not published after 5 min ,need alert.

Example :

We have one customerno.for the customer number ,I have to search whether 3 events meets logs available in the splunk log or not

Ex: index= 1 source type ="abc" "s1 event received" and "s2 event received" and "s3 event received"

When I search above query ,I will be getting like( format as below )

S1 received for 12345 customer,name=abz

S2 received for 12345 customer,name = abz

S3 received for 12345 customer,name =abz

If for one customer,all 3 event are met,next i want to search "created" message available in the splunk for same customer (12345)

Here "created" message index and source type is different

If "created" message not available for 12345 customer no after 5 min once all 3 events meets,I need alert with customer no.pls help on this query..if "created" message available after 5 min also need capture customer number.

Fyi : if we received "created" message in the log ,sample log will be (json format )

Log : created :{"customer no" : "12345",name :"kanunam"} like that.

Please please help me on search query.

Re: Kafka stream alert

Pravinsugi — Mon, 13 Nov 2023 14:00:20 GMT

Could you please replicate and share the search the query in splunk