topic Re: How to create a field on the fly using CASE in Splunk Search

How to create a field on the fly using CASE

NanSplk01 — Wed, 12 Feb 2025 17:26:38 GMT

I have the following values that will go in a field titled StatusMsg:

"Task threw an uncaught and unrecoverable exception"
"Ignoring await stop request for non-present connector"
"Graceful stop of task"
"Failed to start connector"
"Error while starting connector"
"Ignoring error closing connection"
"failed to publish monitoring message"
"Ignoring error closing connection"
"restart failed"|
"disconnected"
"Communications link failure during rollback"
"Exception occurred while closing reporter"
"Connection to node"
"Unexpected exception sending HTTP Request"
"Ignoring stop request for unowned task"
"failed on invocation of onPartitionsAssigned for partitions"
"Ignoring stop request for unowned connector"
"Ignoring await stop request for non-present connector"
"Connection refused"

I am not certain how to do this. This is the base search: index=kafka-np sourcetype="KCON" connName="CCNGBU_*" ERROR=ERROR OR ERROR=WARN

I want to create the field on the fly and have it pick up the appropriate CASE value. I would then put it in a table with host connName StatusMsg

Any assist would be greatly appreciated.

Re: How to create a field on the fly using CASE

richgalloway — Wed, 12 Feb 2025 18:53:11 GMT

The command you're looking for is eval.

index=kafka-np sourcetype="KCON" connName="CCNGBU_*" ERROR=ERROR OR ERROR=WARN | eval StatusMsg = case(<<some expression>>, "Task threw an uncaught and unrecoverable exception", <<some other expression>>, "Ignoring await stop request for non-present connector", ..., <<a different expression>>, "Connection refused", 1==1, "Unknown") | table host connName StatusMsg

The trick is in selecting the appropriate status message. You'll need to key off some field(s) in the results.

Re: How to create a field on the fly using CASE

NanSplk01 — Thu, 13 Feb 2025 16:39:45 GMT

StatusMsg is the field (on the fly field) that I want to be populated by the message so I'm not certain what you mean by

<<some expression>>

So that was why I thought maybe this would be an if then type of query. If StatusMsg="some value" then put that in the table along with the other data. If not, then go to the next status message. So I would want:

Action Host ConnName

"Task through an uncaught..." lx....... CCNBU----

So should this be an if then search?

Re: How to create a field on the fly using CASE

richgalloway — Thu, 13 Feb 2025 16:59:55 GMT

The StatusMsg field is being created on the fly, but it has to come from *somewhere*. The OP has a list of possible messages, but there is no indication of when each is used.

<<some expression>> refers to a Boolean check that decides when to set StatusMsg to a specific string. The expression probably will need to test the values of other fields (perhaps Host and/or ConnName). You know your data better than I do so I can't be more detailed than that.

Re: How to create a field on the fly using CASE

NanSplk01 — Thu, 13 Feb 2025 18:40:39 GMT

So I have been trying to use if statements, but I don't seem to be getting the if statement correct:

index=kafka-np sourcetype="KCON" connName="CCNGBU_*" ERROR=ERROR OR ERROR=WARN Action="restart failed" OR Action="disconnected" OR Action="Task threw an uncaught an unrecoverable exception"
| eval if(Action="restart failed", "restart failed", "OK", Action="disconnected","disconnected","OK", Action="Task threw an uncaught an unrecoverable exception", "ok")
| table Action host connName

I've tried several different formats for the if, but it keeps telling me the if statements are wrong. What am I not seeing here?

Re: How to create a field on the fly using CASE

PickleRick — Thu, 13 Feb 2025 19:07:44 GMT

If this is your literal search, you're not assigning a field correctly with eval.

The eval command must have a destination field name. The if and case commands just return a value. You have to assign this value somewhere.

And you're using if with case syntax.

Re: How to create a field on the fly using CASE

NanSplk01 — Tue, 18 Feb 2025 13:01:39 GMT

Can you provide a sample?

Re: How to create a field on the fly using CASE

PickleRick — Tue, 18 Feb 2025 15:52:05 GMT

The if function works like a ternary ? : operator in C. So the proper syntax for setting a field conditionally is like this:

| eval field=if(something="something","value_when_true","value_when_false")

Re: How to create a field on the fly using CASE

NanSplk01 — Tue, 18 Feb 2025 18:22:28 GMT

I think I'm close, but the error_msg does not display:

index=kafka-np sourcetype="KCON" connName="CCNGBU_*" ERROR=ERROR OR ERROR=WARN
| eval error_msg = case(match(_raw, "Disconnected"), "disconected", match(_raw, "restart failed"), "restart failed", match(_raw, "Failed to start connector"), "failed to start connector")
| dedup host
| table host connName error_msg

Re: How to create a field on the fly using CASE

yuanliu — Wed, 19 Feb 2025 05:30:44 GMT

Get rid of that dedup host. You will see some events with error_msg, some without. I cannot decipher what that dedup is supposed to accomplish, or what real problem you are trying to solve. So, I cannot suggest an alternative. But if you have that dedup and if for each host the last event is NOT a failure or disconnect, you will get no error_msg. Maybe you mean this?

index=kafka-np sourcetype="KCON" connName="CCNGBU_*" ERROR=ERROR OR ERROR=WARN | eval error_msg = case(match(_raw, "Disconnected"), "disconected", match(_raw, "restart failed"), "restart failed", match(_raw, "Failed to start connector"), "failed to start connector") | search error_msg = * | dedup host | table host connName error_msg

Re: How to create a field on the fly using CASE

NanSplk01 — Wed, 19 Feb 2025 12:46:34 GMT

Was able to get it working this way.

index=kafka-np sourcetype="KCON" connName="CCNGBU_*" ERROR!=INFO _raw=*
| eval error_msg = case(match(_raw, "Disconnected"), "disconected",
match(_raw, "restart failed"), "restart failed",
match(_raw, "Failed to start connector"), "failed to start connector")
| search error_msg=*
| dedup connName
| table host connName error_msg ERROR

Re: How to create a field on the fly using CASE

richgalloway — Thu, 20 Feb 2025 05:52:37 GMT

If your problem is resolved, then please click the "Accept as Solution" button to help future readers.