Why does mvappend fail in my alert?

[MonkeyK]

I've been trying to write an alert that notifies our SOC when someone tries to obfuscate their command with base64 encoding.  

I used to be able to append a decoded output (using decrypt2) to the command line run doing something like this (very simplified)

|<search for base64 encoded commands>

| decrypt field=encoded atob emit('decrypted')
| fields - encoded
| eval decrypted="decrypted: ".replace(replace(replace(replace(decrypted,"\.\.\.","~&_&~"),"\.","")," "," "),"~&_&~",".")
| eval command_line="command_line: ".command_line
| eval WhatRan=mvappend(command_line," ",decrypted)

works great in my search, but when I alert on it, the field "decrypted" does not get appended to "WhatRan".  I can click on the alert search result and not see the value of "decrypted" in "WhatRan", and then immediately run the search and get the value of "decrypted" in "WhatRan"

What makes this happen?  Can it be corrected?

[bowesmana]

When you look at the alert that ran does the decrypted value have a value at all, but just does not get appended to the WhatRan field?

If it does not have a value, one thought is that the alert does not have permission to run the decrypt command for whatever reason, whereas you do as your user.

As @PickleRick says, though why create a multivalue field?

[MonkeyK]

yes, I checked the field "decrypted" and it does have a value in the alert, just not added to "WhatRan"

As to permissions, not sure how to understand that one.  Current version of the alert that I am evaluating is still private, owned by me.  So I think that it runs with my permissions.

[bowesmana]

Mmm, ok - I'm also not sure how to explain that one - but it seems like it should work.

Perhaps the embedded newline concatenation will work

An additional one I though of is to use concatenate + split to achieve the same MV output, e.g. based on my other post with a, b, c

| eval WhatRan4=split(a."_#_".b."_#_".c, "_#_")

More visually concise, less fragile and split can take any type of string as delimiter, so is unlikely to clash with your real data you are combining.

[PickleRick]

First and foremost - why use mvappend at all? Do you really want to create a multivalued field? Isn't a normal concatenation what you need?

[MonkeyK]

This is a good question.
My reasoning is that tabular data sent to a ticketing system does not read well for an analyst.   

My opinion is that readability is very important, especially for the analyst who may have just been woken up to handle a critical severity alert that I created.  Maybe not this particular alert, but I got to realizing that I need to make alerts present better.  Furthermore as my company started doing service-now integration, the SN alert action actually only allows for one field as alert data (there's a "short description" that is used for the alert subject and a "description" field that actually describes it)

Consequently, I've taken to creating an mvfield called "Alert" that summarizes everything.  I tend to build something like this as the last line of my alerts

eval Alert=mvappend("time seen: "._time,
"hostname: ".hostname,
"user: ".user,
"details: ",details,
" ", " ",
"deep links",
"splunk: ".myInlineBuiltSplunkSearch,
"source system: ".sourceSystemAddressForMoreInfo)

I suppose that I could use straight up concatenation, if I can figure out how to insert a newline via eval. I've just happened to find that mvappend formats consistently and nicely and all references that I've found say to do newlines with an mv field

[bowesmana · Accepted solution]

You can get line feeds into your data with eval, but it's non-intuitive - you may think that either of these could work...

| eval line=printf("%s\n%s\n%s", a, b, c)
| eval line=replace(string_with_colons, ":", "\n")

but no, you have to do one of these rather fragile, similar solutions

| makeresults
| eval a="hello", b=" ", c="World"
``` Use printf with embedded newlines in the format string ```
| eval WhatRan1=printf("%s
%s
%s", a, b, c)
``` Use simple concatenation with embedded newlines ```
| eval WhatRan2=a."
".b."
".c
``` Use replace with embedded newlines in the replacement string ```
| eval WhatRan3=replace(a.":".b.":".c, ":", "
")

where the newline is actually part of the SPL. It makes it difficult to read, but works. but if you add liberal comments before the formatting, you can minimise the fragility.

I like your justification of the presentation - keeping a cranky first line support bod happy at 2am is pretty important 😀

[MonkeyK]

Hadn't seen the printf.  
That's gonna be my goto for formatting alerts going forward

[PickleRick]

OK. That is a valid point but I wouldn't use mvfields for that. If only for presentation... well, makes some sense but it's a rather "unsplunky" way of working with data. Whatever rocks your boat.

mvappend as such works for me great so if all your fields are defined, I see no reason why it shouldn't work.