Splunk Search

Strange relationship between fieldformat command and postprocess

SplunkTrust
SplunkTrust

The fieldformat command ( http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Fieldformat ) offers a way to affect how a certain field in your search results renders, without actually modifying the field value when it's exported or piped to outputcsv etc.

However it does some weird things with postprocess and I'm curious if anyone has run into this or figured out the root cause. Here's an example:

index=_internal group=per_sourcetype_thruput series=splunkd* | stats count by series | fieldformat count=if(series="splunkd_access","OH HAI",count)

This produces two fields, series and count. The fieldformat command inspects the series field and if the series is "splunkd", it rewrites the count field to "OH HAI".

If you send a postprocess request of "fields series count" to this job, you'd expect to get back the same results, and indeed you do.

But if you send a postprocess request of just "fields count", you'd still expect the "OH HAI" logic to run, ie you'd still expect the count corresponding to splunkd to have been rewritten to "OH HAI", but it isn't. The "OH HAI" logic actually fails completely. as though the series field wasn't defined in the underlying job, and splunkd's count goes back to being the count number.

One explanation is if the fieldformat command acts as a flag for splunk to "sneak in" some postprocess syntax later on certain kinds of requests, and that it does so by tacking its secret "fieldformat postprocess" string onto the end of any user-supplied postprocess string. This would mean our postprocess had already removed the "series" field, and thus the supossed "fieldformat postprocess" couldn't see it.

Has anyone seen this? I ran into this in a more complicated scenario in one of our apps and it was baffling. I'd love to hear if anyone has found the root cause, found any workarounds, or knows how fieldformat is implemented in Splunk.

Esteemed Legend

This makes perfect sense to me; you have not carried over the payload that allows the fieldformat command to function so it got thrown away (I would expect that there is a note in the job logs about it). I have always found fieldformat to be consistent and useful. The only confusing thing about it is that it can be used in 2 different modes: aligned or split (<- These are my names).

By aligned, I mean that the source (right-hand-side) and destination (left-hand-side) field names are the same, such as what is automatically done by Splunk for the _time field. We do not see it but at the beginning of every search is this clause: | fieldformat _time=strftime(_time, "en_US or en_GB or whatever format string here") so there is only 1 field.

By split, I mean that the source (right-hand-side) and destination (left-hand-side) field names are different, such as `| fieldformat time=strftime(_time, "%Y/%m/%d %H:%M:%S").

These 2 do behave a bit differently.

0 Karma

Legend

IMO fieldformat is the unloved stepchild of Splunk commands. My suggestion is to avoid it as much as possible. It is a great idea to separate the value of a field from its displayed representation - but fieldformat doesn't do a very good job of this. I have investigated its behavior for years and across versions and the nicest comment I can make is "it isn't consistent."

In dashboards, if you possibly can, use the new numeric formatting; it's nicer, it's efficient, and it works well. Otherwise, I would use eval.

SplunkTrust
SplunkTrust

Thanks Lisa. I've run into a couple weird things that made me back away and use eval but I'd never really sunk my teeth into any like this one. Didn't know the simple xml dashboards had their own field formatting layer. As you might guess, these are all in Sideview XML dashboards, so I'm out of luck. Of course what I can easily do is actually use postProcess itself - it's a pretty easy conversion. Frankly, the entire space of fieldformat is so well contained within the space of what postprocess can do, I feel like fieldformat is the result of someone saying "what if we could pack all the postprocess functionality into the core SPL + API somehow". (Except that of course postprocess is useful for far more than just reformatting field values. )

0 Karma

SplunkTrust
SplunkTrust

@sideview... While your query is around interpretation of fieldformat behavior, my choice for fieldformat vs eval is a bit different. Only when I want to change the display of a field without having to create a new field, I use fieldformat. For example | fieldformat _time=strftime(_time,"%c"). This way when I put _time in table it displays as String Time and when I use in calculations it works as epoch time. This prevents creating a new field just for display and copying the values to two fields.

If I plan to use the formatted value afterwards, then I use eval instead which will leave me with two separate fields. I know this is not something you are looking for, but I am also interested in trying out the scenario you have described.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

SplunkTrust
SplunkTrust

Yes, that applies here, I've just rewritten my use case so as to greatly simplify it, and to provide a SPL example anyone could run locally. What I'm actually doing is rendering a table that calculates various "duration" fields, and giving the user a field picker. All end-users want all the duration fields formatted as "00:01:39", in the table but an important set of users also want to be able to click "export as CSV", and then get any duration fields as integer numbers of seconds. (for example some of them just want to dump the entire table with all fields into Tableau). The same requirements can be easily achieved with a postprocess search fortunately so I'll just change to evals and pull them into a postprocess. Thank you.

0 Karma