What is the scope of the FILLNULL command?

[RickCurry]

A co-worker has a macro that generates a new field TIME by first testing if the field value is null then converts the _time field for its value, like this:

| fillnull value="-"
| eval Time=if(isnull(Time),strftime(_time,"%m/%d/%Y %H:%M:%S %Z"), Time)

I am unclear myself why he is first testing the field being null, but it is his code. Since the Time field is occasionally showing up as "-" when he uses the "Open in search" option on the table in which the data is displayed, I am wondering if the following is happening:

1. Splunk pre-defines the fields as it parses the SPL.
2. fillnull assigns "-" to the Time field because it is defined and being new is set to null.

Other than than unusual way this is coded, I am interested in knowing if the scenario I posted as a possible cause is plausible. So I guess this is more of a question for one of the Splunk developers or Architects that know the guts of it. What say ye to this riddle my friends?

[ddrillic]

It appears that both commands handle the same situation but differently. It makes more sense to have them in the opposite order, handling Time and then the rest of the fields.

[RickCurry]

Good point @ddrillic. For the specific purpose of what my co-worker is wanting to accomplish, the best solution is to simply create the Time field since it does not exist before hand. Thank you for sharing your thoughts, you provide an excellent alternative.

[ddrillic]

You are welcome Rick!

[elliotproebstel · Accepted solution]

The fillnull command makes the most sense if you think about Splunk taking all events in the current result set and making a table out of them. The column headers are the names of every field that is present in at least one of the events in the result set, and the rows are the events themselves. If a particular field occurs in only one event, the rest of the events may not have that field defined at all, which is the same to Splunk as that field being null. However, if a given field name does not occur in any of the events in the current result set, fillnull will not add it to them. So if the query that precedes this snippet sometimes returns events with the field Time, then the fillnull would populate any events missing the Time field with Time="-".

My best guess is that your coworker didn't think through all the implications of this code sequence and likely wanted the second line to function more like a coalesce:

| eval Time=coalesce(Time, strftime(_time,"%m/%d/%Y %H:%M:%S %Z"))

But I'd recommend asking the coworker directly, since I'm just guessing at intentions here.

[RickCurry]

Thank you for your quick reply Elliot, I appreciate it. I get what you are saying and agree completely. It also makes perfect sense this is how it would work.

I suggested to my co-worker that since the Time field does not exist before hand that he simply code it to create it:

| eval Time=strftime(_time,"%m/%d/%Y %H:%M:%S %Z")

Your suggestion would work as well but I tend to lean toward the simplest answers. He wants a Time field so just create it. (8-D)

[niketn]

@RickCurry, you should also consider the purpose of Time field being created. If it is just for formatting/display you should have fieldformat instead of eval. This will change the value for display while retaining the underlying field as epoch time.

| fieldformat _time=strftime(_time,"%m/%d/%Y %H:%M:%S %Z")

For display in various visualizations _time can be directly changed to Time using Chart Label option.

Since this is inside a macro, usage/application should be carefully decided as well. If this macro is applied on raw events it would create a new Time field for all the events, which would be an overhead in case it is being used just for display.

[RickCurry]

@ niketnilay Good points. Thank you for sharing them.

I followed up with my co-worker to see how the Time field was being used beyond the macro and he indicated that it being used for other than display/reporting purposes that it is also used to do some grouping. I suggested he consider the native _time field for that as it might be more effective. He is looking into that.