Map command doesn't show results for some fields, even though log shows values parsed out correctly?

[patricknguyen]

I have created a search in order to:

1. Pull traffic log from datamodel "DM_1"
2. Use src_ip and dest_ip as token to pass map search in a different index= with src=$src_ip$, answer=$dest_ip$ so that I can pull the domain (URL) name for dest_ip.
3. Then, use eval within the map search to pass in values for all the fields from first search that don't exist in 2nd search.

4. Final results would include all the fields from 1st search and domain field from 2nd search, with map command.

   | tstats count from datamodel=

where

earliest="05/28/2020:13:32:00" latest="05/28/2020:13:33:00"
   log.flags=decrypted
   log.log_subtype=deny
by log.src_ip log.dest_ip log.transport log.app log.dest_port log.src_zone log.flags log.log_subtype _time log.session_end_reason log.rule log.action log.packets_in log.packets_out
| rename log.* AS *
| map search="search index=  answer=$dest_ip$ src=$src_ip$  | eval dest_ip=$dest_ip$ | eval dest_port=$dest_port$ | eval rule=$rule$ | eval map.transport=$transport$ | eval session_end_reason=$session_end_reason$ | eval action=$action$ | eval app=$app$ | eval flags=$flags$ | eval log_subtype=$log_subtype$ | eval packets_in=$packets_in$ | eval packets_out=$packets_out$ | eval time=$_time$"
| eval time = strftime(_time, "%m/%d/%Y:%H:%M:%S") 
| stats values(domain) as domain, values(dest_ip) as dest_ip, values(dest_port) as dest_port, values(rule) as rule, values(packets_out) as packets_out, values(packets_in) as packets_in, values(session_end_reason) as session_end_reason, values(time) as time, values(action) as action, values(flags) as flags, values(log_subtype) as log_subtype, values(transport) as transport, values(app) as app by src_ip

The problem I run into is only some of the fields from the first search that do not exist in the 2nd search index ( index_2 ) would return the value ( packets_out, packets_in, session_end_reason, rule, dest_port ) while others don't return any value ( flags, app, action, map.transport).

From the job log, I could see all those values were parsed out with correct values:

...snip...

06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval time='_time'
06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval packets_out=8
06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval packets_in=5
06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval log_subtype=deny
06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval flags=nat                           <<<<<   value was parsed out correctly but was missing from output
06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval app=ssl                             <<<<<   value was parsed out correctly but was missing from output
06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval action=allowed
06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval session_end_reason="policy-deny"
06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval "map.transport"=tcp
06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval rule="PERMIT-WEB"
06-02-2020 16:31:26.054 INFO  SearchParser - PARSING:  | eval dest_port=443
06-02-2020 16:31:26.055 INFO  SearchParser - PARSING:  | eval dest_ip="y.y.y.y" 
06-02-2020 16:31:26.055 INFO  SearchParser - PARSING: | search (index=index_2 earliest=05/28/2020:13:32:00 latest=05/28/2020:13:33:00 answer="y.y.y.y" src="x.x.x.x")

...snip...

However, at the end it has some waring error to say some of the fields were missing value from the results:

….snip…

06-02-2020 16:31:27.151 WARN StatsProcessor - Specified field(s) missing from results: 'action', 'app', 'flags', 'log_subtype', 'transport' ...snip...

I don't see the different between missing fields with others and wonder why there is such inconsistent output.

Thanks!