Splunk Search
Highlighted

Missing field values in report

Explorer

We have logs in the following format[1]. We created a report with few fields like time, service, operation, method, principle, systemid and count.

But when ever a field is missing in the log, the report is not picking up the log to display it in the report.

[1]

2019-09-18T19:41:20.624+0000 INFO 855 com.l7tech.log.custom.splunk : -4: |https://apiqa.flatball.com/fmpsp2p/service|getReleaseInfo|SOAP|10.88.30.32|p2p_client_esmartct||87d1...; charset=UTF-8

In the above example, the system id is missing and the report didnt pickup this log.

Search used for the report is as below:

| pivot servicestrafficcaapimodel servicesapiqads count(servicesapiqads) AS count SPLITROW _time AS _time PERIOD day SPLITROW service AS service SPLITROW operation AS operation SPLITROW method AS method SPLITROW principal AS principal SPLITROW systemid AS systemid TOP 100 count(servicesapiqads) ROWSUMMARY 0 COLSUMMARY 0 SHOWOTHER 1 | SORT -_time
0 Karma
Highlighted

Re: Missing field values in report

SplunkTrust
SplunkTrust

It looks like the missing field is causing the event to be skipped over in the data model. Check the constraints on the data model to make sure they are correct.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Highlighted

Re: Missing field values in report

Explorer

In the constraints for this data model, I just have the index. SO every event from this index should be in the report.

The issue is the logs that have a missing field determined in the report query is not getting pickedup.

0 Karma
Highlighted

Re: Missing field values in report

SplunkTrust
SplunkTrust
0 Karma
Highlighted

Re: Missing field values in report

Explorer

thank you for the reference @jkat54. I tried adding fillnull with different options but it did not resolve my issue.

 | pivot servicestrafficcaapimodel servicesapiqads count(servicesapiqads) AS count SPLITROW _time AS _time PERIOD day SPLITROW service AS service SPLITROW operation AS operation SPLITROW method AS method SPLITROW principal AS principal SPLITROW systemid AS systemid TOP 100 count(servicesapiqads) ROWSUMMARY 0 COLSUMMARY 0 SHOWOTHER 1 | SORT -_time |  fillnull value=NULL
0 Karma
Highlighted

Re: Missing field values in report

SplunkTrust
SplunkTrust

How about an eval on the datamodels definition instead..

 | eval field=if(isnull(field),"NULL",field)
0 Karma
Highlighted

Re: Missing field values in report

Explorer

Thank you for your answer. I tried to add it in my data model and it works fine. But I couldn't save it there. So this is not reflecting in the report.

my data model

| from datamodel:"servicestrafficcaapimodel.servicesapiqads" | eval systemid=if(isnull(systemid),"NULL",systemid)

I don't have an option to save it. How do I reflect it in the report?

Thank you again

0 Karma
Highlighted

Re: Missing field values in report

SplunkTrust
SplunkTrust

Why can't you save the edits to your data model?

You go to datasets, open the dataset/datamodels, click edit (or manage then edit), click add field, select eval expression from add field drop down, and you add the expression save and rebuild any summaries/access you have.

0 Karma
Highlighted

Re: Missing field values in report

Explorer

I am trying the exact same thing. I put "systemid=if(isnull(systemid),"NULL",systemid)" in eval expression bar and I get the below error while previewing. Name of the field is systemid.

"Error in 'eval' command: Fields cannot be assigned a boolean result. Instead, try if([bool expr], [expr], [expr])."

0 Karma
Highlighted

Re: Missing field values in report

SplunkTrust
SplunkTrust

Maybe it doesn't like "NULL"? Never seen that before.

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.