Splunk Search

Missing field values in report

pdantuuri0411
Explorer

We have logs in the following format[1]. We created a report with few fields like time, service, operation, method, principle, systemid and count.

But when ever a field is missing in the log, the report is not picking up the log to display it in the report.

[1]

2019-09-18T19:41:20.624+0000 INFO 855 com.l7tech.log.custom.splunk : -4: |https://apiqa.flatball.com/fmpsp2p/service|getReleaseInfo|SOAP|10.88.30.32|p2p_client_esmartct||87d1...; charset=UTF-8

In the above example, the system id is missing and the report didnt pickup this log.

Search used for the report is as below:

| pivot servicestrafficcaapimodel servicesapiqads count(servicesapiqads) AS count SPLITROW _time AS _time PERIOD day SPLITROW service AS service SPLITROW operation AS operation SPLITROW method AS method SPLITROW principal AS principal SPLITROW systemid AS systemid TOP 100 count(servicesapiqads) ROWSUMMARY 0 COLSUMMARY 0 SHOWOTHER 1 | SORT -_time
0 Karma

jkat54
SplunkTrust
SplunkTrust
0 Karma

pdantuuri0411
Explorer

thank you for the reference @jkat54. I tried adding fillnull with different options but it did not resolve my issue.

 | pivot servicestrafficcaapimodel servicesapiqads count(servicesapiqads) AS count SPLITROW _time AS _time PERIOD day SPLITROW service AS service SPLITROW operation AS operation SPLITROW method AS method SPLITROW principal AS principal SPLITROW systemid AS systemid TOP 100 count(servicesapiqads) ROWSUMMARY 0 COLSUMMARY 0 SHOWOTHER 1 | SORT -_time |  fillnull value=NULL
0 Karma

jkat54
SplunkTrust
SplunkTrust

How about an eval on the datamodels definition instead..

 | eval field=if(isnull(field),"NULL",field)
0 Karma

pdantuuri0411
Explorer

Thank you for your answer. I tried to add it in my data model and it works fine. But I couldn't save it there. So this is not reflecting in the report.

my data model

| from datamodel:"servicestrafficcaapimodel.servicesapiqads" | eval systemid=if(isnull(systemid),"NULL",systemid)

I don't have an option to save it. How do I reflect it in the report?

Thank you again

0 Karma

jkat54
SplunkTrust
SplunkTrust

Why can't you save the edits to your data model?

You go to datasets, open the dataset/datamodels, click edit (or manage then edit), click add field, select eval expression from add field drop down, and you add the expression save and rebuild any summaries/access you have.

0 Karma

pdantuuri0411
Explorer

I am trying the exact same thing. I put "systemid=if(isnull(systemid),"NULL",systemid)" in eval expression bar and I get the below error while previewing. Name of the field is systemid.

"Error in 'eval' command: Fields cannot be assigned a boolean result. Instead, try if([bool expr], [expr], [expr])."

0 Karma

jkat54
SplunkTrust
SplunkTrust

Maybe it doesn't like "NULL"? Never seen that before.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

It looks like the missing field is causing the event to be skipped over in the data model. Check the constraints on the data model to make sure they are correct.

---
If this reply helps you, Karma would be appreciated.
0 Karma

pdantuuri0411
Explorer

In the constraints for this data model, I just have the index. SO every event from this index should be in the report.

The issue is the logs that have a missing field determined in the report query is not getting pickedup.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...