How to exclude Null Values from field extractions

[dsofoulis]

I am building a TA.

The issue I am having is the log file has a field error="". Even though it is null the error field is still there and causing CIM to tag the logs as error. I am hoping you can help me to only return the error field if there is a value other than null. Also note, I am looking for a way to do this without having to write a regex string as I have the same issue across a bunch of other sourcetypes.

<30>2017:08:27-10:30:12 sophos httpproxy[19742]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="CONNECT" srcip="1.1.1.1" dstip="1.1.1.1" user="" group="" ad_domain="" statuscode="200" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="855" request="0xdffdb" url="https://www.google.com.au/" referer="" error="" authtime="0" dnstime="579003" cattime="288" avscantime="0" fullreqtime="109809548" device="0" auth="0" ua="" exceptions="" category="145" reputation="trusted" categoryname="Search Engines" application="google" app-id="182"

[woodcock]

You would probably be best to strip the null error completely out of the raw event with this on your Indexers:

SEDCMD_remove_empty_error_KVP = "s/\s+error=\"\"//"

[ddrillic]

Similar question at How do I remove a null field?

[dsofoulis]

thanks for sharing, but will only remove the null value when performing a search. I need to this to happen at index time.

[ddrillic]

Oh, maybe @somesoni2 solution can be useful - Is it possible to replace null fields at index-time?

[lfedak_splunk]

Did that solution work @dsofoulis? If so we can close the question.