topic Re: Unexpected results with field values - Splunk Enterprise in Getting Data In

Unexpected results with field values - Splunk Enterprise

akarivaratharaj — Thu, 23 May 2024 09:13:13 GMT

With some of the events, we are facing the unexpected format of the query results. Actually in the raw event there is no issue at all, and each field is showing their own values. But when it is queried and displayed in the statistics section as results, the values of few fields are displaying incorrectly.

Usually the search results show key-values. But with some events, the search results are showing as "fieldname1=fieldname1=value" and in some cases "fieldname1=fieldname3=value".

Example1: Request_id=Request_id=12345

(Expected to be -> "Request_id=12345")

Example2: Parent_id=message_id=456

(Expected to be -> "Parent_id=321")

Example3: Parent_id=category=unknown

(Expected to be -> "Parent_id=321")

Is this related with parser or something else? We are unable to find what could be the issue lying over here.

Could anyone please help us on fixing this issue at the earliest?

Re: Unexpected results with field values - Splunk Enterprise

ITWhisperer — Thu, 23 May 2024 09:17:12 GMT

It looks to like you either have a problem with your data (raw events), your ingest config e.g. transforms.conf or your search query. Unfortunately, since you have shared none of these, it is rather difficult to offer anything more constructive.

Re: Unexpected results with field values - Splunk Enterprise

akarivaratharaj — Thu, 23 May 2024 10:41:08 GMT

If I run a search query, there is no issue with raw events. From the Events tab, everything looks in perfect format and can't say that there is a Data quality issue in the events.

Only when this is visualised from statistics tab I could see this. Also this is happening only with some events in the results set. I have attached the screenshot of the normal results and the results with Data Quality issue.

Expected results with Request Id and other fields.

But what it is displaying (Refer the highlighted rows)

Here is the event of one of the request ids where the key value pair is as expected format

Re: Unexpected results with field values - Splunk Enterprise

ITWhisperer — Thu, 23 May 2024 10:47:49 GMT

What is the search?

Re: Unexpected results with field values - Splunk Enterprise

ITWhisperer — Thu, 23 May 2024 10:48:55 GMT

How are the fields extracted?

Re: Unexpected results with field values - Splunk Enterprise

akarivaratharaj — Thu, 23 May 2024 10:53:35 GMT

I am using just the table command

index=main host=* sourcetype=* source=* | table _time, Request_id, Future_id

Re: Unexpected results with field values - Splunk Enterprise

ITWhisperer — Thu, 23 May 2024 10:54:48 GMT

So it looks like it is to do with how the fields are extracted. Please can you share these details?

Re: Unexpected results with field values - Splunk Enterprise

ITWhisperer — Thu, 23 May 2024 10:58:53 GMT

Also you shared (a picture of) an event which works, but not one which doesn't. Please can you share the raw text of a "failing" event in a code block (rather than a picture) - you can obfuscate any sensitive details as appropriate.

Re: Unexpected results with field values - Splunk Enterprise

akarivaratharaj — Thu, 23 May 2024 11:02:22 GMT

Actually I have shared picture of the raw event of the failed ones only (just masked the confidential fields). They look similar to the other events which work.

Re: Unexpected results with field values - Splunk Enterprise

ITWhisperer — Thu, 23 May 2024 11:14:22 GMT

So it looks like it is to do with how the fields are extracted. Please can you share these details?

Re: Unexpected results with field values - Splunk Enterprise

akarivaratharaj — Thu, 23 May 2024 11:24:32 GMT

I have observed one more thing with these failed events. In the event section, usually at the end of each event, the default fields like host, sourcetype, etc., will be appended and displayed.

Similarly, in addition to those default fields, I could see the Request_ID field is also displayed in that section after each event. In that place I could see the format of Request_ID is in unexpected form.

Please check the below screenshot (After the field CT=1, the section of default fields is shown)

Re: Unexpected results with field values - Splunk Enterprise

ITWhisperer — Thu, 23 May 2024 12:07:04 GMT

This is showing that the fields have been extracted incorrectly.

Yet again I ask if you could please share your configurations which are being used to extract the fields for this sourcetype - this is likely to be where your problem lies, so if you want a resolution, you are going to have to give us more information.

Re: Unexpected results with field values - Splunk Enterprise

akarivaratharaj — Mon, 06 Jan 2025 04:45:59 GMT

This issue is resolved after making few changes to props.conf where the field extraction is set.