Splunk Search

Why is my search on JSON data producing duplicate results for each line, except for the date and time?

rafamss
Contributor

Hi guys,

I have a problem. Every time I try to run the following search, the result is duplicated in each line, but the date and time. What can be? My log is in format JSON.

index="my_index" source="my_source" sourcetype="my_sourcetype"
| rename field1 , field2, field3, ....
| eval Date = strftime(_time, "%d-%m-%Y")
| eval Hour = strftime(_time, "%H-%M-%S")
| spath output=Rules  path=field.sub-field{}.code
| table Date, Hour, field1 , field2, field3, ....

alt text

0 Karma
1 Solution

landen99
Motivator

The data is most likely being indexed with indexed extractions and also searched with search-time extractions. Your indexer props.conf file likely has a line to do indexed extractions (not best practice):
INDEXED_EXTRACTIONS = json
On the search head props.conf:
KV_MODE = json

Deleting "INDEXED_EXTRACTIONS = json" from the indexer will solve the issue for everything indexed after the indexer is restarted. Anything already indexed will retain their indexed extractions, so you might want to delete the indexed data and re-index everything.

View solution in original post

landen99
Motivator

The data is most likely being indexed with indexed extractions and also searched with search-time extractions. Your indexer props.conf file likely has a line to do indexed extractions (not best practice):
INDEXED_EXTRACTIONS = json
On the search head props.conf:
KV_MODE = json

Deleting "INDEXED_EXTRACTIONS = json" from the indexer will solve the issue for everything indexed after the indexer is restarted. Anything already indexed will retain their indexed extractions, so you might want to delete the indexed data and re-index everything.

landen99
Motivator

Or index the field and set KVMODE to none.

0 Karma

landen99
Motivator

The other solution is to leave it as an indexed field and add KVMODE=false to the search head.

0 Karma

rafamss
Contributor

landen,

This occurs every time that I try use the command table. I changed the configuration but continuous similarity. Do you have any ideia ?

0 Karma

woodcock
Esteemed Legend

The table command just accentuates the fact that the fields have multiple/duplicated values. If you do not use table and then click on the field in the left fields panel, you will see that it is also duplicated there, even though you are not using table. This answer is probably the correct explanation.

0 Karma

rafamss
Contributor

Thanks guys. The solution works fine.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...