Splunk 7.2.3: Alternative or solution to table command bug

[_joe]

Hello All,

Has anyone else run into this bug with the table command on Splunk 7.2.3? The table command works just fine unless you mistakenly list a field twice otherwise you have strange and unexpected results.
For me, the issue doesn't happen on Splunk 7.2.5+, but I was wondering if there was a workaround other than upgrading.

Data as expected

| table A B C D
A,B,C,D (header)
a,b,c,d

Using the table command repeating a field usually has no results IF there are few fields

| table A B A D

... no Results ...

| table B B A

... no Results ...

Using the table command incorrectly with more tables tends to eschew results to the right

| table A B B C D E C F
A B C D E F (header)
a,b,b,b,c,c

[woodcock · Accepted solution]

You can use this search to alert you whenever anybody is using table with a duplicated argument:

index=_* sourcetype="audittrail"
| rex "(?ms)search='(?<search>.*)$"
| rex field=search mode=sed "s/', autojoin='1',.*$//"
| rex max_match=0 "(?msi)\|\s*table\s+(?<table_arguments>[^\|]+)($|\S)"
| search table_arguments="*"
| rex field=table_arguments mode=sed "s/', autojoin='1',.*$//"
| table _raw user search table_arguments
| mvexpand table_arguments
| rex field=table_arguments max_match=0 "(?J)\"(?<fieldname1>[^\"]+)\"|(?<fieldname2>\S+)"
| eval fieldnames = mvappend(fieldname1, fieldname2)
| fields - fieldname1 fieldname2
| rex field=fieldnames mode=sed "s/,$//"
| streamstats count AS serial
| eventstats count(fieldnames) AS count_total dc(fieldnames) AS count_distinct BY serial
| where count_distinct < count_total
| eventstats list(fieldnames) AS values_total values(fieldnames) AS values_distinct BY serial
| eval values_total = mvsort(values_total)

[woodcock]

Generally these bugs are caused by errors in the search optimizer code. This code can be disabled on a search-by-search basis by adding this string to the VERY END of your search SPL string:

... | noop search_optimization=false

Alternatively, if you add this to the end, it will add debug to the search.log in the job inspector (don't use on 7.1.0-7.1.6 as it can cause search to crash):

... | noop log_DEBUG=*

For more, see here:
https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Noop

[_joe]

Thanks for your comment. Unfortunately this did NOT work for me on 7.2.3. Personally, I am believing that there is no reliable work around for this (other than to upgrade or educate your users to make sure they don't accidentally add a field twice).

Example:
| stats count by host, sourcetype, _time, index
| table _time host, sourcetype _time index
| noop search_optimization=false

Basically the results show the index field is populated with epoch time values

[_joe]

I wanted to provide the general feedback to everyone that this is related to bug SPL-163831. I was hoping to find an answer other than upgrading so I opened a ticket with Splunk but they were unable to provide a work around.

If you cannot upgrade, obviously your best bet is to educate users to make sure they do not mistakenly list a field twice while using the |head command. In my personal opinion this is still rather devastating because it impacts your user's confidence in the Splunk, especially if they are already using another product.

[evania]

Hi @_joe ,

Did you have a chance to check out any answers? If it worked, please resolve this post by approving it! If your problem is still not solved, keep us updated so that someone else can help you.

Thanks for posting!

[alonsocaio]

I am using version 7.2.6. Repeating fields in table command does not cause any errors, but the output does no repeat the fields in the header.

| table a, b, b, c, d, d, b 
a, b, c, d (header)

I guess that It would be better to not repeat fields in this command, since It will not produce any different results in your output.

[_joe]

@alonsocaio thanks for the feedback. To clear, my concern is uses doing this on accident and then receiving incorrect results.

[codebuilder]

Use fields instead of table.

[richgalloway]

I think the workaround/fix is to not repeat fields in the table command.