indexed_extractions=json not working for FIFO file

[moneybox]

Hi everyone,

In my inputs.conf I am monitoring a fifo file receiving json events.

Inputs.conf :
[fifo:///tmp/a.fifo]
disabled=0
index=main
sourcetype=json_test

Props.conf :
[json_test]
INDEXED_EXTRACTIONS=JSON
KV_MODE=none
AUTO_KV_JSON=false
SHOULD_LINEMERGE=false
TIME_FORMAT=%s.%6N

I see that all fields are parsed correctly, yet I cannot tstats by none metadata fields.
I then tried to add TRANSFORMS-js=test_js to props.conf and have the following stanza in transforms.conf :

[test_js]
REGEX=\"([a-zA-Z0-9_.]+)\":\"([^"]+)\"
FORMAT=$1::$2
REPEAT_MATCH=true
WRITE_META=true

In this case, I am able to tstats on some json fields. However, this regex does not cover all cases of keys and values of a json line and it seems very redundant to reparse all json fields after INDEXED_EXTRACTIONS=json.

Any ideas on how to solve this issue ?

[harsmarvania57]

Hi,

Can you please provide some sample data (Please mask sensitive data) ?

[moneybox]

Hi,

Sure, here is a sample line that will go into the fifo file:
{"value": "New", "onclick": 123}

"value" and "onclick" will not be available in a tstats command such as :

| tstats count where index=main by value

[harsmarvania57]

For me it is working fine with indexing data directly not with fifo input (On Splunk 7.2.3) . Can you please let us know which version of splunk are you running?

I used only below config in props.conf while on boarding the data on my lab.

props.conf

[mysourcetype]
INDEXED_EXTRACTIONS = JSON

[moneybox]

Hi,

I am using Splunk 7.2.4

Yes the above configuration works for monitor or batch.
It does not work with fifo files though

[harsmarvania57]

Yes, reproduced this issue with [fifo://...] stanza, it is not honoring INDEXED_EXTRACTIONS = JSON and due to that value and onclick are not indexing as index fields and due to that you can't use tstats (Because tstats read data from .tsidx files which contains only indexed fields name and data).

[moneybox]

So is it possible to report a bug for splunk ?
Any idea how to achieve that ?
thanks

[harsmarvania57]

If you have active splunk support entitlement then you can raise case with splunk support and if they will say that this is expected behavior then I'll suggest to provide docs feedback on props.conf docs page to mention this as fifo input stanza doesn't support INDEXED_EXTRACTIONS.

[moneybox]

I am afraid I use the free license for this project.
@harsmarvania57 is it possible for you to open a case ?

Thnk you very much