Knowledge Management

JSON auto extraction not occuring for a single field, `src` but occurring for all other fields

mbrownoutside
Path Finder

Me and another engineer were taking a look at `index=corelight sourcetype=corelight_notice signature="Scan::*"`.

We noticed that `src` was not properly parsed given `kv_mode=auto`.


We've attempted the follwing four course of action:
1. performed an EXTRACT on _raw as : "src":"(?<src>[^"]+)",
2. performed a REPORT as: corelight_notice_src
* with a transform as `"src":"(?<src>[^"]+)",` on _raq
3. perform an EXTRACT on _raw as : \"src\":\"(?<src>[^\"]+)\",
4. * performed a REPORT as: corelight_notice_src
* with a transform as `* \"src\":\"(?<src>[^\"]+)\",`


Note that performing the `| rex field=_raw "\"src\":\"(?<src>[^\"]+)\","` at search time works fine.

We also attempted with `AUTO_KV_JSON = false` with the above tests 3 and 4, which failed.

We also attempted with `AUTO_KV_JSON = false` and `KV_MODE = none` with the above tests 3 and 4, which failed


Note that the following works:

```
index=corelight sourcetype=corelight_notice signature="Scan::*"
| spath output=src path=src
```

When AUTO_KV_JSON=true, then most JSON fields are extracted (except for src).

When AUTO_KV_JSON=true and KV_MODE=json, then most JSON fields are extracted (except for src).

 

Any ideas on what the problem is?

 

```

{"_path":"notice","_system_name":"zEEK01","_write_ts":"2021-11-12T23:22:24.722517Z","ts":"2021-11-12T23:22:24.722517Z","note":"Scan::Address_Scan","msg":"kk: 192.168.0.1 scanned at least 27 unique hosts on ports 443/tcp, 80/tcp in 42m29s","sub":"local","src":"192.168.0.1","peer_descr":"proxy-01","actions":["Notice::ACTION_LOG"],"suppress_for":1,"severity.level":3,"severity.name":"error"}

```


Thanks,

Matt

Labels (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @mbrownoutside,

I see FIELDALIAS for src on TA-CorelightForSplunk props.conf . It is trying to alias id.orig_h AS src.

Could you please try deleting this FIELDALIAS to test? id.orig_h null field may be overwriting the src field as null.

If this reply helps you an upvote is appreciated.
0 Karma

PickleRick
Ultra Champion

Are you sure it's a well-formed json?

The value for "suppress_for" field doesn't seem right.

0 Karma

mbrownoutside
Path Finder

I apologize.  I meant to use "N" as a replacement for a number, but didn't test the JSON.

This is usually a number, and I have corrected the above.

This isn't it, unfortunately.

0 Karma

PickleRick
Ultra Champion

Well, what can I say. "Works for me".

PickleRick_0-1636812084573.png

PickleRick_1-1636812261092.png

But this is my home instance of splunk free so I don't have multiple users and access rights. Are you sure nothing overwrites/blocks search-time parsing settings for your user?

0 Karma

mbrownoutside
Path Finder

Yes `AUTO_KV_JSON` being set to true then this should work without issue.

Very abnormal to me as `rex` is working to hit and extract via a named group match.

0 Karma
Get Updates on the Splunk Community!

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...

Five Subtly Different Ways of Adding Manual Instrumentation in Java

You can find the code of this example on GitHub here. Please feel free to star the repository to keep in ...

New Splunk APM Enhancements Help Troubleshoot Your MySQL and NoSQL Databases Faster

Splunk Observability has two new enhancements to make it quicker and easier to troubleshoot slow or frequently ...