Knowledge Management

JSON auto extraction not occuring for a single field, `src` but occurring for all other fields

mbrownoutside
Path Finder

Me and another engineer were taking a look at `index=corelight sourcetype=corelight_notice signature="Scan::*"`.

We noticed that `src` was not properly parsed given `kv_mode=auto`.


We've attempted the follwing four course of action:
1. performed an EXTRACT on _raw as : "src":"(?<src>[^"]+)",
2. performed a REPORT as: corelight_notice_src
* with a transform as `"src":"(?<src>[^"]+)",` on _raq
3. perform an EXTRACT on _raw as : \"src\":\"(?<src>[^\"]+)\",
4. * performed a REPORT as: corelight_notice_src
* with a transform as `* \"src\":\"(?<src>[^\"]+)\",`


Note that performing the `| rex field=_raw "\"src\":\"(?<src>[^\"]+)\","` at search time works fine.

We also attempted with `AUTO_KV_JSON = false` with the above tests 3 and 4, which failed.

We also attempted with `AUTO_KV_JSON = false` and `KV_MODE = none` with the above tests 3 and 4, which failed


Note that the following works:

```
index=corelight sourcetype=corelight_notice signature="Scan::*"
| spath output=src path=src
```

When AUTO_KV_JSON=true, then most JSON fields are extracted (except for src).

When AUTO_KV_JSON=true and KV_MODE=json, then most JSON fields are extracted (except for src).

 

Any ideas on what the problem is?

 

```

{"_path":"notice","_system_name":"zEEK01","_write_ts":"2021-11-12T23:22:24.722517Z","ts":"2021-11-12T23:22:24.722517Z","note":"Scan::Address_Scan","msg":"kk: 192.168.0.1 scanned at least 27 unique hosts on ports 443/tcp, 80/tcp in 42m29s","sub":"local","src":"192.168.0.1","peer_descr":"proxy-01","actions":["Notice::ACTION_LOG"],"suppress_for":1,"severity.level":3,"severity.name":"error"}

```


Thanks,

Matt

Labels (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @mbrownoutside,

I see FIELDALIAS for src on TA-CorelightForSplunk props.conf . It is trying to alias id.orig_h AS src.

Could you please try deleting this FIELDALIAS to test? id.orig_h null field may be overwriting the src field as null.

If this reply helps you an upvote is appreciated.
0 Karma

PickleRick
Ultra Champion

Are you sure it's a well-formed json?

The value for "suppress_for" field doesn't seem right.

0 Karma

mbrownoutside
Path Finder

I apologize.  I meant to use "N" as a replacement for a number, but didn't test the JSON.

This is usually a number, and I have corrected the above.

This isn't it, unfortunately.

0 Karma

PickleRick
Ultra Champion

Well, what can I say. "Works for me".

PickleRick_0-1636812084573.png

PickleRick_1-1636812261092.png

But this is my home instance of splunk free so I don't have multiple users and access rights. Are you sure nothing overwrites/blocks search-time parsing settings for your user?

0 Karma

mbrownoutside
Path Finder

Yes `AUTO_KV_JSON` being set to true then this should work without issue.

Very abnormal to me as `rex` is working to hit and extract via a named group match.

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>