topic Re: missing extraction fields in Splunk Search

missing extraction fields

gersplunk — Sat, 01 Mar 2025 23:57:42 GMT

I've created field extractions in splunkcloud.com, but they don't appear.

Here are my extractions:

settings>fields>field extractions: App: searching & reporting, config source: visible in app, Owner: sc_admin

journal : EXTRACT-destip

Inline

"dest_ip\":\"(?P<destip>[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)\”"

sc_admin

Global | Permissions

Enabled

object should appear: all apps

permissions: apps r/w, sc_admin r/w

journal : EXTRACT-srcip

Inline

"src_ip\":\"(?P<srcip>[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)\”"

sc_admin

App | Permissions

Enabled

object should appear: this app only (search)

permissions: sc_admin r/w

After Add data from a tar.gz file upload,

splunkcloud (login as sc_admin)>search>interesting fields: all fields:all fields doesn't include those fields.

What am I missing?

Btw, if I extract new fields with the same names it objects because they already exist.

Re: missing extraction fields

yuanliu — Sun, 02 Mar 2025 00:53:00 GMT

Forget your extractions. As the code snippet looks exactly like trying to use regex to extract from JSON. Could you clarify whether the full raw event is in JSON? If it is, do not use regex. If JSON is just part of event, the best option is to use extraction to extract the part that is JSON instead of directly extracting information fragment.

Re: missing extraction fields

gersplunk — Sun, 02 Mar 2025 06:33:12 GMT

Thanks for the reply, @yuanliu. Sadly I don't know whether it's actually json; it might be. It's a college assignment, and we just know it's a bunch of data/logs in tar.gz. "src_ip" and the other one have never appeared automatically in interesting fields so far. Would you expect them to appear as their "natural names" if it was json or would I need to do something proactive?

Either way, why doesn't the extracted field appear?

Re: missing extraction fields

livehybrid — Sun, 02 Mar 2025 15:28:11 GMT

Hi @gersplunk

When you search for the data, do you have src_ip or dest_ip in the field list on the left?

You could also add | table *_ip to your search to see if src/dest IP is already an extracted field from the JSON.

If you can post a screenshot and/or sample data then it might help us to work to you getting to the bottom of this 🙂

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

Re: missing extraction fields

isoutamo — Sun, 02 Mar 2025 17:11:09 GMT

You should have raw data from source. Then do needed extraction or use spath if it’s json. Best option is ingest data into your test/dev environment (like your workstation) and when it works copy those into your SCP environment You could/should create app(s) for those KOs to manage those.
As you have SCP in use, you could order dev/test license from splunk to use in your test environment.

Re: missing extraction fields

gersplunk — Tue, 04 Mar 2025 09:24:03 GMT

Nice trick @livehybrid , but no luck.

Here's the field extraction for dest_ip

You can see the preview says '1000 events'... and there's a "dest_ip" at bottom left.

then >Save, >Finish

>Explore the fields I just created in Search:

it has changed time range to last 24h, showing no Results; then I change to All Time

and get the usual result: no sign of my extraction field on the left, in the 105 more fields or in All fields, etc.

Re: missing extraction fields

yuanliu — Wed, 05 Mar 2025 04:24:08 GMT

As several people urged you, please post a complete sample of event, not screen cutouts. You can sanitize the sample any way you like, but keep quotation marks, commas, curly brackets, square brackets in exact place.

Meanwhile, the cutouts give me enough info to determine that part of the event is JSON. Here is an experiment for you.

| rex "^[^{]+(?<only_json>.+})" | spath input=only_json

See if more fields gets out.