Getting Data In

Parse log which contain unstructured and inner-json

gilsegev468
Engager

Hey,

I have a problem to parse my data: 

19-04-2021 gil-server-1 {"systemId":"1254", "systemName":"coffe", "message":"hello dor"}

I want to extract the fields before Splunk index the data.  

How to configure the props.conf or the transforms.conf?

Labels (2)
0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @gilsegev468 

There are two ways of fields extraction search-time and index-time extraction ( which means while parsing extract and write to indexer).

In your case a search-time extraction is fine with a combination of inline rex (same can be configured as props.conf inside Search-head)  and spath as inner-json is the one you want to extract fields from.

venkatasri_0-1618903989423.png

| makeresults 
| eval log_data="19-04-2021 gil-server-1 {\"systemId\":\"1254\", \"systemName\":\"coffe\", \"message\":\"hello dor\"}" 
| rex field=log_data "(?<inner_json>\{\".*)" 
| spath input=inner_json 
| table systemId systemName message

If you want to use in-line rex then field=_raw (default), same regex can be configured to source/sourcetype in props.conf for search-time extraction and deploy it to Search Head.

---------------------------------------------

An upvote would be appreciated if it helps!

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...