Splunk Search

How to extract field name containing as table?

msg4sunil
Path Finder

Hello All,

I have data like below.  How do I extract the field names like prefix:field1, prefix:field2, prefix:field3 in tablular fashion.  Extract all those fields containing the word, "prefix:" in it.

"prefix:field1":"value1","prefix:field2":value2,"prefix:field3":value3,

Expect result

prefix:field1

prefix:field2

prefix:field3

Thank you

Labels (2)
Tags (2)
0 Karma

yuanliu
SplunkTrust
SplunkTrust

Use extract, (aka kv) e.g.,

 

| kv pairdelim=",", kvdelim=":"

 

You get 

_rawprefix_field1prefix_field2prefix_field3
"prefix:field1":"value1","prefix:field2":value2,"prefix:field3":"value3"value1value2value3

Is this what you need?

Tags (1)
0 Karma

msg4sunil
Path Finder

Its not always in this pattern("prefix:field1":"value1","prefix:field2":value2,"prefix:field3":value3,) and rather be  more complex structure as well(could be "prefix:field1":"ABC","TxnMsg":{"prefix:field2":XYZ,"prefix:field3":123},).  Is there any other way?

thank you

 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

The example you give looks like a fragment of a JSON object.  Is your raw data or a part of that data conformant JSON?  If so, use spath instead.  For example, if _raw is 

{
    "prefix:field1": "ABC",
    "TxnMsg": {
        "prefix:field2": "XYZ",
        "prefix:field3": 123
    }
}

spath gives

TxnMsg.prefix:field2TxnMsg.prefix:field3_rawprefix:field1
XYZ123{"prefix:field1":"ABC","TxnMsg":{"prefix:field2":"XYZ","prefix:field3":123}}ABC
Tags (1)
0 Karma

msg4sunil
Path Finder

Yes, part of the data is JSON and not the entire _raw. Isn't there a way to look for String matching "prefix:.*" criteria and extract the complete matched string? thank you

0 Karma

yuanliu
SplunkTrust
SplunkTrust

You should focus on extracting that conformant part into its own field.  Suppose you have a field data  that contains conformant JSON, you can do

| spath input=data

Trying to manipulate structured data as text is labored and unreliable.

0 Karma

msg4sunil
Path Finder

 I was trying the below, but its not helping much as in its not extracting all the data. 😞

*| rex field=_raw "prefix:(?<from>\w+)" | dedup from | table from

0 Karma

yuanliu
SplunkTrust
SplunkTrust

It's not clear what you're trying to achieve with that rex.  If you need help to put the conformant JSON part of your log into a field that spath can operate on, here are some suggestions.

  • Go back to your developers and ask them to do something helpful, namely, place the JSON part in a key-value pair that Splunk can easily handle, e.g.,

 

2021-11-25 00:48:02 something unimportant conformant='{"prefix:field1":"ABC","TxnMsg":{"prefix:field2":"XYZ","prefix:field3":123}}' something else unimportant​

Then, you can use spath input=conformant.  This is the best option.

 

  • Post some sample full log (anonymized) for others to help analyze and determine how to get the conformant part.

 

0 Karma

msg4sunil
Path Finder

Thank you.  In the above case, what would be the exact command to extract prefix:field1, prefix:field2, prefix:field3 in tabular fashion .  What needs to be added to the below?

 spath input=conformant

thank you

0 Karma

yuanliu
SplunkTrust
SplunkTrust

In that path, you need to go back to the developers who produced the logs, ask them to place the JSON part in a key-pair structure as exemplified in my comment. (Or maybe they already did and your data already contained that JSON field?)

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...