hello
I have a question
I have a data that access_log data and json data is mixed and my need is to extract field from mixed data.
splunk auto extraction field from that key=value format data
So, i received extracted json field(field name is "json") like this :
{"key":"value", "key":"value", "key":"value", "key":"value", "key":"value" ....(skip)}
And if I search the json field, I return the data but, I want extracted field from JSON like this :
| fields key, key2, key3, key4 ....(skip)
Finally, I found spath command and I got the results that I wanted.
I tried to modify props.conf to automatically extract the field from json but it is not working.
What should I do?
ps: I tried t0 use the eval command like this :
| eval example = spath(json, 'json.key')
And like this:
| eval example = json.key
But that is also not working.
i solve that like this :
transforms.conf
[my_stanza]
REGEX = "([^"]+)":(.?)"(.?)"
FORMAT = $1::$3
MV_ADD = True
thank you for answer.
i solve that like this :
transforms.conf
[my_stanza]
REGEX = "([^"]+)":(.?)"(.?)"
FORMAT = $1::$3
MV_ADD = True
thank you for answer.
Go to Settings
-> Fields
-> Field extractoins
-> New
.
Enter anything that you like for Name
(I suggest something like ColonCommaKVPs
), Enter the exact name of your sourcetype
in the named
field, keep the default of Inline
for Type
and Sourcetype
for Apply to
, then enter this for Extraction/Transform
:
"(?<_KEY_1>[^"]*)":"(?<_VAL_1>[^"]*)"
"(?<_KEY_1>[^"]*)":"(?<_VAL_1>[^"]*)"
how to get this to work where VAL value is not enclosed by quotes, such as this example :
networkConnect": {
"localIp": "10.132.137.105",
"localPort": 64248,
"networkProtocol": "kIPv4",
"remoteIp": "63.247.145.75",
"remotePort": 88
thank you for answer.
but, that solution is not working.
I think there are two reasons.
1-2) some data format is not "key":"value"
- value has blank before "value"
- "json= {"key1": "value1", "key2": "","key3": "value3" .....}"
- "key":(this place is blank)"value"
What's wrong?
OK i will tried that.
thank you for answer!!
hello woodcock.
i tried that and that look like work, but that is static.(my think...
Perhaps, You gave me the answer that I wanted. but
Maybe it is that i did not understand)
so, In my opinion, your answer is need loop like this :
[report-stanza]
"(?<_KEY_1>[^"])":"(?<_VAL_1>[^"])"
"(?<_KEY_2>[^"])":"(?<_VAL_2>[^"])"
"(?<_KEY_3>[^"])":"(?<_VAL_3>[^"])"
i want dynamic field extraction like this :
1. key1, key2, key3 .... is field name (because that key is changed dynamically)
2. value1, value2, value3 .... is field value
finally i want field extraction to simple.
use a one regular expression or delims or another way.
because that production is Maintenance by customer. so i must make easy to Maintenance.
I think there is a way like that use spath command in SPL
or another way.
thank you for answer
You should do it exactly as i described (from the GUI) and make it work, then repackage it in whatever app. My solution, with a minor adjustment of changing the +
characters to *
will work for null values. I have modified my answer so it now reads like this:
"(?<_KEY_1>[^"]*)":"(?<_VAL_1>[^"]*)"
Please specify
INDEXED_EXTRACTIONS = JSON OR KV_MODE = json but not both .
The above should extract fields at index time for structured json data .
Sorry.
I think you gave me a slightly different answer because I did not give you the exact details.
The access_log and json data are mixed as follows:
"POST /sample.logs HTTP / 1.1" 200 0 "app: /data.swf ..." "another logs ...." 10.10.10.10 - - [November / 2018: 16: 31: "userAgent data. ... "" json = {"key": "value", "key": "value"
The json field is extracted by the automatic extraction setting of the splunk.
What I want is the value of the json field.
The search command was able to extract the fields with spath command like this :
.... | spath input = json.
How can I get the same results in props.conf or transforms.conf?
thank you for answer
Refer to documentation for KV_MODE : https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Automatickey-valuefieldextractionsatse...
and INDEXED_EXTRACTIONS: http://docs.splunk.com/Documentation/Splunk/latest/Data/Extractfieldsfromfileswithstructureddata#Pro...