topic How to extract key-value field extraction? in Splunk Search

How to extract key-value field extraction?

mcbradford — Fri, 20 Apr 2018 14:49:26 GMT

Need help with key value extraction for the following:

Apr 20 10:38:59 10.1.8.25 {"adf": 1, "virtualservice": "virtualservice-blahhhhh-blooob-blahhhh”, "vs_ip": “10.1.1.1”, "client_ip": “123.123.123.123”, "client_src_port": 45040, "client_dest_port": 25, "start_timestamp": "2018-04-20T14:37:00.281459", "report_timestamp": "2018-04-20T14:38:58.829212", "total_time": 118598, "connection_ended": 1, "client_rtt": 16, "mss": 1460, "service_engine": “blah-DC-bl-blob”, "vcpu_id": 1, "log_id": 1419929, "pool": "pool-blahhhh-79a9-4d4a-8e2c-blahhhh”, "pool_name": "mail.blahhh.com-pool", "server_ip": “123.123.123.123”, "server_name": “123.123.123.123”, "server_conn_src_ip": “123.123.123.123”, "server_dest_port": 443, "server_src_port": 49704, "server_rtt": 1, "significant_log": ["ADF_SERVER_SENT_RESET"], "proxy_protocol": "PROXY_PROTOCOL_VERSION_1", "dns_qtype": "DNS_RECORD_OTHER", "dns_response": {"response_code": "DNS_RCODE_NOERROR", "opcode": "DNS_OPCODE_QUERY"}, "dns_etype": "DNS_ENTRY_PASS_THROUGH", "protocol": "PROTOCOL_TCP", "dns_request": {"opcode": "DNS_OPCODE_QUERY"}, "vs_name": "mail.blahh.com"}

Data has been sanitized to remove true IPs and domains, etc.

Any help would truly be appreciated.

Re: How to extract key-value field extraction?

niketn — Fri, 20 Apr 2018 15:32:42 GMT

@mcbradford, try the following replace to extract JSON data from your _raw data, followed by spath command.

<yourCurrentSearch>
| eval _raw=replace(_raw,"^[^\{]+","")
| spath

Following is a run anywhere search based on mocked up sample data:

| makeresults
| eval _raw=" Apr 20 10:38:59 10.1.8.25 {\"adf\": 1, \"virtualservice\": \"virtualservice-blahhhhh-blooob-blahhhh\", \"vs_ip\": \"10.1.1.1\", \"client_ip\": \"123.123.123.123\", \"client_src_port\": 45040, \"client_dest_port\": 25, \"start_timestamp\": \"2018-04-20T14:37:00.281459\", \"report_timestamp\": \"2018-04-20T14:38:58.829212\", \"total_time\": 118598, \"connection_ended\": 1, \"client_rtt\": 16, \"mss\": 1460, \"service_engine\": \"blah-DC-bl-blob\", \"vcpu_id\": 1, \"log_id\": 1419929, \"pool\": \"pool-blahhhh-79a9-4d4a-8e2c-blahhhh\", \"pool_name\": \"mail.blahhh.com-pool\", \"server_ip\": \"123.123.123.123\", \"server_name\": \"123.123.123.123\", \"server_conn_src_ip\": \"123.123.123.123\", \"server_dest_port\": 443, \"server_src_port\": 49704, \"server_rtt\": 1, \"significant_log\": [\"ADF_SERVER_SENT_RESET\"], \"proxy_protocol\": \"PROXY_PROTOCOL_VERSION_1\", \"dns_qtype\": \"DNS_RECORD_OTHER\", \"dns_response\": {\"response_code\": \"DNS_RCODE_NOERROR\", \"opcode\": \"DNS_OPCODE_QUERY\"}, \"dns_etype\": \"DNS_ENTRY_PASS_THROUGH\", \"protocol\": \"PROTOCOL_TCP\", \"dns_request\": {\"opcode\": \"DNS_OPCODE_QUERY\"}, \"vs_name\": \"mail.blahh.com\"}"
| eval _raw=replace(_raw,"^[^\{]+","")
| spath

Re: How to extract key-value field extraction?

mcbradford — Fri, 20 Apr 2018 16:12:23 GMT

Perfect!!!!!