All Apps and Add-ons
×

Are you a member of the Splunk Community?

Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Ask a Question

Parsing Meraki CMX API data on props.conf

iancorrea
Path Finder
‎12-12-2019 01:55 AM

I'm trying to parse data coming from Cisco Meraki CMX using HTTP Event Collector. The raw data was JSON API.
I'm using props.conf

[http_json]
INDEXED_EXTRACTIONS = json
SHOULD_LINEMERGE = false
KV_MODE = json
NO_BINARY_CHECK = 1
TIME_PREFIX = "seenTime": "
TIME_FORMAT = %Y-%m-%d
MAX_DAYS_AGO = 5000

but the event in the Splunk is only parsed to only one event. Do I need to edit transforms.conf for this kind of data or is there any mistake on my props.conf? Thank you.

alt text

This is the sample JSON raw data that I'm trying to parse.

{
"version": "2.0",
"secret": "splunk*****",
"type": "DevicesSeen",
"data": {
"apMac": "00:18:0a:13:dd:b0",
"apFloors": [],
"apTags": [
"dev",
"home",
"test"
],
"observations": [
{
"ipv4": "/192.168.0.56",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 0.20755340376944298,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:14Z",
"ssid": ".interwebs",
"os": null,
"clientMac": "18:fe:34:d7:7c:26",
"seenEpoch": 1474675574,
"rssi": 56,
"ipv6": null,
"manufacturer": "Espressif"
},
{
"ipv4": null,
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 2.355135751488216,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:28Z",
"ssid": null,
"os": "Android",
"clientMac": "98:f1:70:34:1c:3e",
"seenEpoch": 1474675588,
"rssi": 43,
"ipv6": null,
"manufacturer": "Murata Manufacturing"
},
{ ...

Preview file
142 KB
0 Karma
Reply

to4kawa
Ultra Champion
‎12-25-2019 05:34 PM 
| makeresults count=1 
| eval _raw="{
\"version\": \"2.0\",
\"secret\": \"splunk*\",
\"type\": \"DevicesSeen\",
\"data\": {
\"apMac\": \"00:18:0a:13:dd:b0\",
\"apFloors\": [],
\"apTags\": [
\"dev\",
\"home\",
\"test\"
    ],
\"observations\": [
{
\"ipv4\": \"/192.168.0.56\",
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": 0.20755340376944298,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:06:14Z\",
\"ssid\": \".interwebs\",
\"os\": null,
\"clientMac\": \"18:fe:34:d7:7c:26\",
\"seenEpoch\": 1474675574,
\"rssi\": 56,
\"ipv6\": null,
\"manufacturer\": \"Espressif\"
    },
{
\"ipv4\": null,
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": 2.355135751488216,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:06:28Z\",
\"ssid\": null,
\"os\": \"Android\",
\"clientMac\": \"98:f1:70:34:1c:3e\",
\"seenEpoch\": 1474675588,
\"rssi\": 43,
\"ipv6\": null,
\"manufacturer\": \"Murata Manufacturing\"
    },
{
\"ipv4\": \"/192.168.0.95\",
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": 0.30140056490166983,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:05:46Z\",
\"ssid\": \".interwebs\",
\"os\": \"Generic Linux\",
\"clientMac\": \"74:da:38:88:7c:df\",
\"seenEpoch\": 1474675546,
\"rssi\": 53,
\"ipv6\": null,
\"manufacturer\": \"Edimax Technology\"
    },
{
\"ipv4\": \"/192.168.0.42\",
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": 0.11936056928416994,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:06:40Z\",
\"ssid\": \"MR34 Bridge\",
\"os\": \"Mac OS X\",
\"clientMac\": \"f4:5c:89:9b:17:67\",
\"seenEpoch\": 1474675600,
\"rssi\": 53,
\"ipv6\": null,
\"manufacturer\": \"Apple\"
    },
{
\"ipv4\": \"/192.168.0.4\",
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": -0.5,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:06:19Z\",
\"ssid\": \".interwebs\",
\"os\": \"iOS\",
\"clientMac\": \"c0:f2:fb:0b:05:e7\",
\"seenEpoch\": 1474675579,
\"rssi\": 60,
\"ipv6\": null,
\"manufacturer\": \"Apple\"
    },
{
\"ipv4\": \"/192.168.0.63\",
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": 0.20755340376944298,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:06:27Z\",
\"ssid\": \".interwebs\",
\"os\": null,
\"clientMac\": \"18:fe:34:e1:b4:7a\",
\"seenEpoch\": 1474675587,
\"rssi\": 56,
\"ipv6\": null,
\"manufacturer\": \"Espressif\"
    },
{
\"ipv4\": \"/192.168.0.92\",
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": 0.22200004405702012,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:06:39Z\",
\"ssid\": \".interwebs\",
\"os\": null,
\"clientMac\": \"18:fe:34:ce:a1:6d\",
\"seenEpoch\": 1474675599,
\"rssi\": 57,
\"ipv6\": null,
\"manufacturer\": \"Espressif\"
    },
{
\"ipv4\": \"/192.168.0.35\",
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": 1.6658518302986447,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:06:37Z\",
\"ssid\": \".interwebs\",
\"os\": null,
\"clientMac\": \"18:fe:34:f2:95:26\",
\"seenEpoch\": 1474675597,
\"rssi\": 46,
\"ipv6\": null,
\"manufacturer\": \"Espressif\"
    },
{
\"ipv4\": \"/192.168.0.69\",
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": 0.6019477321869419,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:06:02Z\",
\"ssid\": \".interwebs\",
\"os\": null,
\"clientMac\": \"18:fe:34:ed:8b:ea\",
\"seenEpoch\": 1474675562,
\"rssi\": 61,
\"ipv6\": null,
\"manufacturer\": \"Espressif\"
    },
{
\"ipv4\": \"/192.168.0.38\",
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": 1.233417960754815,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:06:23Z\",
\"ssid\": \".interwebs\",
\"os\": null,
\"clientMac\": \"18:fe:34:fc:5a:7f\",
\"seenEpoch\": 1474675583,
\"rssi\": 47,
\"ipv6\": null,
\"manufacturer\": \"Espressif\"
    },
{
\"ipv4\": \"/192.168.0.15\",
\"location\": {
\"lat\": 51.5355157,
\"lng\": -0.06990350000000944,
\"unc\": 1.5497743004111961,
\"x\": [],
\"y\": []
},
\"seenTime\": \"2016-09-24T00:06:40Z\",
\"ssid\": \".interwebs\",
\"os\": \"Generic Linux\",
\"clientMac\": \"74:da:38:56:0a:80\",
\"seenEpoch\": 1474675600,
\"rssi\": 47,
\"ipv6\": null,
\"manufacturer\": \"Edimax Technology\"
    }
]
}
}" 
| spath 
| rename COMMENT as "your log is this state. From here the logic"
| table secret type version data.* 
| rename data.* as * 
| eval counter=mvrange(0,mvcount('observations{}.clientMac')) 
| stats list(*) as * by counter 
| rename observations{}.* as o_* 
| rename o_location.* as o_location_* 
| foreach o_* 
    [| eval <<FIELD>> = mvindex(<<FIELD>>,counter) 
    | rename <<FIELD>> as <<MATCHSTR>>] 
| fields - counter

Apparently the fields are extracted, and this query can be used for expansion. I think there is no problem.

0 Karma
Reply

kamlesh_vaghela
SplunkTrust
SplunkTrust
‎12-12-2019 07:45 AM

@iancorrea

Can you please share clear sample event? Make sure it should from _raw .

0 Karma
Reply

iancorrea
Path Finder
‎12-12-2019 06:38 PM

Hi @kamlesh_vaghela , I'm just trying to simulate the Meraki CMX API process using Postman and the sample events I gathered was from this site https://documenter.getpostman.com/view/897512/71FUpux?version=latest#7f7ed3e0-0ed4-8d55-c39d-b802dad....

Here is the whole event from this site,

{
"version": "2.0",
"secret": "splunk*****",
"type": "DevicesSeen",
"data": {
"apMac": "00:18:0a:13:dd:b0",
"apFloors": [],
"apTags": [
"dev",
"home",
"test"
],
"observations": [
{
"ipv4": "/192.168.0.56",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 0.20755340376944298,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:14Z",
"ssid": ".interwebs",
"os": null,
"clientMac": "18:fe:34:d7:7c:26",
"seenEpoch": 1474675574,
"rssi": 56,
"ipv6": null,
"manufacturer": "Espressif"
},
{
"ipv4": null,
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 2.355135751488216,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:28Z",
"ssid": null,
"os": "Android",
"clientMac": "98:f1:70:34:1c:3e",
"seenEpoch": 1474675588,
"rssi": 43,
"ipv6": null,
"manufacturer": "Murata Manufacturing"
},
{
"ipv4": "/192.168.0.95",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 0.30140056490166983,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:05:46Z",
"ssid": ".interwebs",
"os": "Generic Linux",
"clientMac": "74:da:38:88:7c:df",
"seenEpoch": 1474675546,
"rssi": 53,
"ipv6": null,
"manufacturer": "Edimax Technology"
},
{
"ipv4": "/192.168.0.42",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 0.11936056928416994,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:40Z",
"ssid": "MR34 Bridge",
"os": "Mac OS X",
"clientMac": "f4:5c:89:9b:17:67",
"seenEpoch": 1474675600,
"rssi": 53,
"ipv6": null,
"manufacturer": "Apple"
},
{
"ipv4": "/192.168.0.4",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": -0.5,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:19Z",
"ssid": ".interwebs",
"os": "iOS",
"clientMac": "c0:f2:fb:0b:05:e7",
"seenEpoch": 1474675579,
"rssi": 60,
"ipv6": null,
"manufacturer": "Apple"
},
{
"ipv4": "/192.168.0.63",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 0.20755340376944298,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:27Z",
"ssid": ".interwebs",
"os": null,
"clientMac": "18:fe:34:e1:b4:7a",
"seenEpoch": 1474675587,
"rssi": 56,
"ipv6": null,
"manufacturer": "Espressif"
},
{
"ipv4": "/192.168.0.92",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 0.22200004405702012,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:39Z",
"ssid": ".interwebs",
"os": null,
"clientMac": "18:fe:34:ce:a1:6d",
"seenEpoch": 1474675599,
"rssi": 57,
"ipv6": null,
"manufacturer": "Espressif"
},
{
"ipv4": "/192.168.0.35",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 1.6658518302986447,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:37Z",
"ssid": ".interwebs",
"os": null,
"clientMac": "18:fe:34:f2:95:26",
"seenEpoch": 1474675597,
"rssi": 46,
"ipv6": null,
"manufacturer": "Espressif"
},
{
"ipv4": "/192.168.0.69",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 0.6019477321869419,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:02Z",
"ssid": ".interwebs",
"os": null,
"clientMac": "18:fe:34:ed:8b:ea",
"seenEpoch": 1474675562,
"rssi": 61,
"ipv6": null,
"manufacturer": "Espressif"
},
{
"ipv4": "/192.168.0.38",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 1.233417960754815,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:23Z",
"ssid": ".interwebs",
"os": null,
"clientMac": "18:fe:34:fc:5a:7f",
"seenEpoch": 1474675583,
"rssi": 47,
"ipv6": null,
"manufacturer": "Espressif"
},
{
"ipv4": "/192.168.0.15",
"location": {
"lat": 51.5355157,
"lng": -0.06990350000000944,
"unc": 1.5497743004111961,
"x": [],
"y": []
},
"seenTime": "2016-09-24T00:06:40Z",
"ssid": ".interwebs",
"os": "Generic Linux",
"clientMac": "74:da:38:56:0a:80",
"seenEpoch": 1474675600,
"rssi": 47,
"ipv6": null,
"manufacturer": "Edimax Technology"
}
]
}
}

0 Karma
Reply
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...