topic Statistics to Choropleth map in All Apps and Add-ons

Statistics to Choropleth map

to4kawa — Sun, 12 Jan 2020 00:28:44 GMT

I am trying to draw a map from coordinate data.
Originally a geojson format file, so I did not create lookup and imported it into Index.

count   featureId   geom
1289    Yokohama    { "type": "Polygon", "coordinates": [ [ [ 139.67603133597743, 35.45629161326741 ], [ 139.67587019455391, 35.455984838329698 ], [ 139.67581192028399, 35.455874054444109 ], [ 139.67579549866343, 35.455865027049413 ], [ 139.6757730219075, 35.455862386639865 ], [ 139.67574019485414, 35.45586863962609 ], [ 139.6757145012233, 35.455885386801185 ], [ 139.67570430560932, 35.455912585897181 ], [ 139.67576016879684, 35.456023531660719 ], [ 139.67582211409933, 35.456004135082821 ], [ 139.67586997421995, 35.456092360374164 ], [ 139.67594143974566, 35.456223891619345 ], [ 139.67593316508362, 35.45623641377955 ], [ 139.67595422810518, 35.456285026632713 ], [ 139.67597513644341, 35.45629405402741 ], [ 139.675981284209, 35.456309107779134 ], [ 139.67603133597743, 35.45629161326741 ] ] ] }

This is my Statistics result.
I want to visualization Choropleth map. But I can't.
I think the information is enough , but what is missing?

Re: Statistics to Choropleth map

rmmiller — Mon, 13 Jan 2020 17:04:07 GMT

What if you change your type from "Polygon" to "MultiPolygon"?

Re: Statistics to Choropleth map

to4kawa — Mon, 13 Jan 2020 19:42:36 GMT

I changed it like below, but it was no good.

| eval geom=replace(geom,"Polygon","MultiPolygon")

Re: Statistics to Choropleth map

rmmiller — Wed, 15 Jan 2020 17:55:26 GMT

I think it's because you're missing a field in your result named featureCollection. Without it, choropleth seems confused about what to do with your result.

I was able to recreate this problem using generated data. Your JSON is missing a set of brackets around the coordinates, but that didn't seem to make any difference in my tests. I tried replacing the quad brackets with the triple that appear in your output -- no difference. The same is true with differences in whitespace between your JSON and the output of geom.

| makeresults count=50
| eval raw=split("8.8.8.8,50.19.98.133,54.204.12.32,13.32.83.121",",")
| eval src_ip=mvindex(raw,random()%4)
| iplocation src_ip
| fields - raw
| stats count by Region
| rename Region as featureId
| geom geo_us_states
| eval geom=replace(geom,"\[\[\[\[","[ [ [ ")
| eval geom=replace(geom,"\]\]\]\]","] ] ] ")
| eval geom=replace(geom,"MultiPolygon","Polygon")
`comment("Uncomment the line immediately below to break choropleth")`
`comment("| fields - featureCollection")`
`comment("You will need to change the visualization to something")`
`comment("else and back to choropleth or you will see cache effects")`

IP addresses are for Google DNS, answers.splunk.com, and conf.splunk.com
My guess is if you have a featureCollection field, choropleth will start working as expected.

Hope that helps!
rmmiller

Re: Statistics to Choropleth map

to4kawa — Wed, 15 Jan 2020 18:48:52 GMT

Thank you for your reply.
This works fine. I will try various things.

Re: Statistics to Choropleth map

to4kawa — Thu, 16 Jan 2020 01:25:36 GMT

e.g. 1:

| makeresults
| eval featureCollection="geo_countries"
| eval count=1
| eval featureId="Afghanistan" 
| eval geom="{\"type\":\"MultiPolygon\",\"coordinates\":[[[[71.0498046875,38.40866470336914],[71.65302276611328,36.68701171875],[74.89230346679688,37.23111343383789],[71.22307586669922,36.12539291381836],[69.04010772705078,31.673107147216797],[65.03636932373047,29.5401611328125],[60.84437942504883,29.858179092407227],[61.269676208496094,35.618499755859375],[71.0498046875,38.40866470336914]]]]}"
| table count featureCollection featureId geom

Result: Bad

e.g. 2:

| makeresults
| eval featureCollection="geo_countries"
| eval count=1
| eval featureId="Afghanistan" 
| eval geom="{\"type\":\"MultiPolygon\",\"coordinates\":[[[[71.0498046875,38.40866470336914],[71.65302276611328,36.68701171875],[74.89230346679688,37.23111343383789],[71.22307586669922,36.12539291381836],[69.04010772705078,31.673107147216797],[65.03636932373047,29.5401611328125],[60.84437942504883,29.858179092407227],[61.269676208496094,35.618499755859375],[71.0498046875,38.40866470336914]]]]}"
| table count featureCollection featureId geom
| geom geo_countries

Result: OK

e.g. 3:

| makeresults
| eval featureCollection="geo_countries"
| eval count=1
| eval featureId="Afghanistan" 
| eval geom="{\"type\":\"MultiPolygon\",\"coordinates\":[[[[71.0498046875,38.40866470336914],[71.65302276611328,36.68701171875],[74.89230346679688,37.23111343383789],[71.22307586669922,36.12539291381836],[69.04010772705078,31.673107147216797],[65.03636932373047,29.5401611328125],[60.84437942504883,29.858179092407227],[61.269676208496094,35.618499755859375],[71.0498046875,38.40866470336914]]]]}"
| table count featureCollection featureId geom
| appendpipe 
    [| geom geo_countries]
| dedup geom

Result: Bad

e.g. 4:

| makeresults
| eval featureCollection="geo_countries"
| eval count=1
| eval featureId="Afghanistan" 
| eval geom="{\"type\":\"MultiPolygon\",\"coordinates\":[[[[71.0498046875,38.40866470336914],[71.65302276611328,36.68701171875],[74.89230346679688,37.23111343383789],[71.22307586669922,36.12539291381836],[69.04010772705078,31.673107147216797],[65.03636932373047,29.5401611328125],[60.84437942504883,29.858179092407227],[61.269676208496094,35.618499755859375],[71.0498046875,38.40866470336914]]]]}"
| table count featureCollection featureId geom
| appendpipe 
    [| geom geo_countries]
    | reverse

Result: OK

e.g. 5:

| makeresults
| eval featureCollection="geo_countries"
| eval count=1
| eval featureId="Afghanistan" 
| eval geom="{\"type\":\"MultiPolygon\",\"coordinates\":[[[[71.0498046875,38.40866470336914],[71.65302276611328,36.68701171875],[74.89230346679688,37.23111343383789],[71.22307586669922,36.12539291381836],[69.04010772705078,31.673107147216797],[65.03636932373047,29.5401611328125],[60.84437942504883,29.858179092407227],[61.269676208496094,35.618499755859375],[71.0498046875,38.40866470336914]]]]}"
| table count featureCollection featureId geom
| appendpipe 
    [| geom geo_countries]
    | stats values(*) as *

Result:Bad

e.g. 6:

| makeresults
| eval featureCollection="geo_countries"
| eval count=1
| eval featureId="Afghanistan" 
| eval geom="{\"type\":\"MultiPolygon\",\"coordinates\":[[[[71.0498046875,38.40866470336914],[71.65302276611328,36.68701171875],[74.89230346679688,37.23111343383789],[71.22307586669922,36.12539291381836],[69.04010772705078,31.673107147216797],[65.03636932373047,29.5401611328125],[60.84437942504883,29.858179092407227],[61.269676208496094,35.618499755859375],[71.0498046875,38.40866470336914]]]]}"
| table count featureCollection featureId geom
| appendpipe 
    [| geom geo_countries]
    | reverse
    | stats values(*) as *

Result: sametimes OK

All stats result is 1 line.
wow , what's happen? Statistics results are same, but Visualization is different.

e.g.6 appears to indicate that your browser has cache.

Re: Statistics to Choropleth map

rmmiller — Thu, 16 Jan 2020 02:21:05 GMT

Wow! For each one of those tests, did you change your visualization away from choropleth and back to choropleth? That was the only way I could work around the caching of the previous result.

Re: Statistics to Choropleth map

to4kawa — Thu, 16 Jan 2020 04:05:07 GMT

No, I didn't.
I'll try later.
thanks.

Re: Statistics to Choropleth map

to4kawa — Fri, 17 Jan 2020 06:56:57 GMT

e.g. A

| inputlookup geo_us_states
| head 1
| transpose

Results:

column  row 1
_featureIdField featureId
count   0
featureCollection   geo_us_states
featureId   Alabama
geom    {"type":"MultiPolygon","coordinates":[[[[-88.31002807617188,30.233232498168945],[-88.31002807617188,30.233232498168945]]],[[[-88.47322845458984,31.893856048583984],[-88.20295715332031,35.008026123046875],[-85.60516357421875,34.984676361083984],[-85.00250244140625,31.000682830810547],[-88.02840423583984,30.221132278442383],[-88.47322845458984,31.893856048583984]]]]}

There is _featureIdField . If you delete this field (e.g. |fields - _featureIdField ) , Choropleth Map will not work properly.

e.g. B1

| inputlookup geo_us_states
| head 1

e.g. B2

| inputlookup geo_us_states
| head 1
| fields -  featureId

It is easy to understand when compared.
If you delete featureId, it will be drawn only with the coordinates specified in geom .
What data are Splunk using?

e.g. C

| inputlookup geo_us_states
| head 1
| fields -  featureId
| eval geom=replace(geom,"MultiPolygon","Polygon")

Result: Bad
From this result, type is only MultiPolygon . The way of specifying coordinates is different.

e.g. D

| makeresults 
| eval featureCollection="geo_us_states" 
| eval _featureIdField="featureId" 
| eval geom="{\"type\":\"MultiPolygon\",\"coordinates\":[[-88.31002807617188,30.233232498168945],[-88.31002807617188,30.233232498168945],[-88.47322845458984,31.893856048583984],[-88.20295715332031,35.008026123046875],[-85.60516357421875,34.984676361083984],[-85.00250244140625,31.000682830810547],[-88.02840423583984,30.221132278442383],[-88.47322845458984,31.893856048583984]]}" 
| eval count=1

Result:OK
I modified the coordinate of Alabama . please compare e.g. A
From this result, If the coordinates are correct, display is possible.

Re: Statistics to Choropleth map

to4kawa — Fri, 17 Jan 2020 07:11:03 GMT

As a result of the investigation, the coordinates I prepared were wrong.

To draw on the Choropleth map

Correct coordinates
Prepare each count _featureIdField featureCollection geom fields
The setting of _featureIdField is |eval _featureIdField = "featureId" . If there is not featureId, no problem, too.
The type of geom is MultiPolygon

You need these.

cf. IplocationChoropleth

Re: Statistics to Choropleth map

rmmiller — Fri, 17 Jan 2020 15:01:31 GMT

Great investigation, @to4kawa!!

Re: Statistics to Choropleth map

to4kawa — Sat, 18 Jan 2020 00:18:54 GMT

Thank you @rmmiller
I have to find accurate coordinate information 😞
Also, in this case
I understand the behavior of dedup and stats values ().
These hold on top.
The order of the logs is important.