topic Re: Where is the "state" information stored? How can I create this filter? in Splunk Search

Where is the "state" information stored? How can I create this filter?

MMCC — Wed, 30 Sep 2020 04:35:24 GMT

Hi all,

I'm trying to create a view according to "geo_us_states" for Germany.
So far I was able to add/create the required "geospatial lookup".

| inputlookup geo_us_states

| inputlookup geo_germany

What I currently don't understand within the geom command I can specify the following:

| geom geo_us_states featureIdField="state"

I checked the kml file and I couldn't detect an XML tag explaining what "state" is. Where does Splunk store/retrieve this information?

Does any one know how I can apply it for my "Germany" purpose?

Thank you in advance for any hints.

Re: Where is the "state" information stored? How can I create this filter?

woodcock — Sun, 22 Mar 2020 19:30:16 GMT

First, read the best treatment of Splunk and mapping anywhere:
https://www.splunk.com/en_us/blog/tips-and-tricks/use-custom-polygons-in-your-choropleth-maps.html

The | inputlookup my_lookup is just to see if you can access the featureId and geom fields inside of you KML or KMZ file. If it is built in such a way that Splunk can use it, you should see many lines returned on the Statistics tab. It sounds like you got this far. If you did not, consider using the Shapester - Geo Shape Editor app on Splunkbase (https://splunkbase.splunk.com/app/2893/) to build some shapes into a KML file that definitely should be Splunk-geo-compatible. If you then click on the Visualization tab, you should be able to see the results on a map but you must do ALL of the following:

1: Select the `Choropleth Map` visualization.
2: Keep `zooming` and `centering` your view until it is positioned over the location of the shapes in your file.
3: If your shapes are small, you will find that the default maps do not allow enough `zoom` to see them; to fix this....
4: Click on the `Format` tool (the `paint brush` icon) and go to the `Tiles` section.
5: Look at the comment that says `The URL to use for requesting tiles, ex: http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png` and grab the `http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png` text and paste it into the `URL` field.  Instantly you should have infinite `zoom` detail.  Really, this is probably the `secret magic` that you lacked.  This is not clearly documented anywhere and we only discovered it by accident playing around.

It really helps to take a look at the Choropleth Map Color Modes example with San Francisco Neighborhoods in the Map Elements area of the Splunk Dashboard Examples app on Splunkbase (https://splunkbase.splunk.com/app/1603/). It shows you how to do everything EXCEPT for the magical #5 step. Although the recommended tile set is really good, there are many, MANY, options out there so be sure to try a variety. Here are some alternative tile sets that render instantly in Splunk:

https://wiki.openstreetmap.org/wiki/Tile_servers
OpenStreetMaps: http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png
Wikipedia: https://maps.wikimedia.org/osm-intl/{z}/{x}/{y}.png
OpenCycleMap: http://tile.thunderforest.com/cycle/{z}/{x}/{y}.png
Humanitarian Style: http://a.tile.openstreetmap.fr/hot/{z}/{x}/{y}.png
Hike and Bike: https://tiles.wmflabs.org/hikebike/{z}/{x}/{y}.png

Re: Where is the "state" information stored? How can I create this filter?

to4kawa — Sun, 22 Mar 2020 22:27:42 GMT

I checked Choropleth Map before
https://answers.splunk.com/answers/795337/statistics-to-choropleth-map-1.html

you should make featureId state

geo_germany is official? I want it.
good luck

Re: Where is the "state" information stored? How can I create this filter?

MMCC — Wed, 30 Sep 2020 04:45:01 GMT

Hi @woodcock , Hi @to4kawa

thanks for your hints. @woodcock I really appreciate your detailed answer.

I found what helped me to solve my issue within "Splunk Dashboard Examples"

It was one word that seemed to be required to solve my problem: OUTPUT

Without OUTPUT at the end of my search the "featureId" field was not being generated...

Therefore I couln't further "filter" my map down to the states.

index=cisco_vpn sourcetype=cisco:ios
mnemonic=EZVPN_CONNECTION_* |
iplocation Client_public_addr
Server_public_addr allfields=true |
lookup geo_de_bundeslaender latitude
as lat longitude as lon OUTPUT
featureId | stats count by featureId |
geom geo_de_bundeslaender

This is what my search command looks like now.
I realised that the mapping for my lookup is crucial to. If you don't specify the "as" as the output of your geoinformation command (iplocation in my case) it won't work...

When I was searching through Splunk I found following csv file, which I replicated for my usage:

@to4kawa what I have done is not official. I found the kml file on following page:
https://community.qlik.com/t5/Qlik-Sense-Documents-Videos/KML-files-for-Germany-Austria-Switzerland-County-District-Level/ta-p/1489778

I then extracted only the XML tags with the position information and replaced that in a copy from "geo_us_states".

Only thing that I could not replicate yet is displaying the state codes on the map. If you happen to find how that's done I would appreciate any additional hints.

Re: Where is the "state" information stored? How can I create this filter?

woodcock — Mon, 23 Mar 2020 14:25:02 GMT

If you mean the 2-letter labels (like, "TX"), that should comes from your KML/KMZ file. The Splunk built-in "states" file has this.

You should pick an answer and click Accept to close the question and also UpVote any helpful comments and answers.

Re: Where is the "state" information stored? How can I create this filter?

MMCC — Mon, 23 Mar 2020 15:32:39 GMT

Yeah that's what I mean.

I found the following in the KML example:

      <name>us_states</name>
      <Placemark>
          <name>Alabama</name>
          <Style>
              <LineStyle>
                  <color>ff0000ff</color>
              </LineStyle>
              <PolyStyle>
                  <fill>0</fill>
              </PolyStyle>
          </Style>
          <ExtendedData>
              <SchemaData schemaUrl="#us_states">
                  <SimpleData name="STATE_FIPS">01</SimpleData>
                  <SimpleData name="STATE_CODE">AL</SimpleData>

I replaced it with my input:

      <name>de_bundeslaender</name>
      <Placemark>
          <name>Schleswig-Holstein</name>
          <Style>
              <LineStyle>
                  <color>ff0000ff</color>
              </LineStyle>
              <PolyStyle>
                  <fill>0</fill>
              </PolyStyle>
          </Style>
          <ExtendedData>
              <SchemaData schemaUrl="#de_bundeslaender">
                  <SimpleData name="STATE_FIPS">01</SimpleData>
                  <SimpleData name="STATE_CODE">SH</SimpleData>

But those 2 letter labels are still not being displayed...

Re: Where is the "state" information stored? How can I create this filter?

to4kawa — Mon, 23 Mar 2020 19:11:18 GMT

index=cisco_vpn sourcetype=cisco:ios
mnemonic=EZVPN_CONNECTION_* |
iplocation Client_public_addr
Server_public_addr allfields=true |
lookup geo_de_bundeslaender latitude
as lat longitude as lon OUTPUT
featureId | stats count by featureId

What's your result above?
geom need featureId.

you should have the results following:

count featureId
12 Schleswig-Holstein
34 Bayern
6 Hamburg
....

and check your lookup

| inputlookup geo_de_bundeslaender

this result should be

count   featureCollection   featureId   geom
0   de_bundeslaender    Baden-Württemberg  {"type":"MultiPolygon","coordinates":[[[[137.80606079101562,35.204750061035156],[136.68519592285156,35.16719436645508],[137.80606079101562,35.204750061035156]]]]}
0   de_bundeslaender    Bayern  {"type":"MultiPolygon","coordinates":[[[[137.80606079101562,35.204750061035156],[136.68519592285156,35.16719436645508],[137.80606079101562,35.204750061035156]]]]}
....

last spl is | geom geo_germany ?

thanks for your geo lookup detail. I'll get it.

Re: Where is the "state" information stored? How can I create this filter?

MMCC — Wed, 30 Sep 2020 04:45:32 GMT

Hi @to4kawa

yes you are right the result is as you described. In both cases.

The last spl for creating the map is

| geom geo_de_bundeslaender

As that is the name I gave the lookup definition and the lookup file.
Done it in the same way as they did it for geo_us_states.

Welcome. I think its's a pitty I couldn't find it from a german site. Someone abroad had to create it 😉

Re: Where is the "state" information stored? How can I create this filter?

to4kawa — Tue, 24 Mar 2020 20:24:09 GMT

I have same problem no local map.
I find it from R library.

now, I'm trying to get data from web.

kml to geo lookup makes featureId and geom fields.

good luck.

Re: Where is the "state" information stored? How can I create this filter?

woodcock — Wed, 25 Mar 2020 00:56:32 GMT

Look at the San Francisco neighborhoods Map example in Dashboard Examples App.

Re: Where is the "state" information stored? How can I create this filter?

MMCC — Fri, 27 Mar 2020 09:00:15 GMT

Thank you again. I will