How do I use the Splunk for amMap app posted on Splunkbase

[Will_Hayes]

How do I install, configure and utilize the Splunk for amMap apps flashmaps in my Splunk instance?

http://www.splunkbase.com/apps/All/4.x/Add-On/app:Splunk+for+use+with+amMap+Flash+Maps

[nate015]

Has anyone used ammap without a lookup? The necessary fields exist in my results and I'd like to bypass creating a lookup. See the link below.

http://splunk-base.splunk.com/answers/28740/use-ammap-without-a-lookup

[seanlon11]

I took away the "... | mapit" to see if any data is returned, and I do get data back. However, I get "no results returned" when I add the "mapit" command.

What am I doing wrong?

I am using the command used above (plus an earliest command to reduce the data):

• earliest=-1m | rex "(?\d+.\d+.\d+.\d+)"| search ip!=192.168* ip!=0.0.* ip!=10.*| stats count by ip | head 100 | eval count_label="Event" | eval iterator="ip" | eval iterator_label="IP" | eval movie_color="#FF0000" | eval output_file="home_threat_data.xml" | eval app="amMap" | lookup geoip clientip as ip | mapit

Thanks for your help.

[hammon0u]

I am looking for a way to make the dots on the map a differnet color based on the sourcetype of the search. Like firewall denies are yellow and idps events are red. Then have a method to drop or shade one color or the other from the map. Simular to some of the samples on the Ammap website.

Thanks,

Monty

[lloydd518]

Hi, I had a play with multicoloured dots today following the above answer and i found a slightly better method by using movie_color=case(somefield=number or "string", "#colour",someotherfield=number or "string","#colour", 1=1,"#colour for everything else")

This way you can have as many Boolean cases as you need rather than nested if statements.

Regards Lloyd

[Jason]

You can do this by throwing some logic into the eval statement for movie_color: | eval movie_color=if(sourcetype="...", "#FF0000", if(sourcetype="....", "#009900", "#FFFFFF")) will make one sourcetype red, the next green, and anything else white.

[Oti47]

Hello, I want to use AmMap-Plug-in with my own Lookup-Table. I read that the lookup should return the following fields.

client_city client_region client_country client_lat client_lon

Do you have a format example of the values the the AmMap plug-in expect.

regards Michael

[Will_Hayes · Accepted solution]

Populating the map

** YOU WILL NEED TO INSTALL THE MAXIND APP ON SPLUNKBASE FOR GEO LOCATION INFORMATION **
The app can be fond here: http://splunk-base.splunk.com/apps/22282/geo-location-lookup-script-powered-by-maxmind

I installed the app now populate my map!

By now you have installed the amMap application and should have installed supporting MAXMIND  app as well. You'll notice an empty map on the default landing page of this app. That map is set to be populated by your data on an hourly basis, mapping the top 100 public IP's in your instance that are recorded in the last hour. If you are running Splunk Free you will need to manually populate this map. You'll notice the drill down searches aren't working but I'll explain that later on in this doc.

In order to do a first run and see if things are working correctly you can run the following search:

* | rex "(?<ip>\d+.\d+.\d+.\d+)"| search ip!=192.168* ip!=0.0.* ip!=10.*|  stats count by ip | head  100 | eval count_label="Event" | eval iterator="ip" | eval iterator_label="IP" | eval movie_color="#FF0000" | eval output_file="home_threat_data.xml" | eval app="amMap" | lookup geoip clientip as ip | mapit 

If this search completes but does not display any results then you should see a populated map on the landing page.

  If you do not see anything try the search but remove the | mapit and make sure you are getting back a results table with populated geo info. If a table is returning but the geo fields are empty you have most likely do not have any public IP's in your data for the geoip translation to operate on.

It is possible to create maps on other assets such as internal IPs, subnets, hostnames etc. That is explained further down in this doc under Populating the map with csv lookups.

Breaking down the search

For the purpose of this doc we are going to use the following example search:

eventtype=firewall src_ip=* | stats count by src_ip | eval count_label="Event" | eval iterator="src_ip" | eval iterator_label="src_ip" | eval movie_color="#FF0000" | eval output_file="home_threat_data.xml" | eval app="amMap" | lookup geoip clientip as src_ip | mapit

In the first segment of the search you will see that we are searching across our firewall data looking for events with a src_ip field. (Instructions on how to create a field in splunk can be found here)

eventtype=firewall src_ip=*

This part of the search represents the results you are interested in. You may want to add additional values to have results that represent a particular threat or web traffic or something else you would like to see represented geographically on the flash map.

Next we need to create a stats table for that IP field, which is done here in the example search:

|  stats count by src_ip

The next step is to create the required fields necessary for the map_results.py script to run. These are the varibles which drive your flashmap. That is happening in the above search here:

| eval count_label="Event" | eval iterator="src_ip" | eval iterator_label="IP" | eval movie_color="#FF0000" | eval output_file="home_threat_data.xml" | eval app="amMap"


These eval statements create the REQUIRED fields for the map to work:

count_label - What we want to display on the mouse over (i.e. Security Events, BotNet events, etc.) the default in the example above is simply Events.
iterator - What should the script iterate on, in this case we are counting up the src_ip counts for each location, this is represented in the flashmap as the unique field count.
iterator_label - For better formatting we give a name to the iterator, in this case its an IP. We will use this field like this in the mouse over for a location: Unique <iterator_label>(s)
movie_color - This is the color of the balloon on the map. This can be dynamic using a rangemap or eval for multiple colors.
app - This is the app where the map data will be written to.
output_file - this is the xml file where the map data will be written to. The output file will go into the appserver/static/xml_out directory of the specified app. The map homepage of this app is already set to read from home_threat_data.xml

OPTIONAL zoom - if you are familiar with amMap and would like to set the default zoom for your map you can do so by eval'ing the zoom values into a string i.e "zoom=\"334%\" zoom_x=\"-128.58%\" zoom_y=\"-113.11%\""

Next we need to add the geo information for mapping. This can be done with the MAXMIND geoip lookup app OR by using a lookup on internal or non-ip values.
the example above is done with the MAXMIND app using the following syntax:

lookup geoip clientip as src_ip

This command returns the following fields to your results which are required by the amMap app. client_city client_region client_country client_lat client_lon

Finally we call the mapit script. If all of the above fields are present the script should run and output your flashmap.

mapit

Populating the map with csv lookups

If you would like to populate the map using something other than public IPs you can do so by creating a CSV lookup documented here: http://www.splunk.com/base/Documentation/latest/Knowledge/Addfieldsfromexternaldatasources
Your lookup must return the following fields in order for mapit to run properly: client_city client_region client_country client_lat client_lon

Adding drill down capabilites

If your searches are failing when you drill down it is most likley becuase you are not interating on a searchable field in Splunk. Documentation on how to create fields in your results can be found here: http://www.splunk.com/base/Documentation/4.1.3/User/ExtractNewFields

Adding a amMap map to my own application or view

The sample map lives inside of the ammap_view view. You will see the following stanza:

  <module name="ServerSideInclude" layoutPanel="panel_row1_col1">
    <param name="src">ammap.html</param>
  </module>

This refers to ammap.html in the appserver/static directory. That HTML file specifies the xml being used, we generate the xml with the output_file field in our search.

Creating a new map

• First you will need to do a couple of things in the app you want to render the map in. You will need to copy the ammap folder in amMap/appserver/static to the YOUR_APP/appserver/static directory of your app.
• Next you'll need to create an empty xml_out directory
• Next copy the ammap.html from amMap/appserver/static into the YOUR_APP/appserver/static directory, you can re-name this file. You will need to modify the following line:
          so.addVariable("data_file", escape("/static/app/amMap/xml_out/home_threat_data.xml"));
  updating the home_threat_data.xml and amMap value with your output_file and app eval statements respectively. 
• Finally create a view with a serverside include module for your HTML file like this

  <module name="ServerSideInclude" layoutPanel="panel_row1_col1">
    <param name="src">ammap.html</param>
  </module>

Debugging map_results.py

The map_results log file is indexed into the Splunk internal index. You can view that log with the following search: index="_internal" source="*ammap_map_results.log" Additional debugging statements can be added by un-commenting anywhere you see logger() being called.

Extending the mapping capabilities

I've been playing with lots of different outputs for amMap maps. Please send me your ideas and I may have a parser you can use now or may need to do a little tweaking but in any case please send feed back as I'd love to hear how we can make this feature useful for you!


will@splunk.com 6/4/2010