Splunk Search

Creating usage reports from logfiles

huaraz
Explorer

Hi,

I am new to splunk and heard it can do nearly every type of reporting. I have an ADSL router creating logs in the following format:

Aug 25 23:00:22 Vigor: Local User: 192.168.1.8:50829 -> 212.58.244.67:80 (TCP)Web

How can I create a pie chart showing for each source (e.g. 192.168.1.8) to which destinations (e.g. 212.58.244.67) they went.

I can import the logs and select the appropriate lines, but I don't know I can define variables (are these fields in splunk ?) for src and dst and plot them.

Thank you
Markus

I read a bit about custom fields. I see without any search regex the follwoing events
Aug 25 23:00:22 Vigor: Local User: 192.168.1.8:50829 -> 212.58.244.67:80 (TCP)Web host=ip-10-17-23-243 Options| sourcetype=router-kiwi Options| source=/home/markus/data/router-kiwi-2011-08-25.txt Options

I created the following field extractions for host ip-10-17-23-243 (as it forces me to use either host, source or sourcetype)

"User:\s*(?<mysrc>:.*)
and
->\s*(?<mydst>:.*)"

But when I create a search mysrc="192.168.1.8" I don't get anything. What do I do wrong ?
How can I check the fields are correct ? When I do a search with | rex field=_raw "User:\s*(?<mysrc>:.*) ->\s*(?<mydst>:.*)" I don't get an error, but I also don't know what is mysrc nor mysdst.

Markus

Tags (1)
0 Karma
1 Solution

chris
Motivator

Hi Huaraz

To extract the ips you can try to add this to your search:
| rex field=_raw "User:\s+(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*\s+->\s+(?<dst>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"

Splunk doesn't tell you that the regex you apply to a search is not working the way you want, because it can only detect syntax errors.

There are different ways to get the desired result:

  1. Check if the fields you just tried to create appear in the List if you click on "Pick fields" to the left of your search results (check the values that were found)
  2. Splunk can help you generate the regex if you select "Extract Fields" from the context menu of an event that contains the values you want to extract into fields/variables. You can then test and save field extractions
  3. You could also use 3rd party tool to help you with your regexes (http://regex.larsolavtorvik.com/)

Once you have your fields you can append a reporting command to your search (then click on show report to format the report):

Popular destinations(pie chart:

| chart count(src) by dst

Active Sources (pie chart):

| chart count(dst) by src

Show when a source is active (line chart):

| timechart count(src) by src

Or you could also just create a table of your sources and destination tuples:

| rex field=_raw "User:\s+(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*\s+->\s+(?<dst>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" | where isnotnull(src) | table src dst | sort src

View solution in original post

0 Karma

huaraz
Explorer

I used Manager » Fields » Field extractions to add a new field

Markus

0 Karma

chris
Motivator

Hi Huaraz

To extract the ips you can try to add this to your search:
| rex field=_raw "User:\s+(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*\s+->\s+(?<dst>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"

Splunk doesn't tell you that the regex you apply to a search is not working the way you want, because it can only detect syntax errors.

There are different ways to get the desired result:

  1. Check if the fields you just tried to create appear in the List if you click on "Pick fields" to the left of your search results (check the values that were found)
  2. Splunk can help you generate the regex if you select "Extract Fields" from the context menu of an event that contains the values you want to extract into fields/variables. You can then test and save field extractions
  3. You could also use 3rd party tool to help you with your regexes (http://regex.larsolavtorvik.com/)

Once you have your fields you can append a reporting command to your search (then click on show report to format the report):

Popular destinations(pie chart:

| chart count(src) by dst

Active Sources (pie chart):

| chart count(dst) by src

Show when a source is active (line chart):

| timechart count(src) by src

Or you could also just create a table of your sources and destination tuples:

| rex field=_raw "User:\s+(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*\s+->\s+(?<dst>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" | where isnotnull(src) | table src dst | sort src

0 Karma

huaraz
Explorer

That worked
Thank you
Markus

0 Karma

Ayn
Legend

How are you creating the fields? As part of your search using the rex operator, through the field extractor or directly through a configuration file such as props.conf?

Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...