summation of fields

[creativenitin]

I have IIS webrequests logs which i want to parse to get the fields (websites and bytes) from the following:

1. 2012-05-10 18:39:29 GET /~site/Scripts_Shapes/Shapes.dll CMD=GetRectangleGif&r=0&g=0&b=0 172.17.187.252 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.30;+.NET+CLR+3.0.04506.648;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+InfoPath.2;+IPH+1.1.21.4019;+.NET4.0C;+.NET4.0E) http://xyz.com xyz.com 200 0 958 727 15

2. 2012-05-10 18:39:29 GET /~site/Scripts_Shapes/Shapes.dll CMD=GetRectangleGif&r=0&g=0&b=0 172.17.187.252 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.30;+.NET+CLR+3.0.04506.648;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+InfoPath.2;+IPH+1.1.21.4019;+.NET4.0C;+.NET4.0E) http://xyz.com xyz.com 200 0 859 727 15

3. 2012-05-10 18:39:29 GET /~site/Scripts_Shapes/Shapes.dll CMD=GetRectangleGif&r=0&g=0&b=0 172.17.187.252 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.30;+.NET+CLR+3.0.04506.648;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+InfoPath.2;+IPH+1.1.21.4019;+.NET4.0C;+.NET4.0E) http://abc.com xyz.com 200 0 958 727 15

4. 2012-05-10 18:39:29 GET /~site/Scripts_Shapes/Shapes.dll CMD=GetRectangleGif&r=0&g=0&b=0 172.17.187.252 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.30;+.NET+CLR+3.0.04506.648;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+InfoPath.2;+IPH+1.1.21.4019;+.NET4.0C;+.NET4.0E) http://xyz.com xyz.com 200 0 9581 727 15

5. 2012-05-10 18:39:29 GET /~site/Scripts_Shapes/Shapes.dll CMD=GetRectangleGif&r=0&g=0&b=0 172.17.187.252 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.30;+.NET+CLR+3.0.04506.648;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+InfoPath.2;+IPH+1.1.21.4019;+.NET4.0C;+.NET4.0E) http://def.com def.com 200 0 958 727 15

I want to get the fields in bold extracted. I have done that with the following rex.

| rex field=_raw ".?(?http[s]://.+)[\s|\t]+.?[\s|\t]+\d+[\s|\t]+\d+[\s|\t]+(?\d+)[\s|\t]+\d+[\s|\t]+\d+$" | search bytes="" website="*"

The values obtained are:

1. http://xyz.com 958
2. http://xyz.com 859
3. http://abc.com 958
4. http://xyz.com 9581
5. http://def.com 958

I need to get a table and then a chart as follows:
Table:

Website             Bytes

http://xyz.com    11398

http://abc.com    958

http://def.com    958

Basically this says that the website had sent the total number of bytes(adding the corresponding website bytes and then displaying the sum).

After this the website (x axis ) and the bytes (y axis) are to be plotted on a graph.

Can someone help me with this?

[sdaniels · Accepted solution]

I tried your rex but it didn't work for me so I just used the field extractor to create fields for the URL and bytes. Then run this command using the stats command and your field names.

sourcetype="testwebdata" | stats sum(bytes_test) by web_url

you can add a sort if you need it to look similar to your order | sort -web_url

Field Extractor example here - http://docs.splunk.com/Documentation/Splunk/latest/User/InteractiveFieldExtractionExample