Getting Data In

Apache Access Log Extract Byte Size

doprocess
Engager

I have tons of log lines coming from the Apache access log that look something like this:

11/19/19 1:39:01.000 PM 192.168.1.1 - - [19/Nov/2019:13:39:01 -0500] "GET /jquery/jquery-ui.min.js HTTP/1.1" 200 240427
11/19/19 1:38:36.000 PM 192.168.1.1 - - [19/Nov/2019:13:38:36 -0500] "GET /21ab251b74e7f7c26.js HTTP/1.1" 200 11331980

The last number on each line represents the size of the file in bytes.
I need to extract that last number and chart it, to find the total bandwidth going through my server.
How do I go about this?

0 Karma
1 Solution

doprocess
Engager

Mayurr98, that won't work, size is not defined.

After a bit of testing, I've ended up creating my own query, that takes everything after the last empty space (after 9th character we have a size in bytes) and used it like this to get the bandwidth that is used by .js or .css files, plotting it on 1-hour based timechart:

GET 200 (.js OR .css)
| rex ^(\S*\s){9}(?<byte_size>.*)$
| bucket _time span=1h
| timechart sum(byte_size) as "Bandwidth (GB)"
| eval "Bandwidth (GB)"=round('Bandwidth (GB)'/1024/1024/1024/6,2)

This works like a charm, I hope someone finds this useful.

View solution in original post

0 Karma

doprocess
Engager

Mayurr98, that won't work, size is not defined.

After a bit of testing, I've ended up creating my own query, that takes everything after the last empty space (after 9th character we have a size in bytes) and used it like this to get the bandwidth that is used by .js or .css files, plotting it on 1-hour based timechart:

GET 200 (.js OR .css)
| rex ^(\S*\s){9}(?<byte_size>.*)$
| bucket _time span=1h
| timechart sum(byte_size) as "Bandwidth (GB)"
| eval "Bandwidth (GB)"=round('Bandwidth (GB)'/1024/1024/1024/6,2)

This works like a charm, I hope someone finds this useful.

0 Karma

mayurr98
Super Champion

hi to extract it try this:

| rex "(?<size>\d+$)"

to calculate the total size by server try this:

  | rex "(?<size>\d+$)" | stats sum(size) as size_in_bytes by server
0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Onboarding New Hires & ...

This is the fifth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Now Playing: Splunk Education Summer Learning Premieres

It’s premiere season, and Splunk Education is rolling out new releases you won’t want to miss. Whether you’re ...

The Visibility Gap: Hybrid Networks and IT Services

The most forward thinking enterprises among us see their network as much more than infrastructure – it's their ...