Splunk Search

search by http response code

shangshin
Builder

Hi, I would like to search status=304 or 500 in web server's access log but the search result is empty.
Here is one sample line in the access log. Any advice? Thanks in advance!

2012-04-23      10:56:51        10.39.19.101    "user_D"      "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E; InfoPath.2)"       GET     /ask/faq.html    -       304     0       0.0010
Tags (1)
1 Solution

kristian_kolb
Ultra Champion

I'm assuming that the "304" in your sample line is the status code you want.

Do you have it extracted as a field? If not - you'll need to extract it to make it searchable. You can still search for 304 or 500 or 404 as strings, but then you might also get results where the number is found elsewhere in the event.

For guidance on field extraction, see the docs pages for the Interactive Field Extractor
http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Addfieldsatsearchtime

When you have the field extracted (e.g. as status) you can simply search for it;

sourcetype=your_sourcetype status=304 OR status=500

UPDATE:

I'm not sure I can help you with IFX problems, but you can make these field extractions directly in the props.conf file. Just put the following EXTRACT lines under the stanza for your sourcetype.
Please note that there are several props.conf files, but unless you have a heavy forwarder reading the log files and sending them to the indexer, you could/should create/edit /opt/splunk/etc/system/local/props.conf on the indexer. Just make sure that you have the correct ownership/permissions on the file if you create a new one.

[your_sourcetype]
EXTRACT-wl_status = (?<status>\S+)\s+\S+\s+\S+$
EXTRACT-wl_timetaken = (?<time_taken>\S+)$

Your search would look something like;

sourcetype=your_sourcetype NOT status=200 OR time_taken > 90

Hope this helps,

Kristian

View solution in original post

sowings
Splunk Employee
Splunk Employee

Looks like that data is separated by tabs. I'd suggest:

props.conf:


[your_sourcetype_here]
REPORT-myfields = wl_kv_and_fields

Then in transforms.conf:


[wl_kv_and_fields]
DELIMS = "\t"
FIELDS = date, time, c-ip, proxy-remote-user, user-agent, cs-method, cs-uri-stem, cs-uri-query, sc-status, bytes, time-taken

sowings
Splunk Employee
Splunk Employee

Operator AND is implied. If you don't specify, Splunk assumes AND.

The AND and OR operators must be in all caps to differentiate them from search terms.

In your second example above, Splunk thinks you're also looking for the word "and" in a weblogic_access_log where proxy_remote_user="my_user_id".

shangshin
Builder

This is exactly what we need!!! Many thanks.
Now the backend is set up. One quick question for the search operator:

When should I add the operator AND

Result is returned as expected:
sourcetype="weblogic_access_log" proxy_remote_user="my_user_id"

No result is returned:
sourcetype="weblogic_access_log" and proxy_remote_user="my_user_id"

0 Karma

sowings
Splunk Employee
Splunk Employee

Well, if you're writing the transforms.conf, you can "get it right the first time" by naming the fields whatever you like; each item in the FIELDS list will act as the new field name for the extracted fields.

0 Karma

rcovert
Path Finder

You may also need to set field aliases.

kristian_kolb
Ultra Champion

I'm assuming that the "304" in your sample line is the status code you want.

Do you have it extracted as a field? If not - you'll need to extract it to make it searchable. You can still search for 304 or 500 or 404 as strings, but then you might also get results where the number is found elsewhere in the event.

For guidance on field extraction, see the docs pages for the Interactive Field Extractor
http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Addfieldsatsearchtime

When you have the field extracted (e.g. as status) you can simply search for it;

sourcetype=your_sourcetype status=304 OR status=500

UPDATE:

I'm not sure I can help you with IFX problems, but you can make these field extractions directly in the props.conf file. Just put the following EXTRACT lines under the stanza for your sourcetype.
Please note that there are several props.conf files, but unless you have a heavy forwarder reading the log files and sending them to the indexer, you could/should create/edit /opt/splunk/etc/system/local/props.conf on the indexer. Just make sure that you have the correct ownership/permissions on the file if you create a new one.

[your_sourcetype]
EXTRACT-wl_status = (?<status>\S+)\s+\S+\s+\S+$
EXTRACT-wl_timetaken = (?<time_taken>\S+)$

Your search would look something like;

sourcetype=your_sourcetype NOT status=200 OR time_taken > 90

Hope this helps,

Kristian

shangshin
Builder

Thanks a lot. field extractor for status code is GOOD! But strangely, time_taken doesn't seem to work because nothing is returned from the search.

I also tried to create the index for the user_id on column 4 using
EXTRACT-wl_userid = ^\S+\s+\S+\s+\S+\s+(?\S+)\s+

I would apprecaite if you can shed some light on this so we can demonstarte a usful product.

0 Karma

kristian_kolb
Ultra Champion

see update above. /k

0 Karma

shangshin
Builder

It's a weblogic access log file and the column headers are

date time c-ip cs(proxy-remote-user) cs(user-agent) cs-method cs-uri-stem cs-uri-query sc-status bytes time-taken

We would like to get alerts if status code (sc-status) is not 200 or time-taken is greater than 90 seconds. This is a real use case and I would appreciate if you can provide an example of field extractor.

0 Karma

kristian_kolb
Ultra Champion

Hmm, what kind of log are you working on (post some sample data)? Not really familiar with the error message, are you using transforms?

Show some sample events and tell us what you want to extract.

/k

0 Karma

shangshin
Builder

Thanks -- Kristian.

I followed example in the docs page and tried to add a new error code field. However, I got the error message when hitting the save button. Can you shed some light on this or provide a sample for this real use case?

Encountered the following error while trying to save: In handler 'props-extract': cannot find transform name=device_id=[w+](?<err_code>[^:]+)

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...