Hello splunkers. I am new to splunk and have a question on how to change index for events that e.g. have status 404 on index time?
props.conf
[weblogs]
LINE_BREAKER = (&&&)
NO_BINARY_CHECK = true
REPORT-access = access-extractions
SHOULD_LINEMERGE = false
maxDist = 28
...
TRANSFORMS-change = notfound,changesourcetype
transforms.conf
[notfound]
REGEX = ".+?"\s(404)
DEST_KEY = MetaData:Index
FORMAT = index::notfoundindex
[changesourcetype]
DEST_KEY = MetaData:Sourcetype
REGEX = ^(.*)
FORMAT = sourcetype::access_combined
example of event:
141.146.8.66 - - [13/Jan/2016 21:03:09:200] "POST /category.screen?category_id=SURPRISE&JSESSIONID=SD1SL2FF5ADFF3 HTTP 1.1" 200 3496 "http://www.myflowershop.com/cart.do?action=view&itemId=EST-16&product_id=RP-SN-01" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.38 Safari/533.4" 294
Changing of sourcetype works fine, but index changing doesn't and I really do not know where the mistake is.
Hi @yudzhin ,
At first I don't understand why you want to put 404 events in a different index!
Remember that usually in Splunk are used different indexes basing on two parameters:
You can differentiate events using sourcetype or another field, don't think to Splunk as a DB table with fields.
Anyway, I see two things probably not correct:
TRANSFORMS-notfound = notfound
TRANSFORMS-change = changesourcetype
Anyway, if in your sample 200 is the posizion of 404, you could try this regex in notfound stanza
\"\s+404\s+
Ciao.
Giuseppe
Problem solved just by re-typing transforms.conf stanzas, most probably some unexpected character was present.
thank's all for help 🙂
@richgalloway I have to do some labs and there is a task to do so 🙂
Hi @yudzhin ,
At first I don't understand why you want to put 404 events in a different index!
Remember that usually in Splunk are used different indexes basing on two parameters:
You can differentiate events using sourcetype or another field, don't think to Splunk as a DB table with fields.
Anyway, I see two things probably not correct:
TRANSFORMS-notfound = notfound
TRANSFORMS-change = changesourcetype
Anyway, if in your sample 200 is the posizion of 404, you could try this regex in notfound stanza
\"\s+404\s+
Ciao.
Giuseppe
Hello @gcusello and thank's for response.
example of event with 404
69.80.0.18 - - [13/Jan/2016 21:03:08:169] "GET /product.screen?product_id=K9-CW-01&JSESSIONID=SD7SL2FF10ADFF4 HTTP 1.1" 404 2038 "http://www.myflowershop.com/cart.do?action=view&itemId=EST-1&product_id=K9-CW-01" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" 261
Unfortunatelly changing both regex and separate TRANSFORMS didn't help, after data is uploaded all events are present for webindex, not for notfoundindex.
BTW: I am doing some labs for SE2 certification and I have such task. I understand this has no sense, but I have to do it 🙂
Hi @yudzhin ,
the regex I hinted is correct so try it.
To debug your problem, try if each transformation runs by itself.
Anyway, thinks to your idea to have different indexes!
Ciao.
Giuseppe
@gcusello TRANSFORMS-notfound = notfound is not working separately.