Okay MModestino, that is good suggestion I will try to do both(hope my hadoop team agrees for it :)) and take a call from there ... View more

Hi MModestino, In you Hadoop roll usage suggestion you mentioned "there is a free app that you can use to allow Hadoop tools to read the journal", how effective are these? Do you know of any implementations? My usecase requirement is to send data to hadoop such that hadoop consume it and do reporting by correlating this transferred data with it's own existing data from other sources. With this app can hadoop consume the data? Also this would mean we need to install splunk enterprise and then the Bucket Reader app on the Hadoop Cluster right? Our Kerberized Hadoop Cluster has 2 master nodes and 4 data nodes..so the splunk enterprise and bucket reader app should be installed on just one of the master nodes? ... View more

Thanks A lot cpetterborg, these rex did the tricks, now I pretty much have what I need. I will use your queries instead of mine for the usecase. Thx for introducing me to the power of rex there by creating urge to learn it. Thx a ton. ... View more

please ignore my question on "ERROR" case sensitivity, I understand it is case sensitive and we can leave it that way. Also Thanks a lot for your suggestions they have been very helpful, appreciate it. ... View more

Hi cpetterborg, I agree my approach isn't good and is difficult to maintain and understand but my lack of rex skills made me take that route, my query does not provide proper stats data because the events which I posted I had trimmed or changed the text to hide company specific methods and information hence the variation. I am glad you have taken out time to further look into the data and sharing the rex. Your second rex with some assumption is working great pretty much extracting all the patterns except below ones which have numbers or alpha-numerics which makes the event unique and they show up as individual events causing the stats to spread across multiple pages[rowlimit=100]. Is there any way to ignore numbers or alphanumerics inside () or [] on the end of existing 2nd query to make them unique. I just want to count the occurrence of the Error which is pretty much all events which have that word. end of your second rex I just did | stats count by condStr In your rex "ERROR" will this be case sensitive? can we make case-insensitive. 2nd rex output which has numbers or alphanumerics causing it to show multiple times privilleges response [0085c9bd-b791-44d3-b868-fba063917477] has failed. 1 privilleges error(s) to report Call doodleeRestService.isRefreshAllInProgress(100004444, accounts) failed Call doodleeRestService.startRefresh(100004555) failed with com.yodlee.soap.core.refresh.refresh.InvalidRefreshModeExceptionFaultMessage: InvalidRefreshModeExceptionFaultMessage Call doodleeRestService.startRefreshAll(100004993) failed Illegal argument [CookieAuthenticationPrincipal{cookie='AMCV_73FAC51D54C72AE50A4C98BC%40AdobeOrg=-227196251%7CMCIDTS%7C17351%7CMCMID%7C61491506829527716652284815389133322820%7CMCAAMLH-1499698553%7C7%7CMCAAMB-1499698553%7CcIBAx_aQzFEHcPoEv0GwcQ%7CMCOPTOUT-1499100953s%7CNONE%7CMCAID%7CNONE; s_mtouch=DirectTypeIn; s_gnr=1499093753262-New; s_lv=1499093753266; ADRUM=s=1499353681830&r=https%3A%2F%2FKBC-beta.abc.com%2Fwma%3F0; SESSION=T0xTOjE0OTkzNTM1OTI2NTI6MTQ5OTM2Nzk5MjY1Mjp0cnVlOk9MUw; JSESSIONID=0000hbnsrhh7RwtLuguTaoS3lIp:16aoeo112; abc_KBC_AUTH=j8s6hJ2W0BtD32mR707ZyGsN53eBl1s2j6aNfP31iW622Z1R3136ePf+6I4o+o+EnHDO/x2UNSmqOCkYeXEpGk1SpEndZ1r2PjJj7iD32aWo8UGfU+sjPivmEyGqLYfzUZUag6uMLVp4MwiBFV46RBBEQnmY+OfokjdwjxvEi8ne5YrtFre8ywfm9SJGetfzUiyCnl92rNKBXCEUJ1yr9ztn/X+IG+CEaWl7bsScUywMsHSr60YlJOP+Oc6f4a79OOU6kgWI; ADRUM_BTa=R:63|g:5eb46bbb-1235-4569-8962-dd4616e667fc|n:customer1_3f5d5ee2-41b3-4e59-ad10-de0b93053644; ADRUM_BT1=R:63|i:2180|e:1|d:1', profileid='W1234', hashedprofileid='d122432bac27ddf62ba1848d219fd9ed', authLevel='10', impersonation=true, Id='8E4', sessionId='2017-07-06-11:06:24-471210352'}, com.abc.mma.accounts.dataservice.restinterface.domain.modifyname.modifyNameModificationRequest@6dbee2e8] in modifyAccounts() ... View more

Hi cpetterborg/agarrison, Please refer my below answer for details on my actual query and some sample events. Wanted to ask if you can give a shot to above query in my original post using sample events to see if it can be achieved,thank you in advance. ... View more

[this is not an answer, comment has character limitation hence posting it here] Hi cpetterborg/agarrison, Here is my actual dashboard query, I have various errors under same index , same sourcetype. I have gone through distinct errors in the last 1month data and worked on extracting them using the substr and trying to group them as there is no recurring pattern. I am familiar with field extraction however not good with writing regex. Also listed some of the error patterns are listed below the query, unfortunately can't list them all. Also the text to be extracted is listed after the event, look for text after [EXTRACT] word. index=abc eventtype=err0r |rex field=_raw "(?<shortmessage>.*\s)" | rex field=_raw ".{65}(?<message>.{33})" |eval length=len(shortmessage) |eval length500=substr(_raw,280,30) |eval length499=substr(_raw,279,30) |eval length498=substr(_raw,280,30) |eval length476=substr(_raw,280,30) |eval length464=substr(_raw,178,105) |eval length463=substr(_raw,177,105) |eval length440=substr(_raw,277,88) |eval length439=substr(_raw,276,88) |eval length418=substr(_raw,244,29) |eval length399=substr(_raw,277,122) |eval length398=substr(_raw,276,122) |eval length333=substr(_raw,203,51) |eval length332=substr(_raw,202,51) |eval length331=substr(_raw,194,36) |eval length330=substr(_raw,193,36) |eval length321=substr(_raw,178,88) |eval length300=substr(_raw,176,122) |eval length299=substr(_raw,176,121) |eval length279=substr(_raw,245,39) |eval length278=substr(_raw,178,79) |eval length277=substr(_raw,175,78) |eval length269=substr(_raw,193,53) |eval length268=substr(_raw,192,53) |eval length264=substr(_raw,188,74) |eval length263=substr(_raw,187,74) |eval length261=substr(_raw,194,33) |eval length260=substr(_raw,193,33) |eval length257=substr(_raw,194,25) |eval length256=substr(_raw,193,25) |eval length247=substr(_raw,193,54) |eval length246=substr(_raw,192,54) |eval length245=substr(_raw,191,54) |eval length240=substr(_raw,180,59) |eval length237=substr(_raw,203,33) |eval length236=substr(_raw,202,33) |eval length234=substr(_raw,104,50) |eval length233=substr(_raw,193,41) |eval length232=substr(_raw,192,41) |eval length231=substr(_raw,184,51) |eval length230=substr(_raw,183,48) |eval length226=substr(_raw,192,34) |eval length225=substr(_raw,191,34) |eval length224=substr(_raw,188,36) |eval length223=substr(_raw,188,35) |eval length221=substr(_raw,188,107) |eval length220=substr(_raw,187,107) |eval length210=substr(_raw,186,45) |eval length209=substr(_raw,185,45) |eval length208=substr(_raw,184,45) |eval length207=substr(_raw,182,25) |eval length206=substr(_raw,192,43) |eval length202=substr(_raw,183,57) |eval length194=substr(_raw,51,94) |eval length193=substr(_raw,89,102) |eval length190=substr(_raw,157,34) |eval length189=substr(_raw,156,34) |eval length188=substr(_raw,181,13) |eval length187=substr(_raw,180,13) |eval length175=substr(_raw,99,53) |eval length170=substr(_raw,95,54) |eval length169=substr(_raw,94,54) |eval length164=substr(_raw,81,83) |eval length162=substr(_raw,96,32) |eval length161=substr(_raw,95,32) |eval length157=substr(_raw,95,25) |eval length153=substr(_raw,99,55) |eval length152=substr(_raw,112,41) |eval length151=substr(_raw,111,41) |eval length148=substr(_raw,94,54) |eval length147=substr(_raw,93,55) |eval length146=substr(_raw,92,54) |eval length141=substr(_raw,93,54) |eval length140=substr(_raw,92,54) |eval length138=substr(_raw,81,56) |eval length137=substr(_raw,81,52) |eval length134=substr(_raw,81,53) |eval length133=substr(_raw,84,49) |eval length132=substr(_raw,81,52) |eval length131=substr(_raw,80,51) |eval length128=substr(_raw,94,43) |eval length127=substr(_raw,94,33) |eval length126=substr(_raw,93,33) |eval length124=substr(_raw,87,45) |eval length125=substr(_raw,88,45) |eval length123=substr(_raw,98,25) |eval length121=substr(_raw,88,117) |eval length122=substr(_raw,89,117) |eval length116=substr(_raw,92,35) |eval length117=substr(_raw,93,35) |eval length113=substr(_raw,81,42) |eval length112=substr(_raw,80,42) |eval length109=substr(_raw,55,64) |eval length108=substr(_raw,54,64) |eval length107=substr(_raw,93,43) |eval length103=substr(_raw,66,36) |eval length102=substr(_raw,65,36) |eval length89=substr(_raw,68,30) |eval length88=substr(_raw,65,30) |eval length62=substr(_raw,137,146) |eval length48=substr(_raw,49,42) |eval condStr = if( length=500,length500, if( length=499,length499, if( length=498,length498, if( length=476,length476, if( length=464,length464, if( length=463,length463, if( length=440,length440, if( length=439,length439, if( length=418,length418, if( length=399,length399, if( length=398,length398, if( length=333,length333, if( length=332,length332, if( length=331,length331, if( length=330,length330, if( length=321,length321, if( length=300,length300, if( length=299,length299, if( length=279,length279, if( length=278,length278, if( length=277,length277, if( length=268,length268, if( length=268,length268, if( length=264,length264, if( length=263,length263, if( length=261,length261, if( length=260,length260, if( length=257,length257, if( length=256,length256, if( length=247,length247, if( length=246,length246, if( length=245,length245, if( length=240,length240, if( length=237,length237, if( length=236,length236, if( length=234,length234, if( length=233,length233, if( length=232,length232, if( length=231,length231, if( length=231,length231, if( length=230,length230, if( length=226,length226, if( length=225,length225, if( length=224,length224, if( length=223,length223, if( length=221,length221, if( length=220,length220, if( length=210,length210, if( length=208,length208, if( length=209,length209, if( length=207,length207, if( length=206,length206, if( length=202,length202, if( length=193,length193, if( length=194,length194, if( length=195,length195, if( length=189,length189, if( length=190,length190, if( length=187,length187, if( length=188,length188, if( length=175,length175, if( length=169,length169, if( length=170,length170, if( length=164,length164, if( length=165,length165, if( length=162,length162, if( length=161,length161, if( length=157,length157, if( length=158,length158, if( length=153,length153, if( length=151,length151, if( length=152,length152, if( length=148,length148, if( length=147,length147, if( length=146,length146, if( length=141,length141, if( length=140,length140, if( length=138,length138, if( length=137,length137, if( length=134,length134, if( length=133,length133, if( length=136,length136, if( length=132,length132, if( length=131,length131, if( length=128,length128, if( length=127,length127, if( length=126,length126, if( length=125,length125, if( length=124,length124, if( length=123,length123, if( length=121,length121, if( length=122,length122, if( length=116,length116, if( length=117,length117, if( length=112,length112, if( length=113,length113, if( length=108,length108, if( length=109,length109, if( length=107,length107, if( length=102,length102, if( length=103,length103, if( length=88,length88, if( length=89,length89, if( length=62,length62, if( length=48,length48, if( length <48 OR (length>48 and length<62) OR (length>62 and length<88) OR (length>89 AND length<102) OR (length>103 AND length<107) OR (length>109 and length<112) OR (length>113 and length<116) OR (length>117 and length<121) OR (length>128 AND length<131) OR (length>134 AND length<136) OR (length>138 AND length<140) OR (length>141 AND length<146)OR (length>148 AND length<151) OR (length>153 AND length<157)OR (length>157 AND length<161) OR (length>161 AND length<164) OR (length>165 AND length<169) OR (length>170 AND length<175) OR (length>175 AND length<187) OR (length>187 AND length<189) OR (length>190 AND length<193) OR (length>194 AND length<202) OR (length>202 AND length< 206) OR (length>210 AND length<220) OR (length=222) OR (length>226 AND length<230) OR length=235 OR (length>237 AND length<240) OR (length>240 AND length<245) OR (length>247 AND length<256) OR (length>257 AND length<260) OR length=262 OR (length>264 AND length<268)OR (length>269 AND length<277) OR (length>279 AND length<299) OR (length>300 AND length<321) OR (length>321 AND length<330) OR (length>333 AND length<398) OR (length>399 AND length<418) OR (length>418 AND length<439) OR (length>440 AND length<463) OR (length>464 AND length<476) OR (length>476 AND length<498) OR length>500 OR length=100 OR length=106 OR length=48 OR length=49 OR length=53,message, message )))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) |eval condStr=trim(condStr) |stats count by length condStr ***Sample Events*** 2017-06-10 04:15:02.982 ERROR [http-nio-0.0.0.0-19777-exec-42 - ExceptionHandlerImpl] Access denied Missing required privilleges: [restful-service] -->[EXTRACT] Access denied Missing required privilleges 2017-06-22 17:30:12.865 ERROR [asyncTaskExecutor-7 - RentService] Async call failed to return rent account -->[EXTRACT] Async call failed to return rent account 2017-07-06 03:37:27.307 ERROR [http-nio-0.0.0.0-19717-exec-5 - inteAdapter] Invalid URI encoding; using HTTP default -->[EXTRACT] Invalid URI encoding '2017-06-23 04:20:15.181 ERROR [http-nio-0.0.0.0-19747-exec-18 - ExceptionHandlerImpl] Handled HttpServerError -->[EXTRACT] Handled HttpServerError '2017-06-17 17:54:35.946 ERROR [http-nio-0.0.0.0-40712-exec-1 - InsideAccountsService] Failed to update accounts, retryCount: 9, time spent updating: 107.844s -->[EXTRACT] Failed to update accounts 2017-06-17 12:05:40.487 ERROR [http-nio-0.0.0.0-40712-exec-12 - InsideAccountsService] Call doodleeRestService.isRefreshAllInProgress(1000000, accounts) failed -->[EXTRACT] doodleeRestService.isRefreshAllInProgress 2017-06-17 18:16:38.036 ERROR [http-nio-0.0.0.0-40707-exec-2 - DefaultRestExceptionHandler] Error occurred during execution of controller method. Returning status 500 org.springframework.web.client.HttpServerErrorException: 500 Internal Server Error at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:19) at-->[EXTRACT]500 Internal Server Error 2017-06-17 07:02:30.195 ERROR [http-nio-0.0.0.0-20011-exec-16 - LoggingHandler] Cause : com.java.app.imp.errorhandling.HttpServerErrorExtendedException: associationID: f982269c-1107-4f5a-bce0-efca885354ae SessionID: 2017-06-17-03:01:50-499526829 500 Internal Server Error Request URI: GET https://myapp-accounts:27/appsd/v1/account?context=NOT_AVAILABLE&typeIdFlags=offline_ACCOUNT Request Body: null >>>>>>>>>>>>>>>>>>>>>>>>>> RESPONSE BODY START <<<<<<<<<<<<<<<<<<<<<<<<<< {"timestamp":1497682950183,"statusCode":500,"message":"Internal Server Error","details":null} >>>>>>>>>>>>>>>>>>>>>>>>>> RESPONSE BODY END <<<<<<<<<<<<<<<<<<<<<<<<<< -->[EXTRACT]Internal Server Error Request '2017-07-07 14:13:00.915 ERROR [http-nio-0.0.0.0-19727-exec-31 - ExceptionHandlerImpl] SessionID 2017-07-07-10:09:07-682308382 associationID: Handled HttpClientError Exception associationID: bf7a1a3c-ad36-4b18-ba63-a4d571be9632 SessionID: 2017-07-07-10:09:07-682308382 404 Not Found Request URI: GET https://appsd-community-data-service.she.pwj.com:224/ds/appsd/community/JD1234 Request Body: null >>>>>>>>>>>>>>>>>>>>>>>>>> RESPONSE BODY START <<<<<<<<<<<<<<<<<<<<<<<<<< {""timestamp"":""2017-07-07T14:13:00.863Z"",""status"":404,""error"":""Not Found"",""message"":""No message available"",""path"":""/ds/appsd/community/JD1234""} >>>>>>>>>>>>>>>>>>>>>>>>>> RESPONSE BODY END <<<<<<<<<<<<<<<<<<<<<<<<<<-->[EXTRACT] 404 Not Found Request URI 2017-06-29 19:57:28.818 ERROR [http-nio-0.0.0.0-40772-exec-38 - [dispatcherServlet]] Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.time.format.DateTimeParseException: Text '' could not be parsed at index 0] with root cause java.time.format.DateTimeParseException: Text '' could not be parsed at index -->[EXTRACT]Request processing failed; nested exception is java.time.format.DateTimeParseException 2017-07-10 20:15:21.891 ERROR [http-nio-0.0.0.0-40707-exec-3 - PrincipalPrivillegeChecker] SessionID 2017-07-10-16:05:38-805878958 associationID:67378330-709b-4228-a991-fee5efbb9a08 Entitlement response [null] has failed with error [privilleges Error - Code [400] Description [Unexpected System Error - Request timeout]] -->[EXTRACT] Entitlement response [null] has failed with error '2017-07-01 10:46:10.164 ERROR [http-nio-0.0.0.0-20011-exec-2 - LoggingHandler] SessionID 2017-07-01-06:45:33-827946996 associationID:cf53860f-9c89-f54520ff Cause : java.lang.RuntimeException: java.util.concurrent.ExecutionException: com.java.app.imp.errorhandling.HttpServerErrorExtendedException: associationID: cf53860f-9c89-49ed-910b-f567a74520ff SessionID: 2017-07-01-06:45:33-827946996 500 Internal Server Error Request URI: GET https://myapp-accounts:20207/appsd/v1/accountgroup Request Body: null "-->[EXTRACT]500 Internal Server Error ... View more

Hi cpetterborg, Thank you for your inputs. I cannot use field extraction as I have 100+ different unstructured errors and we do not have any common error identifying patterns so I had to use extrapolated version of above query which i have shared. It works fine except that the rollup does not happen for all occurrences of error description and same description is being displayed more than once as specified in my sample results. I suspect it to be due to varying length(hence I used trim), will continue to check further. Thank you for your time, your suggestion does not work for me. Any other thoughts? ... View more

thank you javiergn...your second search worked for my scenario..been trying since few hours ... ... View more

Was this resolved? If so can you share some details, also was it Kerberos secured cluster? ... View more

Apologies for delay in getting back, I had to put this usecase on hold due to other priorities, now am back to executing this. Thank you for the pointers. Does the data roll approach delete the data from splunk index or just makes a copy or we have option to choose either of these? I see that we have the ability to read the archived data in hadoop from splunk . ... View more

check if the user has access to the indexes involved in the searches ... View more

Thank you somesoni2...updated regex worked perfectly for all my scenarios...thanks a ton. ... View more

Hi Sweenj, I hope you have made the suggested changes to transforms.conf and props.conf on the indexer(and bounce it), if NOT go ahead and do them on indexer. Once done , on your search head run below command by selecting "Verbose Mode" the dropdown next to search icon which displays "Fast Mode" "Smart Mode" "Verbose Mode". Once you run below command with "Verbose Mode" and the search complete, look for interesting fields on the left hand side and you should see the fields which you listed in the transforms.conf. sourcetype="forecast" host="node" Check and let us know ... View more

thank you woodcock for pointing me to these apps, I would check them however I would not be able to install and use them in production for current problem, I will upvote the response for guidance. ... View more

Thank you somesoni2 for the response, it is working in most of the cases except below scenarios[my bad as these were not part of original ask], can the same regex be tweaked to accommodate these as well. I am accepting the answer as you made this complex scenario deal with simple regex. Hope below scenarios can be accommodated as well. A forward slash at the end of desired output is not mandatory, no harm if we have on, if it makes any easier. /home/clothing/dry-fit/000000274/ --> should be /home/clothing/dry-fit /home/women-athelete/kobe/22540S836/--> should be /home/women-athelete/kobe /home/search/products? --> should be /home/search/products and NOT /home/search /home/search/products/?id=home --> should be /home/search/products and NOT /home/search /home/account-summary/purchases --> should be /home/account-summary/purchases and NOT /home/account-summary /purchases --> should be captured, current regex ignores this /home/products/all --> should be captured, current regex makes it /home/products/ ... View more

thanks @somesoni2 ,date_wday=strftime(_time,"%w-%A") suggestion from your query helped crack my usecase ... View more

Hi Sweenj, Are you not seeing key value pairs in the interesting fields in verbose mode? the event might appear as single string with commas but you should have the fields created and you should be able to use them in your search query. Also can you share information on your architecture, all of this is on one single server or you have got search head, indexer and the server where your are trying to forward from? ... View more

Hi Masonmorales, desired output is what I meant by higher roll up, I want to display count for these navigations. Essentially all the URI patterns which have digits or ? are specific to individuals , we want to only consider until digit or ? is encountered. For example in our logs, we will have 3 values for uri field as below /Home/men-kids/summerwear/1059770200 /Home/men-kids/summerwear/1059770201 /Home/men-kids/summerwear/1059770202 While we just want to count it as --> uri count /Home/men-kids/summerwear 3 desired output URI Count /Home/ 50 /Home/account-settings/ 100 /Home/men/clothing-360/ 20 /Home/men-kids/summerwear/ 123 /Home/women-athelets/shoes 100 /Home/article/reviews/ 200 /Home/account-summary or /Home/account-summary? 220 /Home/search or /Home/search? 112 ... View more

Try escaping it using \ Solace_JMS_VPN=sysvpn*,* Solace_JMS_SSL_Truststorepassword=axd=jkf, Solace_JMS_SSL_Keystore OR Solace_JMS_VPN=sysvpn*,* Solace_JMS_SSL_Truststorepassword=axd=\jkf, Solace_JMS_SSL_Keystore ... View more

What is your db connect version? Are you using batch or tailing? If batch, is the first schedule run completed? Also when searching in index try searching for all time as data can go to past date due to date formats messup. ... View more