Hello fellow splunkers, right now I'm working through the 7 labs for SE II which are necessary to be able to start the finishing accreditation quiz. I've been able to finish 5 of them by now but am totally lost with lab 6. here the instructions are: - events should begin with <Interceptor> and end with </Interceptor> (so Linebreaking is needed) - Extract (at search time) all fields and values in between the Interceptor lines and throw away any of the header lines before the first <Interceptor> and the line after the very last </Interceptor> - Use the ActionDate and ActionTime field as the timestamp - have Splunk auto extract the fields and values   how they say I'd know I've done it: - I'll have x amount of events and the fields broken out using SPATH notation - the correct timestamp - no text before the first and after the last Interceptor   What I have so far: - I'm able to extract ActionDate and ActionTime to create a new timestamp - I'm able to linebreak with LINE_BREAK = \<Interceptor\>()   My Issue: - When I linebreak I save the new sourcetype and try to proceed to alter it given the other things to do like extract timestamp or delete the header text. but when I change ANYTHING it just disregards the linebreaker argument and goes back to be one huge event again and I can't do anything about it. - even if I could linebreak and extract everything as stated, I don't really understand what they mean with "broken out using SPATH mean". do they mean via SPL ? cause they clearly stated that Splunk should "auto extract the fields and values" How the data looks: <?xml version="1.0" encoding="UTF-8" ?><dataroot><Interceptor><AttackCoords>-80.33100097073213,25.10742916222947</AttackCoords><Outcome>Interdiction</Outcome><Infiltrators>23</Infiltrators><Enforcer>Ironwood</Enforcer><ActionDate>2013-04-24</ActionDate><ActionTime>00:07:00</ActionTime><RecordNotes></RecordNotes><NumEscaped>0</NumEscaped><LaunchCoords>-80.23429525620114,24.08680387475695</LaunchCoords><AttackVessel>Rustic</AttackVessel></Interceptor><Interceptor><AttackCoords>-80.14622349209523,24.53605142362535</AttackCoords><Outcome>Interdiction</Outcome><Infiltrators>6</Infiltrators><Enforcer>Cunningham</Enforcer><ActionDate>2013-04-26</ActionDate><ActionTime>00:23:00</ActionTime><RecordNotes></RecordNotes><NumEscaped>0</NumEscaped><LaunchCoords></LaunchCoords><AttackVessel>Raft</AttackVessel></Interceptor><Interceptor><AttackCoords>-80.75496221688965,24.72483828554483</AttackCoords><Outcome>Interdiction</Outcome><Infiltrators>11</Infiltrators><Enforcer>Forthright</Enforcer><ActionDate>2013-05-15</ActionDate><ActionTime>23:35:00</ActionTime><RecordNotes></RecordNotes><NumEscaped>0</NumEscaped><LaunchCoords>-79.65932674368925,23.70743135623052</LaunchCoords><AttackVessel>Rustic</AttackVessel></Interceptor><Interceptor><AttackCoords>-80.32020594311533,25.02156920297054</AttackCoords><Outcome>Interdiction</Outcome><Infiltrators>6</Infiltrators><Enforcer>Pompano</Enforcer><ActionDate>2013-02-25</ActionDate><ActionTime>15:35:00</ActionTime><RecordNotes></RecordNotes><NumEscaped>0</NumEscaped><LaunchCoords></LaunchCoords><AttackVessel>Raft</AttackVessel></Interceptor><Interceptor><AttackCoords>-80.15149489716094,24.57412215015249</AttackCoords><Outcome>Interdiction</Outcome><Infiltrators>6</Infiltrators><Enforcer>Tripoteur</Enforcer><ActionDate>2013-04-13</ActionDate><ActionTime>15:40:00</ActionTime><RecordNotes></RecordNotes><NumEscaped>0</NumEscaped><LaunchCoords>-79.65999190070923,23.73619147168514</LaunchCoords><AttackVessel>Raft</AttackVessel></Interceptor></dataroot> I hope someone can help understand how to proceed here. EDIT: in Lab 4 there was almost the same data to input - the only difference is that in lab6 it has no linebreaks whatsoever. here is my props.conf from lab4: [dreamcrusher] DATETIME_CONFIG = FIELD_HEADER_REGEX = <Interceptor> LINE_BREAKER = \<Interceptor\> MAX_DAYS_AGO = 4000 NO_BINARY_CHECK = true category = Custom disabled = false pulldown_type = true REPORT-actiondate = actiondate EVAL-_time = strptime(ActionDate +" " + ActionTime,"%Y-%m-%d %H:%M:%S") and my transforms.conf: #[actiondate] #REGEX = \<ActionDate\>(?P<ActionDate>\d+-\d+-\d+)\<\/ActionDate\>\s*\<ActionTime\>(?P<ActionTime>\d+:\d+:\d+) #FORMAT = $1::$2   ... View more

Hello fellow splunkers!   atm I'm trying to break up a huge multiline event that is merged together with &&&. When I try to explicitly tell Splunk to BREAK_ONLY_AFTER = &&& it doesn't work. I also tried BREAK_ONLY_BEFORE = \d+.\d+.\d+.\d+\s-\s- and BREAK_ONLY_AFTER = \d{3}&&& it seems that nothing I try works. please help here is the source log:      141.146.8.66 - - [13/Jan/2016 21:03:09:200] "POST /category.screen?category_id=SURPRISE&JSESSIONID=SD1SL2FF5ADFF3 HTTP 1.1" 200 3496 "http://www.myflowershop.com/cart.do?action=view&itemId=EST-16&product_id=RP-SN-01" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.38 Safari/533.4" 294&&&130.253.37.97 - - [13/Jan/2016 21:03:09:185] "GET /category.screen?category_id=BOUQUETS&JSESSIONID=SD7SL2FF1ADFF8 HTTP 1.1" 200 2320 "http://www.myflowershop.com/cart.do?action=changequantity&itemId=EST-12&product_id=AV-CB-01" "Opera/9.20 (Windows NT 6.0; U; en)" 361&&&141.146.8.66 - - [13/Jan/2016 21:03:09:167] "GET /product.screen?product_id=RP-LI-02&JSESSIONID=SD9SL9FF8ADFF1 HTTP 1.1" 200 3855 "http://www.myflowershop.com/cart.do?action=changequantity&itemId=EST-20&product_id=RP-LI-02" "Googlebot/2.1 ( http://www.googlebot.com/bot.html) " 929     ... View more

this is how my xml events look like:   <AttackCoords>-80.33100097073213,25.10742916222947</AttackCoords> <Outcome>Interdiction</Outcome> <Infiltrators>23</Infiltrators> <Enforcer>Ironwood</Enforcer> <ActionDate>2013-04-24</ActionDate> <ActionTime>00:07:00</ActionTime> <RecordNotes></RecordNotes> <NumEscaped>0</NumEscaped> <LaunchCoords>-80.23429525620114,24.08680387475695</LaunchCoords> <AttackVessel>Rustic</AttackVessel>   I didn't find a good explanation on how to do this. my painpoint is that I don't know how to glue the values from ActionDate and ActionTime together so I can generate a _time field out of it.  so, what I have :    <ActionDate>2013-04-24</ActionDate> <ActionTime>00:07:00</ActionTime>   what I want:   _time = 2013-04-24 00:07:00   I hope anyone can help ... View more

I've been trying to extract fields from a log at search time with only the help of props.conf. in the spunk docu I read that EXTRACT would be good in that case, especially cause I try to extract multiple fields at once.    this is my EXTRACT in props.conf so far:     EXTRACT-fields = (?P<src_ip>\d*\.\d*\.\d*\.\d*)(?=\ \d* TCP_)\s(?P<bits>\d*)\s(?P<tcp_state>\w*_\w*)\s     this would go on for a while with different fields but it doesn't work. what do I do wrong?  this is how the log looks like for example:      2014-03-27 12:39:32 20 10.71.15.207 304 TCP_HIT 367 1470 GET http www.computerworld.com 80 /elqNow/elqFCS.js - - - - 23.196.74.53 application/x-javascript http://www.computerworld.com/s/article/9247206/Gameover_malware_takes_aim_at_Monster.com_and_CareerBuilder.com "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)" OBSERVED "Technology/Internet" - 163.252.254.201 23.44.202.53 52809       Thanks a lot for any help! ... View more

I'm trying to extract multiple fields out of my log. my problem is that I do have multiplie ip adresses - one for the source, one from the webserver etc. so to counter having extracted four ip adresses in every event into the same field I want to use a positive lookahead to tell the regex "yes this ip but only if afterwards there comes x and y" this is an example log:   2014-03-27 23:54:58 1 10.5.6.121 304 TCP_HIT 422 501 GET http assets.razerzone.com 80 /eeimages/products/13785/razer-naga-2014-right-03.png - - - - 54.230.18.168 image/png http://imgur.com/gallery/u3o7l "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Iron/31.0.1700.0 Chrome/31.0.1700.0 Safari/537.36" OBSERVED "Technology/Internet;Shopping" - 163.252.254.203 - 54351 2014-03-28 23:54:59 90670 10.62.0.120 200 TCP_NC_MISS 601 693 GET http realtime.services.disqus.com 80 /api/2/thread/2221828111 ?bust=1780 - - - realtime.services.disqus.com application/json http://disqus.com/embed/comments/?base=default&disqus_version=6c05c0ca&f=bootsnipp&t_i=9WgD&t_u=http%3A%2F%2Fbootsnipp.com%2Fsnippets%2Ffeatured%2Fminimal-preview-thumbnails&t_d=Viewing%20snippet%20Minimal%20Preview%20Thumbnails%20%7C%20Bootsnipp.com&t_t=Viewing%20snippet%20Minimal%20Preview%20Thumbnails%20%7C%20Bootsnipp.com&s_o=default "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.154 Safari/537.36" OBSERVED "Newsgroups/Forums" - 163.252.254.203 184.173.90.195 52401     here I want  for example extract the source_ip (the first ip in the event) like this:   (\d+\.\d+\.\d+\.\d+)[[:blank:]](?=\d*)    still this gives me many other ips... so I think I have to use multiple lookaheads? but I really don't know how to use it in that case. Especially when I want to extract the ip and give it a field name. In my head this means that I have to encapsulate EVERYTHING including the lookaheads ?    thanks a lot for your help! ... View more

I've tried using props.conf.spec and transforms.conf.spec and some regex to extract a value from a logfile in order to use it as my hostname value. I see that with my regex I can extract the given value but I have two problems:    1.) when I use the gui to get data in I can only choose a given value for hostname pre indexing or use regex only for the path in which my logfile lies. When I put in my tested regex in the hostname field it ofc doesn't work.  So I guess I first have to set up the sourcetype in props.conf and configure the extraction in transforms.conf 2.) I can't seem to find an explanation on how to configure the extraction correctly. like I said the regex seems okay but in transforms I seem to need the following fields which I don't know how to use: SOURCE_KEY DEST_KEY FORMAT   In the Logfile it looks similar to that  (the host value is "DC1ASM1.dc1.greendotcorp.com"):   Sep 20 11:13:36 10.50.3.100 Sep 20 11:13:33 DC1ASM1.dc1.greendotcorp.com ASM:"MONEYPAK_WEBAPP","MONEYPAK_CLASS","Blocked","Attack signature detected","4523972057501657341","207.154.35.240","GET /Content/Images/img_logo04_module02.gif HTTP/1.1\r\nHost:...   mostly it's this host name. and I want to extract  and use it as hostname at indextime.  this is what I did so far: props.conf:   [f5asm] BREAK_ONLY_BEFORE = \w+ \d+ \d+:\d+:\d+ \d+ BREAK_ONLY_BEFORE_DATE = DATETIME_CONFIG = LINE_BREAKER = \w+ \d+ \d+:\d+:\d+ \d+ NO_BINARY_CHECK = true TIME_FORMAT = %b %d %H:%M:%S TIME_PREFIX = \d+.\d+.\d+.\d+ category = Custom disabled = false pulldown_type = true TRANSFORMS-hostname = changehost   transforms.conf   [changehost] DEST_KEY = MetaData:Host SOURCE_KEY = MetaData:Host REGEX = ([a-zA-Z0-9]([a-zA-Z0-9\-]{0,61}[a-zA-Z0-9])?\.)+[a-zA-Z]{2,6} +?(?=ASM)FORMAT = host::$1   I'm fairly certain that I have to change up something in transforms.conf but I can't seem to find an answer. any ideas how to set up the FORMAT, DEST_KEY and SOURCE_KEY correctly in that case?  ... View more