About neusse

neusse · ‎06-16-2020

It looks like you are doing an all time search. If the data is coming in at a good rate then they would not have the same results. it would be impossible for them to see the same exact data. If you change the search to look for a particular hour of data and the two searches use that same hour then it should match. You would not match if you searched relative for the last few hours as well. If the start and end are the same then the results should be the same.

neusse · ‎05-06-2020

I downvoted this post because this is not helpful or insightful.

neusse · ‎05-06-2020

Related to the question and I think a proper answer. I have played around with and think this is a very good tool. It can be run as a one shot as well as being run to keep real version control without having to edit in GIT. It is called "Versioncontrol for Splunk" and can be found on Splunkbase. https://splunkbase.splunk.com/app/4355/ It is possible to write an exporter that uses the REST API to do an export of any of the Splunk knowledge objects. I have done this myself and something like that would be the simplest. Here is curl example that will export a the dashboard from a given app. The output can be used to inport the dashboard back if needed and would be similar to what "versioncontrol for Spunk" does. curl -k -u username:password https://localhost:8089/servicesNS/admin/search/data/ui/views/my_dashboard There is another API call to get a list of all the dashboards in an app. Also if you omit the dashboard name in the API you will get all the dashboards in one output file. A good toll to play with these endpoints is "insomnia", it is free and is feature rich. Also here is a link to the docs. https://docs.splunk.com/Documentation/Splunk/8.0.3/RESTUM/RESTusing The REST API can do just about anything in splunk and this is how splunk itself does most things. I hope this helps people get going with exporting knowledge objects from splunk.

neusse · ‎10-28-2016

Is there a way to detect gaps in the middle like the chart shows above? So the red is two time spans. I can think of a way to detect that. If the data is always overlapping this works. But my data is not always overlapping. Thanks,

neusse · ‎10-28-2016

I need to roll up several events with overlapping start and stop times. I need the total time of the events without doing a sum of the elapsed time for each event. Each event in the series has a start time and an end time. The times may or may not overlap. The results need to be one or more time spans that equal the earliest and latest for each consecutive amount of time. The green are the event time spans and the red is the needed result time. I am at a loss as to how to approach this. Thanks.

neusse · ‎03-08-2011

Yes there is a solution. I found nothing anywhere describing this in splunk.com. My issue of not being able to search into the event deep enough at index time was solved by using the simple command LOOKAHEAD in transforms.conf. Turns out splunk does not look far ahead related to REGEX at all when indexing. It seems to only be looking for the end ofthe transaction as a priority. Here are my working props.conf and transforms.conf: transforms.conf [nomore] LOOKAHEAD = 100000 REGEX=(?m)(404\sNot\sFound) DEST_KEY=queue FORMAT=nullQueue props.conf [mod_security] SHOULD_LINEMERGE = true MUST_NOT_BREAK_AFTER = (--[a-z0-9]+-A--) MUST_BREAK_AFTER = (--[a-z0-9]+-Z--) TRUNCATE = 0 TRANSFORMS-notfounderror = nomore At least it is working now!

neusse · ‎03-08-2011

Yes there is a solution. I found nothing anywhere describing this in splunk.com. My issue of not being able to search into the event deep enough at index time was solved by using the simple command LOOKAHEAD in transforms.conf. Turns out splunk does not look far ahead related to REGEX at all when indexing. It seems to only be looking for the end ofthe transaction as a priority. Here are my working props.conf and transforms.conf: transforms.conf [nomore] LOOKAHEAD = 100000 REGEX=(?m)(404\sNot\sFound) DEST_KEY=queue FORMAT=nullQueue props.conf [mod_security] SHOULD_LINEMERGE = true MUST_NOT_BREAK_AFTER = (--[a-z0-9]+-A--) MUST_BREAK_AFTER = (--[a-z0-9]+-Z--) TRUNCATE = 0 TRANSFORMS-notfounderror = nomore At least it is working now!

neusse · ‎03-04-2011

I replaced all my stuff with yours. No go. It still indexes ALL Events. It is not regexing past the first line. If I change the match from 404 to -A-- like the first line, then Nothing indexes. But anything other than the first line and it never matches. I wish it was as simple as this or I would have been done a week ago.

neusse · ‎03-03-2011

The above was a test. If it could match the last section then I would have no data indexed. But it is indexing ALL data.

neusse · ‎03-03-2011

I want to throw away the entire even. What I really want is to eliminate events that have "404 Not Found" in side them. But I cant get the regex to match past the first line. The events are Mod_Security logs. Each event has sections that start with "--ab123cd-A--" A-Z for all sections. The last section has no data but marks the end of the event "-ab123cd-Z--" Within section H there is the error for a "404 Not Found" I would like to match. But I thought I could just match the error text. But the regex only matches against the first line. (m?) has no effect. I am stumped at this point.

neusse · ‎03-03-2011

I am trying to match text inside a large multi line Event. I have the index working ok. But in transforms.conf it fails to match anything past the first line. I was able to verify this my matching . and then a regex that matched the first line. Then I tried to match the last line and it fails yet the data ends up in the index. Here are my props.conf and my transforms.conf: props.conf [mod_security] LEARN_MODEL = false sourcetype = mod_security TRUNCATE = 0 #SHOULD_LINEMERGE = true MUST_NOT_BREAK_AFTER = (--[a-z0-9]+-A--) MUST_BREAK_AFTER = (--[a-z0-9]+-Z--) TRANSFORMS-nomore = nomore transforms.conf [nomore] REGEX=(m?)--[a-z0-9]+-Z-- DEST_KEY=queue FORMAT=nullQueue As a test I tried to match the last line. It fails. But if I match the first line of the event or . it works. I makes no sense. Also I have tried both with and without SHOULD_LINEMERGE. Thanks in advance to any help. Here is an example Event as requested. --6c7cd57c-A-- [12/Jan/2011:10:59:29 --0600] TS3d8UijBKywIAAABu 53047 99.99.99.99 80 --6c7cd57c-B-- GET /pcgi-bin/sreg2/register/0 HTTP/1.1 X-NATPath: 99.99.99.99:53047, 99.99.99.99, 99.99.99.99:60136 Host: www.neusse.com X-Forwarded-For: 99.99.99.99, 99.99.99.99 Accept: */* Accept-Language: en-us User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.1; MS-RTC LM 8) True-Client-IP: 99.99.99.99 Pragma: no-cache X-Akamai-CONFIG-LOG-DETAIL: true Accept-Encoding: gzip Akamai-Origin-Hop: 2 Via: 1.1 v1-akamaitech.net(ghost) (AkamaiGHost), 1.1 akamai.net(ghost) (AkamaiGHost) Cache-Control: no-cache, max-age=0 SM_AUTHTYPE: Auto SM_SDOMAIN: .neusse.com Max-Forwards: 10 X-Forwarded-Host: www.neusse.com X-Forwarded-Server: www.neusse.com --6c7cd57c-F-- HTTP/1.1 404 Not Found Last-Modified: Wed, 25 Oct 2006 03:10:45 GMT ETag: "3cdd" Accept-Ranges: bytes Content-Length: 15581 CDCHOST: neusse-prod1-203 Content-Type: text/html X-Pad: avoid browser bug CDCWPB: neusse-prod1-02 CDCXRP: neusse-prod1-01 Connection: close --6c7cd57c-H-- Apache-Handler: proxy-server Stopwatch: 1294851569931732 32492 (168 9176 -) Producer: ModSecurity v2.1.7 (Apache 2.x) Server: Apache/2.2 --6c7cd57c-Z--

neusse · ‎03-02-2011

I made the change and it males no deference. I still get ALL events indexed. It seems this is a wide spread problem for folks, It would be nice if splunk had a little better method for filtering since there is a lot of noise in many logs. I am at an impasse with this. For the volume of data, we need to filter to make better use of analysis time and system resources.

neusse · ‎03-01-2011

I am trying to filter with many transform statements. I believe everything is configured correctly. But I get ALL events indexed. None are going to the nullQueue. Please help! It seems easy enough but I am not getting this. As I understand it: In my props.conf I have two sections. One is [mod_security] it does a few things for parsing the event and collects some fields. Also in my props.conf I have a [source::/some/path/to/directory/]. This has all the transform statements that are checking if it should send the event to the nullQueue. What happens is all events are indexed. None are filtered. I have also tried adding all the transform statements to the [mod_security] section. But I get the same, everything is indexed, issue. Last I wanted to check that a couple of strings exist in the event after filtering. If they do then index the event. If not send the event to the nullQueue. Here are my props.conf and transforms.conf. Any help would be appreciated. Props.conf [mod_security] TRUNCATE = 0 SHOULD_LINEMERGE = true BREAK_ONLY_BEFORE = (--[a-z0-9]+-A--) REPORT-get = get REPORT-post = post REPORT-severity = severity [source::/nas/log/cache/httpd-www-80/] TRANSFORMS-f1 = f1 TRANSFORMS-f2 = f2 TRANSFORMS-f3 = f3 TRANSFORMS-f4 = f4 TRANSFORMS-f5 = f5 TRANSFORMS-f6 = f6 TRANSFORMS-ok = null,ok ========================================= transforms.conf [get] REGEX = (GET.+?)$ FORMAT = AA_get::$1 [post] REGEX = (POST.+?)$ FORMAT = AA_post::$1 [severity] REGEX = (severity.+?)\] FORMAT = AA_severity::$1 [f1] REGEX = (99\.99\.99\.38) DEST_KEY = queue FORMAT = nullQueue [f2] REGEX = .*404\sNot\sFound.* DEST_KEY = queue FORMAT = nullQueue [f3] REGEX = .*401\sUnauthorized.* DEST_KEY = queue FORMAT = nullQueue [f4] REGEX = .*500\sInternal\sServer\sError.* DEST_KEY = queue FORMAT = nullQueue [f5] REGEX = .*403\sForbidden.* DEST_KEY = queue FORMAT = nullQueue [f6] REGEX = FILTERED\sTO\s DEST_KEY = queue FORMAT = nullQueue [null] REGEX = . DEST_KEY = queue FORMAT = nullQueue [ok] REGEX = (Pattern\smatch)|(Matched\ssignature) DEST_KEY = queue FORMAT = indexQueue

Posts	21
Solutions	2
Karma Given	7
Karma Received	13
Member Since	‎03-01-2011

Online Status	Offline
Date Last Visited	‎09-19-2024 01:43 PM

How to derive earliest and latest from a list of e...

Service not available error.

Transforms REGEX will not search past first line o...

Yet another filtering problem. Many transforms in ...

Re: Why would the SPL run in the Search App give d...

Re: How can I backup and maintain a version of das...

Re: How can I backup and maintain a version of das...

Re: How to derive earliest and latest from a list ...

How to derive earliest and latest from a list of e...

Re: Yet another filtering problem. Many transforms...

Re: Transforms REGEX will not search past first li...

Re: Transforms REGEX will not search past first li...

Re: Transforms REGEX will not search past first li...

Re: Transforms REGEX will not search past first li...

Transforms REGEX will not search past first line o...

Re: Yet another filtering problem. Many transforms...

Yet another filtering problem. Many transforms in ...