Hi jbandautrgv, The easiest way to get the fields extracted on an xml is to use KV_MODE = xml in your props.conf If you are determined to use props / transforms... i believe this works: props: [directaccess:connections] NO_BINARY_CHECK = 1 TIME_FORMAT = %a %b %d %H:%M:%S %T %Y pulldown_type = 1 REPORT-xmlkv = xmlkv-alternative REPORT-xmlkv2 = xmlkv-alternative2 transforms: [xmlkv-alternative] REGEX = <([^\s\>]*)[^\>]*\>([^<]*)\<\/\1\> FORMAT = $1::$2 [xmlkv-alternative2] REGEX = <Data\sName='([^']+)'>([^<]+)<\/Data> FORMAT = $1::$2 This adds a second extraction that matches the Data Name bits Hope this helps ... View more

On your indexing layer, run the following from the command line: splunk btool indexes list <INDEXNAME> --debug Replacing with the name of the index that you are seeing issues with. There are a few properties to take note of: 1) coldPath.maxDataSizeMB -- The total size in MB of the Cold path for data. If this size is exceeded, data will roll to frozen (and if there is no Cold-To-Frozen archiving strategy in place, will be deleted) 2) frozenTimePeriodInSecs -- The number of seconds before data is frozen 3) maxTotalDataSizeMB -- The maximum total size across all hot/warm/cold data locations See if any of these are lower than you expect. ... View more

Well...if we're counting by sourcetype, let me throw this into the ring: | tstats count where index=_internal [| tstats count where index=_internal earliest=-60m by sourcetype | search count > 100 | fields - count | format] by _time span=1m sourcetype | timechart span=1m sum(count) as count by sourcetype Mind you, this won't work if you are not working with an OOB Accelerated, or data model, or indexed extraction..ed field... but in this specific case it's marvelous. ./DF ... View more

you didn't give the field a name and you didn't capture the period. Try something like: rex field=message"\d{3}\s\d{2}\s\-\s(?<Response_Time>[\d\.]+)\sms" ./DF ... View more

Here is my attempt: | makeresults | eval formatted_test_data="|<host1>C:,D: 500 GB,450 GB|<host2>E:,D: 250 GB,150 GB|<host3>C:,E: 250 GB,1 TB|<host4>S:,D:,C:,G: 120 GB,450 GB,250 GB,800 GB|" | rex field=formatted_test_data "\|(?<test_data>[^\|]+)" max_match=0 | table _time test_data | mvexpand test_data | rex field=test_data "<(?<host>\w+)>" | rex field=test_data "(?<drive>\w\:)" max_match=0 | rex field=test_data "(?<capacity>\d+\s[MGTP]B)" max_match=0 | eval drive_plus_capacity=mvzip(drive, capacity) | fields - test_data capacity drive | mvexpand drive_plus_capacity | rex field=drive_plus_capacity "(?<drive>\w\:)\,(?<freespace>\d+\s[MGTP]B)" | sort hostname drive | streamstats count as drivenum by hostname reset_on_change=true | chart list(drive_plus_capacity) over hostname by drivenum | rex field=1 "(?<drive1>\w\:)\,(?<freespace1>\d+\s[MGTP]B)" | rex field=2 "(?<drive2>\w\:)\,(?<freespace2>\d+\s[MGTP]B)" | rex field=3 "(?<drive3>\w\:)\,(?<freespace3>\d+\s[MGTP]B)" | rex field=4 "(?<drive4>\w\:)\,(?<freespace4>\d+\s[MGTP]B)" | fields - 1 2 3 4 ./DF ... View more

If your deployment server is forwarding its internal logs to your indexing layer, you should be able to use a query like: index=_internal host="Your_deployment_server_hostname" "/services/broker/phonehome/" | stats max(_time) AS last_checkin_epoch by clientip | eval now_epoch=now() | eval time_since_last_checkin=now_epoch-last_checkin_epoch | sort - time_since_last_checkin ./DF ... View more

Hi Saurabh0912, No, Changing your license pool will not cause any data to be deleted. What you may find, if the new pools are not set up correctly and your groups of indexers start to exceed the volume assigned to them, is that you could see new license exceeded warnings. Even that message is saying that all of the Indexers in the pool you are deleting, will lose the shared license volumes...not their data volumes. As long as, at the end of your exercise, every one of your indexers are in license pools that cover the intake volume of them and their license pool brethren, you should be good to go. ... View more

Hi newbie2tech, You need to create a mongo formatted query (since kvstore is a mongodb at its core, albeit a modified mongodb, so you can't just point a mongodb tool at it to manage it.) Like so, if you have a query to look at your kvstore like so: |inputlookup my_kv_store where LastUpdateTime<1551139200 in Mongodb query format that would look like so: {"LastUpdateTime": {"$lt": 1551139200}} Next, you need to take that and url encode it, using a tool like: http://meyerweb.com/eric/tools/dencoder/ which turns this: {"LastUpdateTime": {"$lt": 1551139200}} Into this: %7B%22LastUpdateTime%22%3A%20%7B%22%24lt%22%3A%201551139200%7D%7D Lastly, run the following from a command line that supports curl, and can access your Splunk search head: curl -k -u admin:changeme -X DELETE https://splunkhost:8089/servicesNS/nobody/<appname_where_kvstore_is_defined>/storage/collections/data/<kv_store_name>?query=%7B%22LastUpdateTime%22%3A%20%7B%22%24lt%22%3A%201551139200%7D%7D Making sure you replace the and placeholders with the actual values relevant to your environment and pasting your mongo query after the query= point in the command. Last point, measure twice, cut once.. Maybe create a test kvstore, put some dummy data in it and test this before you run it against your 3 years kvstore data and accidentally blow it all away. A backup of the kvstore might also be in order before this. (Call me paranoid). Good luck. Darren ... View more

Hi @mvagionakis, Here is my thought on a solution: | makeresults | eval testdata="^26/02/2020 16:34:21|toto|test|600|440|Session End|device=titi|sessionId=3bee772f147|ext=External^26/02/2020 16:34:21|toto|test|600|440|Upload|sessionId=3bee772f147|ext=External|username=mvag^" | rex field=testdata max_match=0 "\^(?<events>[^\^]+)" | table testdata events | fields - testdata | mvexpand events | rename events AS _raw | rex field=_raw "^(?<timestamp>[^\|]+)\|" | eval _time=strptime(timestamp, "%d/%m/%Y %H:%M:%S") | rex field=_raw "device=(?<device>\w+)\|" | rex field=_raw "sessionId=(?<sessionId>\w+)\|" | rex field=_raw "ext=(?<ext>\w+)" | rex field=_raw "([^\|]+\|){5}(?<action>[^\|]+)\|" | rex field=_raw "username=(?<username>\w+)" | eval starttime=if(action="Upload", _time, null) | eval endtime=if(action="Session End", _time, null) | stats values(username) min(starttime) AS starttime max(endtime) AS endtime by sessionId | eval sessionlength=endtime-starttime Explaination: | makeresults | eval testdata="^26/02/2020 16:34:21|toto|test|600|440|Session End|device=titi|sessionId=3bee772f147|ext=External^26/02/2020 16:34:21|toto|test|600|440|Upload|sessionId=3bee772f147|ext=External|username=mvag^" | rex field=testdata max_match=0 "\^(?<events>[^\^]+)" | table testdata events | fields - testdata | mvexpand events | rename events AS _raw | rex field=_raw "^(?<timestamp>[^\|]+)\|" | eval _time=strptime(timestamp, "%d/%m/%Y %H:%M:%S") | rex field=_raw "device=(?<device>\w+)\|" | rex field=_raw "sessionId=(?<sessionId>\w+)\|" | rex field=_raw "ext=(?<ext>\w+)" | rex field=_raw "([^\|]+\|){5}(?<action>[^\|]+)\|" | rex field=_raw "username=(?<username>\w+)" Everything up to this point is to create my run anywhere test data for your test case. In all likelihood, depending on how your sourcetype was created, you will not need to extract some of the fields (any with fieldname=fieldvalue should be extracted by themselves... I created the field "action" to get the "Session End" / "Upload" values | eval starttime=if(action="Upload", _time, null) | eval endtime=if(action="Session End", _time, null) This bit creates a field called starttime, for all the events where action=Upload (if you have more than one possible action that can signify the start of the transaction, this may need to be more detailed, either with a nested if, or a case) | stats values(username) min(starttime) AS starttime max(endtime) AS endtime by sessionId Here we group each sessionId by its starttime endtime and username combinations | eval sessionlength=endtime-starttime Lastly, i am using the difference between the start and end to set a duration. Let me know if this helps. . ./Darren ... View more

Do you have dev tools available? You could install sar from source. The source for it can be found at : https://github.com/sysstat/sysstat tar -xvf sysstat-12.1.4.tar.bz2 cd sysstat-12.1.4/ ./configure --enable-install-cron make make install ... View more

Try this: [answers786699] disabled = false DATETIME_CONFIG = CURRENT SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+)\*\*\*\*xxxfail TRUNCATE = 10000 SEDCMD-01-Remove_lines_part_1 = s/[\r\n]+(?!.*(tell_group\.pl)).*//g SEDCMD-02-Remove_lines_part_2 = s/^(?!.*(tell_group\.pl)).*[\r\n]//g Explanation: 1) ingest the whole file as a single event... This is done with this line: LINE_BREAKER = ([\r\n]+)\*\*\*\*xxxfail Which tells splunk to only break when it reaches a carriage return followed by the exact string "****xxxfail" . If your files could be larger than 10000 lines, then also adjust the "TRUNCATE =" to be larger than your largest file (and probably include a buffer above that)... In the unlikely event that you do have ****xxxfail in your data, just change this to be an even more ridiculous and unlikely string... like It\sturns\sout\sthat\sthe\searth\sis\sflat or something 2) Remove all lines that don't have "tell_group.pl" somewhere in the line. This is accomplished with the three SEDCMD lines .. they operate as follows: SEDCMD-01-Remove_lines_part_1 = s/[\r\n]+(?!.*(tell_group\.pl)).*//g This removes all lines from the file that do not have tell_group.pl in them ... When this line is applied by itself, the above file ingests as so: ROWS tell_group.pl MSG "NUMBER OF ALTCARRIER_A TABLE UPDATES REQUIRED THIS RUN : $MEDSA_AMINF" + tell_group.pl MSG NUMBER OF ALTCARRIER_A TABLE UPDATES REQUIRED THIS RUN : 1245 tell_group.pl MSG "NUMBER OF ALTCARRIER_B TABLE UPDATES REQUIRED THIS RUN : $MEDSB_AMINF" + tell_group.pl MSG NUMBER OF ALTCARRIER_B TABLE UPDATES REQUIRED THIS RUN : 350 tell_group.pl MSG "NUMBER OF ALTCARRIER_C TABLE UPDATES REQUIRED THIS RUN : $MEDSC_AMINF" + tell_group.pl MSG NUMBER OF ALTCARRIER_C TABLE UPDATES REQUIRED THIS RUN : 164 tell_group.pl MSG "NUMBER OF ALTCARRIER_D TABLE UPDATES REQUIRED THIS RUN : $MEDSD_AMINF" + tell_group.pl MSG NUMBER OF ALTCARRIER_D TABLE UPDATES REQUIRED THIS RUN : 0 That first regex will work on all lines except the first line in the file (and it leaves a bunch of empty lines as well). To get rid of those, i used a variation of the first SEDCMD, only with the [\r\n]+ at the end of the match. SEDCMD-02-Remove_lines_part_2 = s/^(?!.*(tell_group\.pl)).*[\r\n]//g after this is done, we are left with: tell_group.pl MSG "NUMBER OF ALTCARRIER_A TABLE UPDATES REQUIRED THIS RUN : $MEDSA_AMINF" + tell_group.pl MSG NUMBER OF ALTCARRIER_A TABLE UPDATES REQUIRED THIS RUN : 1245 tell_group.pl MSG "NUMBER OF ALTCARRIER_B TABLE UPDATES REQUIRED THIS RUN : $MEDSB_AMINF" + tell_group.pl MSG NUMBER OF ALTCARRIER_B TABLE UPDATES REQUIRED THIS RUN : 350 tell_group.pl MSG "NUMBER OF ALTCARRIER_C TABLE UPDATES REQUIRED THIS RUN : $MEDSC_AMINF" + tell_group.pl MSG NUMBER OF ALTCARRIER_C TABLE UPDATES REQUIRED THIS RUN : 164 tell_group.pl MSG "NUMBER OF ALTCARRIER_D TABLE UPDATES REQUIRED THIS RUN : $MEDSD_AMINF" + tell_group.pl MSG NUMBER OF ALTCARRIER_D TABLE UPDATES REQUIRED THIS RUN : 0 Which i believe answers your requirements. Hope this helps ./Darren ... View more

Is there a timestamp anywhere in the file or should the props just use the index time? ... View more

The minimum required capabilities to run the debug | refresh are: edit_scripted refresh_application_licenses web_debug (If you have just web_debug, it will allow you to see the refresh button, but when you press it, it throws a gnarly 5XX error, with these two others, everything runs fine.) ... View more

That would depend... If you are setting up the multi-site environment because you are wanting DR or for the purpose of ensuring redundancy then you would likely want to have the monitoring setup with redundancy as well. That way if you lose a site, your monitoring will continue unfettered. If you are setting up the multi-site environment to deal with network latency between sites or to enable search / forwarder affinity and are not worried about DR, then a single MC will likely be sufficient. ... View more

ファイルの数またはイベントの数をカウントしますか？ ... View more

A join with type=left means show all values in the original search whether there is a match in the subsearch or not. (which is why i filled null just after the join) A join without the type... would only show rows where they exist in both searches...so an error that has occurred in the past, but not in the subsearch timeframe would not show up. ... View more

and....did it? ... View more

Here is my attempt... rex field=full_log "File\s\'(?<pathname>.+[\/\\])[^\\\/]+\'\schecksum\schanged\." https://regex101.com/r/87ro6z/1 ... View more

I am a little confused about your line breaking question, so i am assuming a second event with the same format will follow what you have pasted in, and so the line breaker is the newline following a line that starts with a timestamp (see https://regex101.com/r/uB6tJJ/1 )... This also uses the Local_Time as the timestamp for the event. [sourcetypename] disabled = false LINE_BREAKER = [\r\n]\s+?\d{4}\-\d{2}\-\d{2}T\d{2}\:\d{2}\:\d{2}\.\d{3}\s.+([\r\n]+) SHOULD_LINEMERGE = false TIME_PREFIX = Local\sTime\:\s+ TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3n MAX_TIMESTAMP_LOOKAHEAD = 25 TRUNCATE = 10000 ... View more

I usually go with a props/transforms/nullQueue for these type of situations where the field names are known # props.conf [782506] disabled = false TIME_PREFIX = ^ TIME_FORMAT = %Y-%m-%d MAX_TIMESTAMP_LOOKAHEAD = 15 LINE_BREAKER = ([\r\n]+) SHOULD_LINEMERGE = false INDEXED_EXTRACTIONS = csv TRANSFORMS_01_killheader = Delete_csv_header with #transforms.conf [Delete_csv_header] disabled = false REGEX = ^time\,interface\,utilization DEST_KEY = queue FORMAT = nullQueue This resulted in two events, no header, and field extractions indexed. ... View more