Environment : Distributed Splunk Enterprise (indexer cluster) Version: 9.0.5 Issue: After setting journalCompression to zstd in indexes.conf, we noticed that the setting is applied for warm but not for frozen buckets. The setting was applied months ago. In the following example, we can see that files timestamped from today are zst in warm and gzip in frozen. I did not find any related information in documentation indexesconf Is it an expected behavior or am I missing some setting in my configuration? Evidence: ## WARM BUCKETS [splunk@indexer (PROD) ~]$ ls -latr /var/lib/splunk/warm/<index_name> [...] drwx--x---. 3 splunk splunk 4096 Apr 30 11:19 db_1714450734_1714041906_2521_1B4FA1BE-AA81-459F-B38A-1FB23A018EDB [splunk@indexer (PROD) ~]$ ls -latr /var/lib/splunk/warm/<index_name>/db_1714450734_1714041906_2521_1B4FA1BE-AA81-459F-B38A-1FB23A018EDB/rawdata/ [...] -rw-------. 1 splunk splunk 113295494 Apr 30 11:19 journal.zst ## FROZEN BUCKETS [splunk@indexer (PROD) ~]$ ls -latr /var/lib/splunk/frozen/<index_name> [...] drwx------. 3 splunk splunk 29 Apr 30 11:20 rb_1709121660_1709115460_2204_3BF8DDF1-9874-4848-9DB4-880DA5EBA00F [splunk@indexer (PROD) ~]$ ls -latr /var/lib/splunk/frozen/<index_name>/rb_1709121660_1709115460_2204_3BF8DDF1-9874-4848-9DB4-880DA5EBA00F/rawdata/ [...] -rw-------. 1 splunk splunk 2342045 Feb 28 19:08 journal.gz ... View more

My propfs.conf file for my app looks like the following : [bit_fuse_log] pulldown_type = true NO_BINARY_CHECK = true TIME_PREFIX = ^ TIME_FORMAT = %H:%M:%S\,%3Q MAX_TIMESTAMP_LOOKAHEAD = 12 SHOULD_LINEMERGE = false BREAK_ONLY_BEFORE_DATE = true TRUNACTE = 0 [bit_fuse_access_proxy] SHOULD_LINEMERGE = false NO_BINARY_CHECK = true MAX_TIMESTAMP_LOOKAHEAD = 32 disabled = false TRUNCATE = 0 MAX_EVENTS = 10240 and my log files look like the following (note that every line does not start with a blank space) <texttexttext> <text> <text>texttexttext</text> <text>text text text</text> </text> </texttexttext> </text> </text> </text> </texttext> -------------------------------------- 11:35:11,715 | INFO | 12345678-12345 | texttexttexttext | 107 - texttexttexttexttexttext | Outbound Message --------------------------- ID: xxxxxxx Address: http://url Encoding: UTF-8 Http-Method: POST Content-Type: text/xml Headers: {Accept=[*/*], SOAPAction=[""]} Payload: <texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext> <texttexttext> <texttexttexttexttexttexttexttexttexttexttexttext"texttexttexttexttexttexttexttexttexttext"> <texttext> <text>text</text> <text>text</text> <text/> <text/> <text>text</text> <text>001</text> <text/> <text>texttext</text> <OPERATION>QueryCardDtlsLst</OPERATION> <SOURCE_OPERATION/> <SOURCE_USERID/> [...] -------------------------------------- 08:20:22,972 | INFO | 12345678-234567890 | texttexttexttext | texttexttexttexttexttexttexttexttext | Outbound Message --------------------------- ID: 1234 Response-Code: 200 Content-Type: application/json Headers: {Content-Type=[application/json], Date=[Mon, 06 Jan 2020 01:20:22 GMT]} Payload: {"texttexttext":{"texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext} -------------------------------------- 08:20:24,862 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,862 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,865 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,865 | WARN | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,865 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,865 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,865 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,866 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,867 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,867 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla ---------------------------- ID: 123456 Address: texttexttexttexttexttexttexttext Encoding: texttext Http-Method: POST Content-Type: application/json Headers: {Accept=[application/json], texttexttexttexttexttexttexttexttexttexttexttexttexttexttext} Payload: {"startDate":"2020-01-01T00:00:00.000+0700","endDate":"2020-01-06T23:59:59.000+0700","pageNumber":1,"pageSize":300} -------------------------------------- 08:20:24,862 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,862 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,865 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla 08:20:24,865 | WARN | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla <text> I would like to break at every timestamp at the beginning of line (every arrow in the following picture). I don't understand Splunk's behavior considering my props.conf file. My issue is that Splunk sometimes breaks as expected and sometimes not. The most common behavior is : Breaking before Header Not breaking single line consecutive events I can't find a pattern for this behavior, do you have an idea ? ... View more

Hi, I would like to break my logs at every time + log level but it is not working as expected. Here's my props.conf : [log_name] TIME_FORMAT = %H:%M:%S,%3N MAX_TIMESTAMP_LOOKAHEAD = 12 NO_BINARY_CHECK = true disabled = false MAX_EVENTS = 10240 TRUNCATE = 0 [other_name] SHOULD_LINEMERGE = false NO_BINARY_CHECK = true MAX_TIMESTAMP_LOOKAHEAD = 32 disabled = false TRUNCATE = 0 MAX_EVENTS = 10240 My log file : 09:31:51,359 | INFO | 640512999-933058 | someinformation | 72 - text.texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,359 | WARN | 640512999-933058 | someinformation | 204 - texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,359 | INFO | 640512999-933058 | someinformation | 72 - texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,359 | INFO | 640512999-933058 | someinformation | 72 - texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,359 | INFO | 640512999-933058 | someinformation | 243 - texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,360 | INFO | 640512999-933058 | someinformation | 243 - texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,361 | INFO | 640512999-933058 | someinformation | 240 - I tried to use BREAK_ONLY_BEFORE option in props.conf with the following regexp : (?:[01]\d|2[0-3]):(?:[0-5]\d):(?:[0-5]\d),[0-9]{3}.\|.*?\| Same behavior with LINE_BREAKER option. Can someone give me a hint on how to configure my props.conf ? What am I missing ? ... View more

Hello, I'm having a little trouble solving this one. I managed to extract all hosts in Splunk in a table with events counted by path with the following search : search index=* | rex field=source "(?<file_path>.*\\\)" | rex field=source "(?<file_path>.*\/)" | chart count over file_path by host limit=0 | fields - source In the other hand, I have a full inventory of our network in a csv file. I would like to be able to filter Splunk hosts by Location (this Location field is present in the full inventory csv). I believe that I have to use the Lookup command but I can't find out how to use it. (Both the csv and the search table have a hostname field) The Splunk query above gives a table like this : ------------+-------+-------+-------+-----+ | Path1 | Path2 | Path3 | ... | ------------+-------+-------+-------+-----+ Host_1 | 100 | 0 | 200 | ... | Host_2 | 250 | 1200 | 2 | ... | Host_3 | 0 | 10 | 0 | ... | ------------+-------+-------+-------+-----+ And I would like to add a Location column with information from the CSV to look like this : ------------+--------+-------+-------+-------+-----+ |Location| Path1 | Path2 | Path3 | ... | ------------+--------+-------+-------+-------+-----+ Host_1 | USA | 100 | 0 | 200 | ... | Host_2 | U.K | 250 | 1200 | 2 | ... | Host_3 | USA | 0 | 10 | 0 | ... | ------------+--------+-------+-------+-------+-----+ Thank you for your help ! ... View more