About Jakub

Jakub · ‎09-22-2021

Hello All, My issue is: We are receiving files from Source1, where are more types of logs. We want to split them and send them into another indexes. Issue is that there are no some unique fields in the logs, so its difficult to create REGEX in transform.conf. Sep 21 08:48:29 10.128.38.16 2021-09-21 06:48:28.7004|INFO|Robot|21.09.2021 08:46:21|21.09.2021 08:48:28|AttendedBot|OK|Account: .... Hence we have decided to put FLAG there. -- "Y@9?" Sep 21 08:48:29 10.128.38.16 2021-09-21 06:48:28.7004|INFO|Y@9?|Robot|21.09.2021 08:46:21|21.09.2021 08:48:28|AttendedBot|OK|Account: .... And now I would like to trigger this flag in transform.conf (lets say send it to new index) and after that remove it by SEDCMD. Everything works fine, SEDCMD and transforms.conf when they are not configured at the same time, but when they are, the SEDMCD is applied first so transfrom.conf has nothing to trigger. Question is: Is there possibility to apply transform.conf before SEDCMD? Thank you very much.

Jakub · ‎08-05-2021

Hi @kamlesh_vaghela , thank you so much for your answer. But it does not work. I was thinking if it can't be below issue/solution. (I am collecting it via HTTP Event Collector) Solved: HEC: How to set _time on base of a specific JSON f... - Splunk Community But my SEDCMD works correctly and I was able to remove "log": property from events, hence the props.conf in HF has to work correctly and its not case of the "event" end point of HEC. Also SEDCMD is only thing which works. SEDCMD-RemoveLogProp = s/("log":)(.*)(?="stream":)// Could it be that INDEXED_EXCTRACTION is OFF and KV_MODE is ON in search? Thank you very much.

Jakub · ‎08-04-2021

Hi All, I have Event timestamp with miliseconds: _time with Unix epoch seconds: and during search the timestamp is from _time, and I would like to have it with milliseconds. I am using KV_MODE in Search cluster props.conf. [k8s:dev] KV_MODE = json and I am trying to do changes in HF props.conf , like TIME_FIELDS, TIME_PREFIX, TIME_FORMAT, but none of them work. INDEXED_EXCTRACTION is turned OFF in HF props.conf HF props.conf [k8s:dev] #temporary removed to fix https://jira/browse/DEVA-61153 #INDEXED_EXTRACTIONS = JSON #TIME_PREFIX = {\\"@timestamp\\":\\" TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N TIMESTAMP_FIELDS = @timestamp TRUNCATE = 200000 TRANSFORMS-discard_events = setnull_whitespace_indented,setnull_debug_logging SEDCMD-RemoveLogProp = s/("log":)(.*)(?="stream":)// this is log, which is coming into the Splunk by HEC. {"log":"{\"@timestamp\":\"2021-08-03T09:00:57.539+02:00\",\"@version\":\"1\",\"message\": My question is: Do changes like TIME_FIELDS, TIME_PREFIX, TIME_FORMAT in HF have effect on this process when INDEXED_EXCTRACTION is not in use? Thank you very much for your answers.

Jakub · ‎08-04-2021

Actually nothing works. What else I can try?

Jakub · ‎08-03-2021

Also SEDCMD should not affect that as that is done after timestamp parsing. Thank you!

Jakub · ‎08-03-2021

Hi, hello, Splunk is not showing up miliseconds for JSON logs. I have find some Questions and Answers here in splunk community, but without success. Description: I have HFs, indexer cluster and search head cluster. HF props.conf [k8s:dev] #temporary removed to fix 123123 #INDEXED_EXTRACTIONS = JSON TIME_PREFIX = {\\"@timestamp\\":\\" TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N TRUNCATE = 200000 TRANSFORMS-discard_events = setnull_whitespace_indented,setnull_debug_logging SEDCMD-RemoveLogProp = s/("log":)(.*)(?="stream":)// HF transforms.conf [setnull_java_stacktrace_starttab] SOURCE_KEY = field:log REGEX = ^\tat\s.* DEST_KEY = queue FORMAT = nullQueue [setnull_whitespace_indented] SOURCE_KEY = field:log REGEX = ^\s+.* DEST_KEY = queue FORMAT = nullQueue [setnull_debug_logging] SOURCE_KEY = field:log REGEX = .*?\sDEBUG\s DEST_KEY = queue FORMAT = nullQueue Search props.conf #workaround, see 123123 [k8s:dev] KV_MODE = json Everything looks fine in web ADD DATA in HF and SEARCH too. But not when I search it. I can insert only part of the JSON. {"log":"{\"@timestamp\":\"2021-08-03T09:00:57.539+02:00\",\"@version\":\"1\",\"message\": Also when I am in HF ADD DATA and I remove TIME_PREFIX and TIME_FORMAT the miliseconds still appear, but when I a little bit "destroy" TIME_PREFIX there is error and file timestamp is used(I think its file timestamp). Question is: 1.what am I doing wrong? 2. Is it possible to configure TIME_PREFIX and TIME_FORMAT for KV_MODE on search? Because as I know they are used in HF during parsing. 3. Is it possible to configure KV_MODE? Thank you very much for your suggestions.

Jakub · ‎04-12-2021

Hi All, we are going to upgrade from splunk 7.3.3 to 8, but first I need to check compatibility of all apps with Splunk Platform Upgrade Readiness App. We have indexer cluste, search head cluster, master node, deployer, deployment server, splunk monitoring web etc. , there are plenty of apps. Question is very general: Where needs to be Splunk Platform Upgrade Readiness App installed to cover all apps and makes it sufficient? Where it is not needed? Can you mention some approach how to decide? Are there some tips from experience? I would like to create some list like: I will install it here here and here and now I know I checked all apps. Thank you very much

Posts	7
Solutions	0
Karma Given	2
Karma Received	0
Member Since	‎04-12-2021

Online Status	Offline
Date Last Visited	‎05-17-2022 11:32 AM

How apply SEDCMD after TRANSFORMS?

Splunk JSON timestamp in KV_MODE

Json timestamp miliseconds

Apps - Upgrade to 8 - Splunk Platform Upgrade Read...

How apply SEDCMD after TRANSFORMS?

Re: Splunk JSON timestamp in KV_MODE

Splunk JSON timestamp in KV_MODE

Re: Json timestamp miliseconds

Re: Json timestamp miliseconds

Json timestamp miliseconds

Apps - Upgrade to 8 - Splunk Platform Upgrade Read...

Join the Conversation