All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

@cherrypick  Splunk cannot natively parse multiple timestamp formats for the same field at index time—it only allows a single TIME_FORMAT per sourcetype. If you can preprocess or route events diffe... See more...
@cherrypick  Splunk cannot natively parse multiple timestamp formats for the same field at index time—it only allows a single TIME_FORMAT per sourcetype. If you can preprocess or route events differently, you can assign different sourcetypes based on the date format # props.conf [test_json] TRANSFORMS-set_sourcetype = set_sourcetype_datetime, set_sourcetype_dateonly [test_json_datetime] TIME_PREFIX = "date":\s*" TIME_FORMAT = %Y-%m-%d %I:%M:%S %p [test_json_dateonly] TIME_PREFIX = "date":\s*" TIME_FORMAT = %Y-%m-%d # transforms.conf [set_sourcetype_datetime] REGEX = "date":\s*"\d{4}-\d{2}-\d{2} \d{1,2}:\d{2}:\d{2} [AP]M" DEST_KEY = MetaData:Sourcetype FORMAT = sourcetype::test_json_datetime [set_sourcetype_dateonly] REGEX = "date":\s*"\d{4}-\d{2}-\d{2}" DEST_KEY = MetaData:Sourcetype FORMAT = sourcetype::test_json_dateonly Regards, Prewin  Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a kudos. Thanks!  
Again, this does not work because this filters events during search time and not using shared time picker.  Let's say I have an event that has been indexed with date 2025-04-01 but I ingest it so ... See more...
Again, this does not work because this filters events during search time and not using shared time picker.  Let's say I have an event that has been indexed with date 2025-04-01 but I ingest it so that _time is 2025-04-02 (so date and _time is mismatched), if I use a timechart command to filter alerts over 2025-04-01, this event will not appear on the timechart because it is first filtered on _time. Even if I specify timechart by date, this event will not appear.  My core issue is how to ensure _time and date fields are the same in the index (NOT SEARCH TIME) when ingesting data with mismatched formats.
@cherrypick  Then you can try this  props.conf  [json_splunk] SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+) TIME_PREFIX = "date":\s*" TIME_FORMAT = %Y-%m-%d %I:%M:%S %p  MAX_TIMESTAMP_LOO... See more...
@cherrypick  Then you can try this  props.conf  [json_splunk] SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+) TIME_PREFIX = "date":\s*" TIME_FORMAT = %Y-%m-%d %I:%M:%S %p  MAX_TIMESTAMP_LOOKAHEAD = 60 TRANSFORMS-normalize = fix_date_field, fix_time_hour transforms.conf [fix_date_field] REGEX = ("date":\s*")(\d{4}-\d{2}-\d{2}|\d{2}-\d{2}-\d{2})(") FORMAT = $1$2 12:00:00 AM$3 DEST_KEY = _raw [fix_time_hour] REGEX = ("date":\s*".*?\s)(\d{1})(:\d{2}:\d{2}\s(?:AM|PM)) FORMAT = $10$2$3 DEST_KEY = _raw output: Sample events which i tried: {"date": "2025-05-23 9:35:35 PM", "event": "Login"} {"date": "2025-05-23", "event": "Logout"} {"date": "2025-05-24 10:15:00 AM", "event": "Login"} {"date": "2025-05-24", "event": "Logout"} {"date": "2025-05-25 11:45:00 AM", "event": "Update"} {"date": "2025-05-25", "event": "Login"} {"date": "2025-05-26 12:00:00 PM", "event": "Logout"} {"date": "2025-05-26", "event": "Update"} {"date": "2025-05-27 1:30:00 PM", "event": "Login"} {"date": "2025-05-27", "event": "Logout"}    
I need this to be done at ingest and not during search. The reason is that Splunk first filters on _time so not having the correct _time values will filter out results that shouldn't.  
@cherrypick  SPL with Dummy Data Using makeresults | makeresults count=10 | streamstats count as id | eval raw_json=case( id=1, "{\"date\": \"2025-05-23 9:35:35 PM\", \"event\": \"Login\"}", id... See more...
@cherrypick  SPL with Dummy Data Using makeresults | makeresults count=10 | streamstats count as id | eval raw_json=case( id=1, "{\"date\": \"2025-05-23 9:35:35 PM\", \"event\": \"Login\"}", id=2, "{\"date\": \"2025-05-23\", \"event\": \"Logout\"}", id=3, "{\"date\": \"2025-05-24 10:15:00 AM\", \"event\": \"Login\"}", id=4, "{\"date\": \"2025-05-24\", \"event\": \"Logout\"}", id=5, "{\"date\": \"2025-05-25 11:45:00 AM\", \"event\": \"Update\"}", id=6, "{\"date\": \"2025-05-25\", \"event\": \"Login\"}", id=7, "{\"date\": \"2025-05-26 12:00:00 PM\", \"event\": \"Logout\"}", id=8, "{\"date\": \"2025-05-26\", \"event\": \"Update\"}", id=9, "{\"date\": \"2025-05-27 1:30:00 PM\", \"event\": \"Login\"}", id=10, "{\"date\": \"2025-05-27\", \"event\": \"Logout\"}" ) | spath input=raw_json | eval parsed_time = if(match(date, "\\d{4}-\\d{2}-\\d{2} \\d{1,2}:\\d{2}:\\d{2} [APMapm]{2}"), strptime(date, "%Y-%m-%d %I:%M:%S %p"), strptime(date, "%Y-%m-%d")) | eval _time = parsed_time | table _time, date, event | makeresults count=4 | streamstats count AS row | eval _raw=case( row=1, "{\"date\":\"2025-05-23 21:35:35\"}", row=2, "{\"date\":\"2025-05-22\"}", row=3, "{\"date\":\"2025-05-21 15:20:00\"}", row=4, "{\"date\":\"2025-05-20\"}" ) | spath | eval _time=if(match(date, "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}"), strptime(date, "%Y-%m-%d %H:%M:%S"), strptime(date, "%Y-%m-%d")) | table date _time  
@livehybrid  You're absolutely right that the public documentation (including the Restore indexed data from a self-storage location guide) outlines the DDSS process in detail, and it is technically ... See more...
@livehybrid  You're absolutely right that the public documentation (including the Restore indexed data from a self-storage location guide) outlines the DDSS process in detail, and it is technically possible for customers to manage this independently, especially those with in-house Splunk expertise. 
I have a Json file which contains a "date" field. The date field in my data that can either be of format %Y-%m-%d %H:%M:%S (e.g. 2025-05-23 9:35:35 PM) or %Y-%m-%d (e.g. 2025-05-23). The only way to ... See more...
I have a Json file which contains a "date" field. The date field in my data that can either be of format %Y-%m-%d %H:%M:%S (e.g. 2025-05-23 9:35:35 PM) or %Y-%m-%d (e.g. 2025-05-23). The only way to ingest this Json is via manual ingestion. When trying to set the _time field on ingest, setting the timestamp format to %Y-%m-%d %H:%M:%S will fail and default to the wrong _time value for date fields with format %Y-%m-%d. However, setting timestamp to format %Y-%m-%d won't capture the HMS part. Was there a way to coalesce these so that it will check if HMS is present, and if so, then apply %Y-%m-%d %H:%M:%S format? Or is there a workaround so at least the data ingestion for _time is accurate?
See these events in splunkd: 05-22-2025 21:07:58.608 -0400 ERROR KVStoreAdminHandler [1848035 TcpChannelThread] - An error occurred. 05-22-2025 21:07:36.668 -0400 ERROR KVStoreIntrospection [184803... See more...
See these events in splunkd: 05-22-2025 21:07:58.608 -0400 ERROR KVStoreAdminHandler [1848035 TcpChannelThread] - An error occurred. 05-22-2025 21:07:36.668 -0400 ERROR KVStoreIntrospection [1848033 TcpChannelThread] - failed to get introspection data 05-22-2025 21:07:19.587 -0400 WARN KVStoreConfigurationProvider [1845927 MainThread] - Action scheduled, but event loop is not ready yet Tried cleaning up kvstore by running "splunk clean kvstore --local --answer-yes" but that didn't change anything, status is still stuck in starting.
No it’s not normal. Usually it should be ready quite quickly. Anything in your internal logs?
You definitely should read what Harendra said!
I followed these steps to add new box to existing SHC, everything looks fine on the SHC side but kvstore status has been 'status : starting' since, and it's been over an hour. Is this normal or I mis... See more...
I followed these steps to add new box to existing SHC, everything looks fine on the SHC side but kvstore status has been 'status : starting' since, and it's been over an hour. Is this normal or I missed something?
When you have more than one target where you are sending then if any of those will blocked the traffic then all traffic will be blocked quite soon after that. Basically after blocked targets queue is ... See more...
When you have more than one target where you are sending then if any of those will blocked the traffic then all traffic will be blocked quite soon after that. Basically after blocked targets queue is full then all other targets will be blocked. This is default behavior of splunk. There are two options which you could change to change this behavior but it means that probability of lost some event will increase.
Hi @kiran_panchavat  I challenge that this is either incorrect or missing some context. I appreciate that this is the sort of thing PS get involved with but I know a number of customers who have man... See more...
Hi @kiran_panchavat  I challenge that this is either incorrect or missing some context. I appreciate that this is the sort of thing PS get involved with but I know a number of customers who have managed this themselves, as once it is in DDSS it isnt much different to a standard thaw process. Infact the process is detailed in the public docs (Restore indexed data from a self-storage location) with a step-by-step process which does not reference requirement for PS. I created a script to convert DDSS to SmartStore for a customer who wanted a small on-prem SH to be able to access old data which you might find useful https://github.com/livehybrid/ddss-restore  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
@MuSThat got me part of the way there but I think I may have accidentally oversimplified my question a bit. I'll post another question to get the 2nd half answered. Thanks for the help!
According to the Splunk Cloud Overview Technical Enablement, Splunk recommends engaging Professional Services.. @livehybrid     
Hi @Cheng2Ready  In the SPL you have shared you are appending a makeresult with count=0, then stats sum(count) as total, but then in your WHERE clause you have total>1 OR total=0 If total=0 which I... See more...
Hi @Cheng2Ready  In the SPL you have shared you are appending a makeresult with count=0, then stats sum(count) as total, but then in your WHERE clause you have total>1 OR total=0 If total=0 which I guess it will, then you will still get 1 result returned, even if the returned result says total=0 it will still match the criteria of No. of events >0. I think you meant to close the append after the eval count=0. Can you try the following instead? index=xxxxxx | eval HDate=strftime(_time,"%Y-%m-%d") | search NOT [ | inputlookup Date_Test.csv | fields HDate ] | stats count | append [ | makeresults | eval count=0 | fields count) ] | stats sum(count) AS total | where total>1 Although I'm confused as to why you couldnt do this? index=xxxxxx | eval HDate=strftime(_time,"%Y-%m-%d") | search NOT [ | inputlookup Date_Test.csv | fields HDate ] | stats count | where count>0  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
Hi @kaeleyt  Use the Splunk lookup feature by saving Dataset 2 (ip-to-hostname mapping) as a CSV lookup file and then using the lookup command to enrich Dataset 1. This fully bypasses subsearch, joi... See more...
Hi @kaeleyt  Use the Splunk lookup feature by saving Dataset 2 (ip-to-hostname mapping) as a CSV lookup file and then using the lookup command to enrich Dataset 1. This fully bypasses subsearch, join, and append limits.   Create CSV lookup table from Dataset 2   index=my_hosts | stats values(hostname) as hostname by ip | outputlookup ip_to_hostname.csv   Join Dataset 1 and lookup to enrich logs with hostnames   index=my_logs | stats count by ip | lookup ip_to_hostname.csv ip OUTPUT hostname | table ip, count, hostname   The lookup command does not have the same limiting factors as join, append, or subsearch for reasonable file sizes, you could use either CSV or kvstore lookups.   If Dataset 2 changes regularly you could overwrite the lookup via a scheduled search. For very large lookups, Splunk recommends KV store lookups for scale, but CSV lookups generally perform well up to 1M+ rows. Confirm that the field names (ip, hostname) match exactly between lookup and base data.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
Hi there, Oldy but goldy   https://community.splunk.com/t5/Splunk-Search/How-to-compare-fields-over-multiple-sourcetypes-without-join/m-p/113477#M29849 Hope this helps ... Cheers, MuS
Professional Services is not required to configure or use DDSS. If you are moving away from Splunk Cloud then a customer can setup DDSS using an sc_admin account via the Web UI and/or ACS and then c... See more...
Professional Services is not required to configure or use DDSS. If you are moving away from Splunk Cloud then a customer can setup DDSS using an sc_admin account via the Web UI and/or ACS and then configure their indexes to use the DDSS location.  To migrate away from Splunk Cloud the customer will then need to reduce the retention on these indexes which will trigger existing buckets from DDAS (Active Searchable) to roll to "frozen" (DDSS).  At this point the buckets in S3 are the same as any other frozen bucket from Splunk Enterprise or Splunk Cloud and can be thawed. (see https://docs.splunk.com/Documentation/Splunk/9.4.2/Indexer/Restorearchiveddata) If only the raw data is required then this can be extracted from the journal.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
Hi @_olivier_  I certainly don't doubt my PS colleagues in their recommendations, as they will certainly have more information that I do about this particular set up, but wanted to make sure you kne... See more...
Hi @_olivier_  I certainly don't doubt my PS colleagues in their recommendations, as they will certainly have more information that I do about this particular set up, but wanted to make sure you knew about the queue etc. If its helpful, there is a really good explanation of pipelines at https://community.splunk.com/t5/Getting-Data-In/How-many-pipelines-should-I-use-on-a-forwarder/m-p/410734 which is worth a read.  Interesting that you say about sending to two platforms, as it does sound like congestion outbound from the UF rather than an actual issue with the parsing part of the pipeline. It might be worth (if possible) monitoring the network egress to confirm its not hitting a ceiling, and also check if either of the two outputs are blocking (Check the _internal logs in both platforms for "TcpOutputProc" errors. Another resource worth checking is https://conf.splunk.com/files/2019/slides/FN1570.pdf which might also help.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing