All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Hi Splunkers, a colleague team si facing some issues related to .csv file collection. Let me share  the required context. We have a .csv file that is sent to a sftp server. The sending is 1 per day:... See more...
Hi Splunkers, a colleague team si facing some issues related to .csv file collection. Let me share  the required context. We have a .csv file that is sent to a sftp server. The sending is 1 per day: this means that every day, the file is write once and never modified. In addiction to this, even if the file is a csv one, it has a .log extension. On this server, the Splunk UF is installed and configured to read this daily file. What currently happen is the following: The file is read many time: multiple occurrence of error message like:  INFO  WatchedFile [23227 tailreader0] - File too small to check seekcrc, probably truncated.  Will re-read entire file=<file name here> can be got from internal logs   The csv header is viewed like an event. This means that, for example, the file contains 1000 events, performing a search in assigned index we have 1000 + x  events; each of this x events does not contains real events, but the csv header file. So, we see the header as an event/logs. For the first problem, I suggested to my team to use the initCrcLength parameter, properly set. For the second one, I shared them to ensure that following parameter are set: INDEXED_EXTRACTIONS = csv HEADER_FIELD_LINE_NUMBER = 1 CHECK_FOR_HEADER = true In addition to this, I suggested them to avoid the default line breaker; in the inputs.conf file is set the following one: LINE_BREAKER = ([\r\n]+) That could be the root cause/one of the cause of header extraction as events. I don't know if those changes has fixed the events (they are still performing required restarts), but I would ask you if any other possible fix should be applied. Thanks!
| rest splunk_server_group=dmc_group_license_master /services/licenser/groups | search is_active=1 | eval stack_id=stack_ids | mvexpand stack_id | join type=outer stack_id splunk_server [ rest s... See more...
| rest splunk_server_group=dmc_group_license_master /services/licenser/groups | search is_active=1 | eval stack_id=stack_ids | mvexpand stack_id | join type=outer stack_id splunk_server [ rest splunk_server_group=dmc_group_license_master /services/licenser/pools] | fields splunk_server, stack_id, effective_quota, used_bytes | stats sum(used_bytes) as used_bytes max(effective_quota) as stack_quota by stack_id | eval usedGB=round(used_bytes/1024/1024/1024, 3),totalGB=round(stack_quota/1024/1024/1024, 3), percentage=round((used_bytes / stack_quota) * 100, 2) | eval alert_level=case(percentage >= 90, "Critical: >90%",percentage >= 80, "Warning: >80%",percentage >=70, "Info: >70%",true(), null()) | where isnotnull(alert_level) | rename stack_id AS Instance, percentage AS "License quota used (%)",usedGB AS "License quota used (GB)",totalGB AS "Total license quota (GB)",alert_level AS "Alert Level"
Hi, @gcusello  Thanks for your reply! The configurations were applied on the main Splunk instance, not on the Heavy Forwarder. However, within the same configuration files on the main instance (pro... See more...
Hi, @gcusello  Thanks for your reply! The configurations were applied on the main Splunk instance, not on the Heavy Forwarder. However, within the same configuration files on the main instance (props.conf and transforms.conf), there are already similar settings that are currently working as expected. Also, we have tested the regex patterns used in the transforms.conf, and they are working correctly — they match the intended log lines when tested separately. That’s why we’re a bit puzzled — some TRANSFORMS stanzas are effective, while the ones mentioned in the original post aren’t having any impact. Any further insights would be greatly appreciated! Best regards,
Hi @Cybers1 , I suppose that you already checked the regexes that you're using and they are correct to filter logs. The question is: where do you located these conf files? They must be located on ... See more...
Hi @Cybers1 , I suppose that you already checked the regexes that you're using and they are correct to filter logs. The question is: where do you located these conf files? They must be located on the first full Splunk instance that dara pass trhough, in other words on the first Heavy Forwarder (if present) or in the Indexers. Ciao. Giuseppe
Hi Splunk Community, We’re currently trying to drop specific logs using props.conf and transforms.conf, but our configuration doesn’t seem to be working as expected. Below is a summary of what we’ve... See more...
Hi Splunk Community, We’re currently trying to drop specific logs using props.conf and transforms.conf, but our configuration doesn’t seem to be working as expected. Below is a summary of what we’ve done: transforms.conf [eliminate-accesslog_coll_health] REGEX = ^.*(?:H|h)ealth.* DEST_KEY = queue FORMAT = nullQueue [eliminate-accesslog_coll_actuator] REGEX = ^.*actuator.* DEST_KEY = queue FORMAT = nullQueue props.conf [access_combined] TRANSFORMS-set = eliminate-accesslog_coll_actuator, eliminate-accesslog_coll_health [iis] TRANSFORMS-set = eliminate-accesslog_coll_health [(?::){0}kube:*] TRANSFORMS-set = eliminate-accesslog_coll_actuator The main issue is that events are not being dropped, even when a specific sourcetype is defined (like access_combined or iis). Additionally, for logs coming from Kubernetes, there is no single consistent sourcetype, so we attempted to match using [source::] logic via a regex ([(?::){0}kube:*]), but this doesn’t seem to be supported in this context. From what we've read in the documentation, it looks like regex patterns for [source::] are not allowed in props.conf, and must instead be written explicitly. Is that correct? And if so, what’s the best way to drop events from dynamic sources or where the sourcetype is inconsistent? Any help or suggestions would be greatly appreciated. Thanks in advance!  
Hello Guys,   We have SCOM on physical box & want to onboard in AppDynamics for monitoring. customer wants to onboard without agent installation on SCOM. Could you please let me know what is best a... See more...
Hello Guys,   We have SCOM on physical box & want to onboard in AppDynamics for monitoring. customer wants to onboard without agent installation on SCOM. Could you please let me know what is best approach to SCOM monitoring in APpDynamics.   Thanks  
Hi @Naoki  If you are using a standalone SearchHead then yet you can upload via the UI, ticking the upgrade button - however you might hit issues due to the size of the tarball file being higher tha... See more...
Hi @Naoki  If you are using a standalone SearchHead then yet you can upload via the UI, ticking the upgrade button - however you might hit issues due to the size of the tarball file being higher than the limits.conf set for file uploads. To overcome this see the following snippet from https://splunk.my.site.com/customer/s/article/Python-for-Scientific-Computing 1) To upgrade via UI needs to change max_upload_size at path $SPLUNK_HOME/etc/system/local/web.conf: [settings] max_upload_size = 2048 2) For clustered environment, increase the bundle size to the same setting as max_content_length on the SHs as follow: 1. Navigated to /opt/splunk/etc/system/local/distsearch.conf on the server. 2. Append/Update the following parameters:- [replicationSettings] maxBundleSize = 2048 or 3072 3. Save the file and restart all SHs.  Alternatively you can copy it to the Splunk instance's app folder ($SPLUNK_HOME/etc/apps) overwritting the previous content.  If in any doubt please make a backup copy of the PSC app on your Splunk instance first.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
Hi @RAVISHANKAR  Whilst you are right in that the 8.0.x UF forward can send events/metrics to 9.4.x it is important to note that 8.0.x UFs are no longer supported by Splunk. So technically, yes, it ... See more...
Hi @RAVISHANKAR  Whilst you are right in that the 8.0.x UF forward can send events/metrics to 9.4.x it is important to note that 8.0.x UFs are no longer supported by Splunk. So technically, yes, it will work - but from a support standpoint you need to upgrade UFs to 9.1.x to still be supported by Splunk, although that is only until 28th June (17 days!) so I would recommend a minimum of 9.2.x For more info on supported Splunk versions check out https://www.splunk.com/en_us/legal/splunk-software-support-policy.html?locale=en_us  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing  
Hi @cdevoe57  If you want to use the lookup as a source of truth for the list of hosts I would use the following, also just a note that I'm suggesting tstats here which is *much* more performant tha... See more...
Hi @cdevoe57  If you want to use the lookup as a source of truth for the list of hosts I would use the following, also just a note that I'm suggesting tstats here which is *much* more performant than a regular index= search. | tstats latest(_time) as _time WHERE index=servers sourcetype=logs by host | eval last_seen_ago_in_seconds = now() - _time | eval System_Name = host | append [|inputlookup system_info.csv | eval last_seen_ago_in_seconds=9999] | stats min(last_seen_ago_in_seconds) as last_seen_ago_in_seconds, values(Location) AS Location, values(Responsible) AS Responsible by System_Name | eval MISSING = if(isnull(last_seen_ago_in_seconds) OR last_seen_ago_in_seconds>7200, "MISSING", "GOOD") | where MISSING=="MISSING" | sort -last_seen_ago_in_seconds This works by appending the system_info.csv with a large last_seen_ago_in_seconds which is updated by a lower last_seen_ago_in_seconds value if the host has been found.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
Hi Team, Planned to upgrade Splunk Enterprise from Version 9.2.1 to 9.4.2 Latest - Currently my Splunk UF version is 8.0.5. Will 8.0.5 support or i need to upgrade UF version too? Compatibility ... See more...
Hi Team, Planned to upgrade Splunk Enterprise from Version 9.2.1 to 9.4.2 Latest - Currently my Splunk UF version is 8.0.5. Will 8.0.5 support or i need to upgrade UF version too? Compatibility between forwarders and Splunk Enterprise indexers - Splunk Documentation It says UF 8.0.X will be compatible for 9.4.X (E,M) Events and metrics. Need further clarification on this whether i should upgrade UF or it's ok to be on 8.0.X version. Thanks  
  also in future we would be Decommissioning the indexer after i have send the data to sh and then i will be sending data directly to sh
also in future we would be Decommissioning the indexer after i have send the data to sh and then i will be sending data directly to sh
@SN1  If you must move indexed data from the indexer to the Search Head, you can copy the data files,  Stop Splunk on both the indexer and the Search Head. Copy the index data directories from... See more...
@SN1  If you must move indexed data from the indexer to the Search Head, you can copy the data files,  Stop Splunk on both the indexer and the Search Head. Copy the index data directories from the indexer to the Search Head: Example: Copy $SPLUNK_HOME/var/lib/splunk/<index_name> from the indexer to the same path on the Search Head. Ensure file ownership,permissions,storage size,os and splunk versions are correct on the Search Head. Also make sure you have configuration for the indexes.conf for the indexes you have. Start Splunk on the Search Head. Regards, Prewin Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a kudos/Karma. Thanks!
In the meantime, if you are wanting to move the existing data from your indexer to your SH then stop Splunk on both servers and copy the full directory structure for each index (usually under $SPLUNK... See more...
In the meantime, if you are wanting to move the existing data from your indexer to your SH then stop Splunk on both servers and copy the full directory structure for each index (usually under $SPLUNK_DB, by default $SPLUNK_HOME/var/lib/splunk/<indexname>) from the old indexer to the new server. After copying, ensure Splunk points to the correct path for these indexes in indexes.conf on the new instance. Restart Splunk on the new instance for the data to be available. If there are no existing indexes with the same name on the new instance, you can simply copy the directories.  Both source and destination should use the same OS and compatible Splunk versions and don't copy buckets from newer Splunk versions to much older versions. If your SH is still setup to search your IDX then you should probably disconnect it at this point as your may see duplicate data.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
Hi @SN1  I'm not 100% sure if I'm following what your requirements are here, which scenario is this? 1) You want to move existing stored data from your indexer to be stored on your SH to turn it in... See more...
Hi @SN1  I'm not 100% sure if I'm following what your requirements are here, which scenario is this? 1) You want to move existing stored data from your indexer to be stored on your SH to turn it into an All-In-One? 2) Configure the indexer to forward new data as it arrives to the SH? 3) Move existing data *and* configure forwarding of new data to the SH? Please let me know so we can provide a better response.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
ok let me explain briefly ,  we are making our search head a standalone . so now i want to send some important data from indexer which is currently being stored and onboarded from the source to Se... See more...
ok let me explain briefly ,  we are making our search head a standalone . so now i want to send some important data from indexer which is currently being stored and onboarded from the source to Search head . Is this clear.
@SN1  To clarify your scenario, You want Search Head to query the indexer for data as needed or having data on the Search Head for  testing/some reason without using the indexer?
we have a index where the data is currently being stored and indexed on the indexer . Now i am making Search head standalone and i want to send the data from indexer to sh . How to do it.
There could be maximum 3 versions per day.
We are now using the Python for Scientific Computing app (v2.0.2) on a on-premise Linux instance, and planning to upgrade the app to the latest version 4.2.3. When upgrading, should we just upload t... See more...
We are now using the Python for Scientific Computing app (v2.0.2) on a on-premise Linux instance, and planning to upgrade the app to the latest version 4.2.3. When upgrading, should we just upload the app package through the splunk web and check the upgrade checkbox? Python for Scientific Computing (for Linux 64-bit) | Splunkbase