All Topics

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Topics

Machine agent log shows error as below. We have restarted the machine agent and the issue still persists. Can someone help in fixing this? [system-thread-0] 07 Mar 2022 00:20:17,926 INFO SimAgentRep... See more...
Machine agent log shows error as below. We have restarted the machine agent and the issue still persists. Can someone help in fixing this? [system-thread-0] 07 Mar 2022 00:20:17,926 INFO SimAgentRepetitiveLoggingModule - The turnover time for the SIM agent repetitive logger is 5 minutes [system-thread-0] 07 Mar 2022 00:20:17,926 INFO SimAgentRepetitiveLoggingModule - The cache size for the SIM agent repetitive logger is 1000 [system-thread-0] 07 Mar 2022 00:20:17,926 INFO SystemAgent - #################################################################################### [system-thread-0] 07 Mar 2022 00:20:17,926 INFO SystemAgent - Agent Install Directory [C:\Program Files\AppDynamics\AppD_Infra_Monitoring] [system-thread-0] 07 Mar 2022 00:20:17,926 INFO SystemAgent - Using Agent Version [Machine Agent v4.4.3.1214 GA Build Date 2018-04-28 05:12:10] [system-thread-0] 07 Mar 2022 00:20:17,926 INFO SystemAgent - JVM Runtime: java.home=c:\program files\appdynamics\appd_infra_monitoring\jre java.vm.vendor=Oracle Corporation java.vm.name=Java HotSpot(TM) 64-Bit Server VM java.runtime.version=1.8.0_111-b14 java.io.tmpdir=C:\Windows\TEMP\ user.language=en user.country=US user.variant= Default locale=en_US [system-thread-0] 07 Mar 2022 00:20:17,926 INFO SystemAgent - OS Runtime: os.name=Windows Server 2012 os.arch=amd64 os.version=6.2 user.name=<servername is removed>$ user.home=C:\Windows\system32\config\systemprofile user.dir=C:\Program Files\AppDynamics\AppD_Infra_Monitoring\bin [system-thread-0] 07 Mar 2022 00:20:17,926 INFO SystemAgent - JVM Args : -Dexe4j.semaphoreName=Local\c:_program_files_appdynamics_appd_infra_monitoring_bin_machineagentservice.exe0 | -Dexe4j.isService=true | -Dexe4j.moduleName=C:\Program Files\AppDynamics\AppD_Infra_Monitoring\bin\MachineAgentService.exe | -Dexe4j.tempDir=C:\Windows\TEMP\e4j9F57.tmp_dir1646630412 | -Dexe4j.unextractedPosition=316127 | -Djava.library.path=D:\apps_srv\webagent\win32\bin;D:\apps_srv\webagent\win64\bin;D:\apps_srv\webagent\win64\install_config_info\lib;D:\apps_srv\webagent\win64\bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\IBM\ITM\InstallITM;;C:\Program Files\BMCperform\Patrol3\BEST1\11.5.00\bgs\bin;C:\Windows;C:\Windows\system32;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\IBM\ITM\bin;C:\IBM\ITM\TMAITM6;C:\Program Files\PKWARE\pkzipc;C:\Program Files\NCache\bin\tools;;c:\program files\appdynamics\appd_infra_monitoring\jre\bin | -Dexe4j.consoleCodepage=cp0 | -Dlog4j.configuration=file:..\conf\logging\log4j.xml | -XX:-CreateMinidumpOnCrash | -Xrs | [system-thread-0] 07 Mar 2022 00:20:17,926 INFO SystemAgent - Machine Agent is resolving bootstrap info.... [system-thread-0] 07 Mar 2022 00:20:17,942 INFO SystemAgent - Orchestration is disabled - disabling virtualization resolvers by default. [system-thread-0] 07 Mar 2022 00:20:17,942 INFO SystemAgent - Full Agent Registration Info Resolver found system property [appdynamics.agent.create.agent.info.if.missing] for appdynamics.agent.create.agent.info.if.missing [false] [system-thread-0] 07 Mar 2022 00:20:17,942 INFO SystemAgent - Default Host Identifier Resolver using host name for unique host identifier [<servername is removed>] [system-thread-0] 07 Mar 2022 00:20:18,445 INFO SystemAgent - Default IP Address Resolver found IP addresses [[169.171.32.81, 169.254.128.132]] [system-thread-0] 07 Mar 2022 00:20:18,445 INFO SystemAgent - Full Agent Registration Info Resolver using selfService [false] [system-thread-0] 07 Mar 2022 00:20:18,445 INFO SystemAgent - Full Agent Registration Info Resolver using application name [null] [system-thread-0] 07 Mar 2022 00:20:18,445 INFO SystemAgent - Full Agent Registration Info Resolver using tier name [null] [system-thread-0] 07 Mar 2022 00:20:18,445 INFO SystemAgent - Full Agent Registration Info Resolver using node name [null] [system-thread-0] 07 Mar 2022 00:20:18,445 INFO SystemAgent - XML Controller Info Resolver found controller host [appdync-nam-icg-p1.wlb2.nam.nsroot.net] [system-thread-0] 07 Mar 2022 00:20:18,445 INFO SystemAgent - XML Controller Info Resolver found controller port [8181] [system-thread-0] 07 Mar 2022 00:20:18,445 INFO SystemAgent - XML Agent Account Info Resolver using account name [customer1] [system-thread-0] 07 Mar 2022 00:20:18,445 INFO SystemAgent - XML Agent Account Info Resolver using account access key [****] [system-thread-0] 07 Mar 2022 00:20:18,523 WARN FrameworkBootstrap - Failed to start framework com.google.inject.CreationException: Unable to create injector, see the following errors: 1) Error injecting constructor, java.lang.NoClassDefFoundError: org/apache/commons/fileupload/FileUploadException at com.singularity.ee.agent.systemagent.SystemAgent.<init>(SystemAgent.java:50) at com.appdynamics.agent.sim.legacy.LegacyAgentModule.configure(LegacyAgentModule.java:59) while locating com.singularity.ee.agent.systemagent.SystemAgent Caused by: java.lang.NoClassDefFoundError: org/apache/commons/fileupload/FileUploadException at com.singularity.ee.agent.systemagent.Agent.setupHttpClientWrapper(Agent.java:265) at com.singularity.ee.agent.systemagent.Agent.<init>(Agent.java:248) at com.singularity.ee.agent.systemagent.SystemAgent.<init>(SystemAgent.java:42) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.google.inject.internal.DefaultConstructionProxyFactory$2.newInstance(DefaultConstructionProxyFactory.java:86) at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:105) at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85) at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:267) at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) at com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:205) at com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:199) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:199) at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:180) at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110) at com.google.inject.internal.InjectorImpl.createChildInjector(InjectorImpl.java:226) at com.appdynamics.voltron.FrameworkBootstrap.createInjector(FrameworkBootstrap.java:105) at com.appdynamics.voltron.FrameworkBootstrap.start(FrameworkBootstrap.java:162) at com.appdynamics.agent.sim.main.SecondStageSystem.start(SecondStageSystem.java:175) at com.appdynamics.agent.sim.main.SecondStageSystem.loadFeatures(SecondStageSystem.java:128) at com.appdynamics.agent.sim.main.SecondStageSystem.access$000(SecondStageSystem.java:48) at com.appdynamics.agent.sim.main.SecondStageSystem$1.run(SecondStageSystem.java:115) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: org.apache.commons.fileupload.FileUploadException at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 36 more
  I have following splunk fields Date,Group,State State can have following values InProgress|Declined|Submitted I like to get following result Date. Group. TotalInProg... See more...
  I have following splunk fields Date,Group,State State can have following values InProgress|Declined|Submitted I like to get following result Date. Group. TotalInProgress. TotalDeclined TotalSubmitted. Total ----------------------------------------------------------------------------- 12-12-2021 A. 13. 10 15 38 I couldn't figured it out. Any help would be appreciated
I have a log like below:     index=login sourcetype=login new_user=1     I also have logs without new_user label     index=login sourcetype=aa     What's the difference between I  search ... See more...
I have a log like below:     index=login sourcetype=login new_user=1     I also have logs without new_user label     index=login sourcetype=aa     What's the difference between I  search which specify the sourcetype     index=login sourcetype=login new_user=1     and do not specify the sourcetype just using new_user like     index=login new_user=1     I wonder which one is faster or perform better and why? And if I make new_user=1 as a sourcetype, does      index=login sourcetype=new_user     better than     index=login sourcetype=login new_user=1     Thank you in advance
Hi have a results from my mail index say log look like below sender=abc recipient=xyz@sample.com,ghi@nonsample.com country=abc sender=def recipient=team@nonsample.com country=xyz sender=gfh r... See more...
Hi have a results from my mail index say log look like below sender=abc recipient=xyz@sample.com,ghi@nonsample.com country=abc sender=def recipient=team@nonsample.com country=xyz sender=gfh recipient=tip@nonsample.com country=efg sender=abc recipient=none@sample.com,sample@nonsample.com country=pqr I want to shows in a table only the non comma separated recipients only (as highlighted in bold where there are no multiple recipients)   can some one help me on this  
Can we suppress the trigger actions of the alert at its first trigger and then on the next time when the alert triggers it should do the trigger actions.... Please assist if this can be doable and ho... See more...
Can we suppress the trigger actions of the alert at its first trigger and then on the next time when the alert triggers it should do the trigger actions.... Please assist if this can be doable and how to configure it to achieve this.   Thanks a ton !
As the title suggests, I want to index data from Splunk user email account's inbox folder. Splunk version - 8.2.4 Have already checked out TA-mailclient and IMAP Mailbox addons but none of them w... See more...
As the title suggests, I want to index data from Splunk user email account's inbox folder. Splunk version - 8.2.4 Have already checked out TA-mailclient and IMAP Mailbox addons but none of them work and are unsupported In the first add-on, no matter how many times I change the attribute disabled to 0 in inputs.conf, it goes back to 1 after a restart. In the second addon, after using the troubleshooting command, I get the following error   File "/opt/splunk/etc/apps/IMAPmailbox/bin/get_imap_email.py", line 104 self.port = 993 ^ TabError: inconsistent use of tabs and spaces in indentation     Hence, can someone please advise how best to achieve this ?
I am using 2 lookup tables to correlate and combine data to create a new .csv. In this process, I have a field that has numerical values in it that i want to sum based on the values of another field.... See more...
I am using 2 lookup tables to correlate and combine data to create a new .csv. In this process, I have a field that has numerical values in it that i want to sum based on the values of another field.  The raw data looks something like this: stage,resource,hours x,rick,1 x,rick,10 x,dave,1 y,rick,5 y,dave,3 y,dave,8   I want the output to look like: x,rick,11 x,dave,1 y,rick,5 y,dave,11   Below is the search I have, it almost works but it does not sum the totals for rick and dave individually, it sums them all, so the output looks like: x,rick,12 x,dave,12 y,rick,16 y,dave,16   |inputlookup mod_master |lookup lookuptable1 Engagement OUTPUTNEW ResourceLastName,RegularHours | eval Resource=mvdedup(ResourceLastName) | mvexpand Resource | eval Hours=sum(RegularHours) | fillnull value=0 Hours | table Stage,Resource,Hours     How can i change the logic in this search so I get the correct individual sums for Rick and Dave and not the combined total for each? Thank you in advance!  
Hello everyone, when I imported the entities, the operating system family value was mapped to 'Entity Type'. I'm currently setting up a couple services, one for Linux and another for Windows. In ... See more...
Hello everyone, when I imported the entities, the operating system family value was mapped to 'Entity Type'. I'm currently setting up a couple services, one for Linux and another for Windows. In the Entity Rules builder, I can't find a way to include a condition 'Entity Type matches Windows Server'. Is it really so? (bit suprised). It seems to me I'll have to re-import the entities (using the update option) with an extra field for OS Family and set it as informational field. Any hints or tips are welcome! Thanks!
I'm trying to see if there is a report or a query I can run to sum up all the events in all the indexers with a month total? I tried this but i'm not sure if this is the right search: index=* | sta... See more...
I'm trying to see if there is a report or a query I can run to sum up all the events in all the indexers with a month total? I tried this but i'm not sure if this is the right search: index=* | stats count by Period
I am using timewrap to get the info for last 3 weeks and show data correspondly for 3 week with 3 different pie chart(when i activate the trellis layout). Although the data are correct it shows the c... See more...
I am using timewrap to get the info for last 3 weeks and show data correspondly for 3 week with 3 different pie chart(when i activate the trellis layout). Although the data are correct it shows the chart where the dates are  incorrect like they are showing the dates for current week for others previous week and last 2 weeks. How would i make it show each date data in 3 pie chart with 7 days data information.   index=mysearch earliest=-21d@d latest=@d | timechart count span=1d | timewrap w 
hi, i am a bit lost, i am trying to extract some % values of specific parameters. but with no luck example i want to extract the anti-virus value of 12%   this is my command | rex field=_raw ... See more...
hi, i am a bit lost, i am trying to extract some % values of specific parameters. but with no luck example i want to extract the anti-virus value of 12%   this is my command | rex field=_raw "Anti-Spam\s*<(?<cpu>.*)>" Gauges: Current System RAM Utilization 65% Overall CPU load average 43% CPU Utilization MGA 20% Anti-Virus 12% Reporting 0% Quarantine 0%
Hi, The start-up script for the spring boot application is like this java -javaagent:/opt/jeppesen/crew-leave/Appagent/AppServerAgent-ibm-22.2.0.33545/javaagent.jar -jar /opt/app/app.jar. The A... See more...
Hi, The start-up script for the spring boot application is like this java -javaagent:/opt/jeppesen/crew-leave/Appagent/AppServerAgent-ibm-22.2.0.33545/javaagent.jar -jar /opt/app/app.jar. The AppDynamics log shows that the app agent is created successfully. But nothing is getting captured.  Also, under agents, I see that the uptime is 0%. This is for a Linux server. I get a few warnings at the end [AD Agent init] 07 Mar 2022 19:56:16,708 WARN InstrumentationHandler - Retransformation not enabled. [AD Agent init] 07 Mar 2022 19:56:16,708 INFO AnnotationPropertyListenerManager - Class.privateGetPublicMethods() is not available, will try the public methods [AD Agent init] 07 Mar 2022 19:56:16,708 INFO AnnotationPropertyListenerManager - Registered NodeProperty [enable-async-correlation-for] to method [public void com.singularity.ee.agent.appagent.services.transactionmonitor.common.activity.CustomActivityRuleApplier.setEnableAsync(java.lang.String)] in class class com.singularity.ee.agent.appagent.services.transactionmonitor.common.activity.CustomActivityRuleApplier 1 [AD Agent init] 07 Mar 2022 19:56:16,715 INFO BCIFixer - Scheduling BCIFixer at [300000] ms intervals. [AD Agent init] 07 Mar 2022 19:56:16,715 INFO BCIFixer - Retransformation is NOT supported on this JVM. BCIFixer is active - but will NOT retransform classes [AD Agent init] 07 Mar 2022 19:56:16,715 INFO BCIEngineService - Pinging to retransform classes by worker [AD Agent init] 07 Mar 2022 19:56:16,715 WARN InstrumentationHandler - Retransformation not enabled. [AD Agent init] 07 Mar 2022 19:56:16,719 INFO JavaAgent - JVM Process Persistence File <pid = 41888 nodeDirectory = /opt/jeppesen/crew-leave/Appagent/AppServerAgent-ibm-22.2.0.33545 appName = CrewRequest tierName = WebNode nodeName = 103 homeDirectory = / machineId = 24821 uniqueHostId = mrnf-cl103 controllerHost = cubits202202230044249.saas.appdynamics.com controllerPort = 443 accountKey = c754eab83f6444f17a46da72929b318b82407869 versionDirectory = /opt/jeppesen/crew-leave/Appagent/AppServerAgent-ibm-22.2.0.33545/ver22.2.0.33545> was written [AD Agent init] 07 Mar 2022 19:56:16,719 INFO JavaAgent - Started AppDynamics Java Agent Successfully. [AD Thread Pool-Global0] 07 Mar 2022 19:56:16,746 INFO TransformationManager - Class transformations will take place in a background thread [main] 07 Mar 2022 19:56:17,601 INFO ClassMetaDataManager - Unable to locate class meta data for org.springframework.boot.loader.archive.Archive$EntryFilter [main] 07 Mar 2022 19:56:17,688 INFO ClassMetaDataManager - Unable to locate class meta data for org.springframework.boot.loader.jar.JarEntryFilter
Hi We are sending in Opentelemtory metrics into Splunk via HTTP Event Collector. However, we got the following errors the other days "server is busy" . I can see the data did come in at that time... See more...
Hi We are sending in Opentelemtory metrics into Splunk via HTTP Event Collector. However, we got the following errors the other days "server is busy" . I can see the data did come in at that time, but it gets retried so that explains that. How do I stop this from happening in the future? Another question is what is the max throughput Splunk can take in via HTTP? The below code came from the OP - Python scripts      2022-03-04T19:41:36.125+0100 info exporterhelper/queued_retry.go:215 Exporting failed. Will retry the request after interval. {"kind": "exporter", "name": "splunk_hec/logs", "error": "Post \https://dell425srv:9088/services/collector\: context deadline exceeded (Client.Timeout exceeded while awaiting headers)", "interval": "5.6081835s"}       Thanks in advance Rob
Hi, after a Windows system crash of the raid controller, I only get empty reports.  I moved the installation to a VM and everything looked good... But: empty dashboard reports! Getting data from ... See more...
Hi, after a Windows system crash of the raid controller, I only get empty reports.  I moved the installation to a VM and everything looked good... But: empty dashboard reports! Getting data from the windows eventlogs into splunk still works, but most entries have this problem: Message=Splunk could not get the description for this event. Either the component that raises this event is not installed on your local computer or the installation is corrupt. FormatMessage error... I checked wecutil gs and I found, it was set to "rendered text" for both of my subscriptions. I've set it back to "events" but still no luck. Restart of the service: not luck. I'm running SPLUNK Enterprise 8.2.5 on Windows Server 2016. Any hints are highly appreciated! Best, EL
I wanted to ask if it was easy or possible to forward logs if some may be in text format from a HF to another device and send in JSON format?
We have an instance where KV store is not running and we're looking to clean the whole thing out. However, we would like to see if we're able to keep the data. So, my question is; is there any othe... See more...
We have an instance where KV store is not running and we're looking to clean the whole thing out. However, we would like to see if we're able to keep the data. So, my question is; is there any other way to do a backup of KV store data than using the "splunk backup kvstore" command?
Hello community, I have a problem with my research. My searches are then sent to Splunk OnCall to manage alerts. However, I noticed that some fields in my request can sometimes have a lot of chara... See more...
Hello community, I have a problem with my research. My searches are then sent to Splunk OnCall to manage alerts. However, I noticed that some fields in my request can sometimes have a lot of characters, and in this case, the information does not go to Splunk OnCall. The large number of characters is not useful and I would like to limit the number so that it does not block towards Splunk OnCall. I therefore seek to limit the number of characters in a field of my request, whatever the result. I came across this post (https://community.splunk.com/t5/Splunk-Search/Limit-length-of-field-of-characters-in-search-results/mp/134908#M36872) but the command does not work. Here is an example request:     index=my_index state=* | fields hostname service_description output state     It is the "output" field that I would like to limit. Based on the link above, I wrote:     index=my_index state=* | fields hostname service_description output state | eval output=substr(output,1,15)     But that doesn't fix it. Do you have any idea where I should dig for this? FYI, I'm on Enterprise 8.2.3 Thank you in advance Rajaion
Hi all, I have a stream of events which come in to SOAR. When the event is loaded in SOAR, a playbook runs against it automatically - and this calls an App action, which completes successfully ... See more...
Hi all, I have a stream of events which come in to SOAR. When the event is loaded in SOAR, a playbook runs against it automatically - and this calls an App action, which completes successfully and returns data. In another playbook, which will be manually run, how do I access the data returned by the App action? Do I need to save the app action data as a new artifact, and call it that way, or is there a way to list all app actions in a container, and get the app action ID, and retrieve the data? Thanks!
Hi I have a query based on response times from a  service. index=homebanking "/soa/mcoi-rc-services/ContractService" Time="*" | rex field=_raw "\/(?<time>[^_\/]+)[\w\.]+($|\s)" | stats count by T... See more...
Hi I have a query based on response times from a  service. index=homebanking "/soa/mcoi-rc-services/ContractService" Time="*" | rex field=_raw "\/(?<time>[^_\/]+)[\w\.]+($|\s)" | stats count by Time | fields - count I get these results......   I'd like to now create a graph of these results showing a graduation or reduction of the response times. Can someone help? Best, Sheldon.        
Hi  All, I am going to work on a multisite architecture (3-sites) where we are keeping 6 indexers in a cluster (2 in each sites) ,and 6 SHs in a cluster (2 in each sites) , 3 SHs in another cluste... See more...
Hi  All, I am going to work on a multisite architecture (3-sites) where we are keeping 6 indexers in a cluster (2 in each sites) ,and 6 SHs in a cluster (2 in each sites) , 3 SHs in another cluster (1 in each site) for reporting .  1 Indexer cluster 2 SHs clusters i wanted to understand what will be the best site search factor & site replication factor for a 3-sites multisite clustering . Any suggestion will be appreciated .  Thanks , Sushree