In general, it is better to use  entires  with  xxxxxx.mystack.splunkcloud.com where mystack is your stackid HEC : https://http-inputs.mystack.splunkcloud.com/   ... View more

Hi,   for configuration, state and kvstore backup, you could use splunkconf-backup app (https://splunkbase.splunk.com/app/5600 , https://github.com/splunk/splunkconf-backup/wiki )   for indexed data, in classic architecture, you mainly have to rely on traditional backup solutions (and their challenges) in Smartstore architecture , you could leverage object store feature like versioning, lifecycle policy for delete, compliance option like objectlock and/or ability to sync object to another site in the background, depending on what you want to achieve   ... View more

So it works as expected.   If you configure only outputs, you change what is used to send data but not the CA Splunk software trust internally. For this you need to configure in server.conf I would not put this in a output app but probably create a app for uf specific custom configuration   Once you have server.conf configured, you may simplified outputs configuration when the TLS configuration is the same (as it will use server.conf first) ... View more

Hi, where do you get this error ? indexers ? Universal forwarders ? What about other components like sh, cm,...? On the uf, you should have done configuration in server.conf to change the trusted CA to your and usually provide a throwaway uf certificate On the indexer, you dont say that you configured the server cert in server.conf. Is that done ? (server.conf -> [sslConfig] -> serverCert ) Obviously, you need to do the same on all other components (sh, ...) as that will change the certificate for splunk management communications on port 8089   ... View more

Hi, you should not edit files in system/default as they belong to splunk and will be overwritten when you upgrade So either edit in system/local context or in a application. You just have to place you in the context with a stanza [xxxx] and then add the line you want to change ... View more

btw reports is a search time extraction ... View more

Hi, I think this should work with the transform that change sourcetype uncommented. Then move the REPORTS stanza is in a sourcetype scope not a source so [pan:trafic] REPORT-trafic_fields = pan_trafic_fields (make sure this config is also present on sh so deploy the whole sh + idx)   that makes at least things much easier to debug with such things scoped at sourcetype level     ... View more

web port is 8000 (dont leave that open to Internet, you have no auth with the free license !) hec port is 8088 use this to test hec launch this from browser https://hec-splunk.mydomain.com:8088/services/collector/health/1.0 by replacing hec-splunk.mydomain.com with the public dns name you use from the ELB that does HEC you should have a valid certificate on the elb or Kinesis Firehose will refuse to connect (generating one is free via ACM for ELB) ... View more

Hi, I find a bit unusual that your partition doesnt get mounted before splunk start but you can add a dependency on a mounted partition following something like https://unix.stackexchange.com/questions/246935/set-systemd-service-to-execute-after-fstab-mount   ... View more

Hi, it is a known issue that affect some versions and should be fixed as of now see https://docs.splunk.com/Documentation/Splunk/7.3.7/ReleaseNotes/Fixedissues#Splunk_Enterprise_7.3.7.1 8.1.2 should have the fix I think. If not the case, please open a support case and have the support investigate with you.   ... View more

Hi, do you have something occuring at start that could not have completed when splunk service is started nut still be needed by splunk ? like a partition mount ? or a security feature ? you could use systemd-analyze to understand your boot order ... View more

humm, your props doesnt really tell how to merge. One possible solution would be to add date form at the end of your LINE_BREAKER ... View more

the path dont matter (only the index name is sent over the wire) but it need to be defined and the dir writable by splunk If you are using a common app with volume primary, then one solution is to : - have a indexes app that ref the volume and is used everywhere (indexer, sh, hf) - have a indexer volume app that have the path on indexer and a volume limit if applicable (so you deploy it only on indexer) - have a search volume app (deployed everywhere except on idx) , that define the same volume put point to $SPLUNK_DB     ... View more

Go to settings->Advanced search->Search macros You will have a delete button on the right for the macros you have the right to Create a test one to verify ... View more

Humm, not sure what is your systemd file at the moment.... One fix to fix this may be to: as root :   /opt/splunk/bin/splunk disable boot-start   Note : you obviously need to adapt the home path to your splunk installation dir      (this should remove the system file) then recreate it from v8   /opt/splunk/bin/splunk enable boot-start -systemd-managed 1 -user splunk which should recreate the systemd file with the one from v8   Alternatively you could also just add to the systemd unit file   # change needed for 8.0+ ExecStartPost to ExecStartPre to change the permissions before Splunk is started (replace id with the ones from splunk user and group) ExecStartPre=/bin/bash -c "chown -R 1001:1001 /sys/fs/cgroup/cpu/system.slice/%n" ExecStartPre=/bin/bash -c "chown -R 1001:1001 /sys/fs/cgroup/memory/system.slice/%n"     followed by   systemctl daemon-reload systemctl restart Splunkd.service     + if ever Splunk wrote files as root then stop splunk and give back files to splunk before starting again the service as splunk   ... View more

You will have at least to : - make sure your lambda function run in your vpc (so the network connection start in your vpc and can reach your hec port) (see https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html ) (unless your hec can be accessed from outside your vpc) - allow port TCP 8088 inbound in the security group (where you receive HEC) (connection refused -> probably your main issue as already pointed above) - you may have also to play with certificate checking options in your lambda function (especially if you dont use a valid certificate for hec) ... View more

Hi   You could look/search in web_service.log (from splunk or directly) for the time where the error occured. You should see there the complete error message.   ... View more

You can use useack functionality with HEC if you want. See opensourced hec client If all indexers are down or not available, the source should stop sending and either queue or propagate the issue down the chain (ie to stop eating new data). For Syslog, Splunk Connect for Syslog has this functionality for example, see disk buffer here ... View more

This has positively changed with version 7.2.0 look at sslRootCAPathHonoredOnWindows in server.conf short story, il will use the new parameters in both linux and windows with 7.2+ (that is by default) It has been backported in minor versions of 7.0 and 7.1 but in that case you have to opt in to use them with this setting. ... View more

The solution in the linked answer is much more safer to use as leverage the os mechanism to give access to systemd from splunk. (with a little bit of help of a additional script to be more fine grained) ... View more

Hmm, The first lines are fine but they require to call sudo The last 2 lines seems risky. Adding a sudo sentence that run the splunk binary as root is kind of defeating the whole initial thing of not having splunk run as root If you are the attacker running as splunk, you could just replace the splunk binary with whatever you want and become easily root.... The proper and secure way is to use to the os way of allowing a unpriviledge process to call systemd, which is policykit. Unfortunaltely, it is os systemd and policykit dependent and requir some conf but that' perfectly work on at least the RH7/8 + Centos7/8 + AWS2 distributions. The answer post linked below is pointing at this solution ... View more

I am not a subject expert on this but if look like the error message say it doesnt find matplotlib library which fbprophet use As this is reported to stderr , that goes up to the fit command which complain Also , please make sure you choose the appropriate version of mltk if the algo you wish add is made for python 3, you probably need to be using mltk 5.0 + splunk 8.0 + adapt the path to not use python 2.x) If the version are unaligned, that could explain the lib not found error. What are the versions you are playing with ? ... View more