All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Same here but we use Ansbile.
I had made the cahnge to props.conf but I'm not seeing anything show up today (06/07/24).  06/06/24 logs were fine as I'd expect. I'd like to make sure I did the props.conf right: I put this under ... See more...
I had made the cahnge to props.conf but I'm not seeing anything show up today (06/07/24).  06/06/24 logs were fine as I'd expect. I'd like to make sure I did the props.conf right: I put this under etc/system/local/props.conf [auditd] {is this the right text here?  I thought I had read this should be the sourcetype.  I got the sourcetype from inputs.conf of rlog.sh under the Splunk_TA_nis app.} TIME_FORMAT=%m/%d/%y %T:%3N {I did this to mimic what I see when I do a search from the splunk webserver and this is what shows up at the left before the actual log entry}
Can you screenshot a mock up of the setup screen as you configured it with your details redacted? 
Hello, I need to monitor some critical devices (stored in a lookup file) connected to the Crowdstrike console, in particular if they will be disconnected from it. We receive one event every 2 hours ... See more...
Hello, I need to monitor some critical devices (stored in a lookup file) connected to the Crowdstrike console, in particular if they will be disconnected from it. We receive one event every 2 hours for each device from Crowdstrike device json input in Splunk, so basically if after 2 hours there is not the new event, the alert should trigger reporting the hostname. Has anyone some idea for implementing this?
Did you put "default" as the org
Hi. I am new to splunk. I have configured everything. I am trying to solve this issue for 2 days. I have universal forwerder on the ubuntu server with different network. I have downloaded splunk ent... See more...
Hi. I am new to splunk. I have configured everything. I am trying to solve this issue for 2 days. I have universal forwerder on the ubuntu server with different network. I have downloaded splunk enterprise to my windows 10 computer. My port 9997 is enabled. Firewall is disabled. Even with zyxel interface i bypassed the port 9997. My splunk is listening on port 9997. The thing is with telnet from any other source to my computer (i tried with both my mobile internet and UF client) is still getting denied. How should i proceed to make it work. Im stuck so bad Thanks for your helps this is the mobile internet test with Test-NetConnections to my pc (splunk server i guess) ComputerName : x.x.x.x <desired.connection> RemoteAddress : x.x.x.x <desired connection> RemotePort : 9997 InterfaceAlias : Wi-Fi SourceAddress : X.x.x.x <my ip> PingSucceeded : False PingReplyDetails (RTT) : 0 ms TcpTestSucceeded : False  
We have a pre-production environment which is totally separate to our production environment. A number of our dashboards utilise hyperlinks back into source systems to view complete records, and part... See more...
We have a pre-production environment which is totally separate to our production environment. A number of our dashboards utilise hyperlinks back into source systems to view complete records, and part of the urls are different due to IP addresses on the two environemnts. What I would like to do is be able to set a global setting so that we can utilise the same dashboards without having to recode the link when we migrate from pre-produciton to production. I have tried to use a marco but as these are limited to search it did not effect the link target, is there any other way that we can do this without having to put in steps in migration to ammend the URLs? Thanks, Steven
Hey guys I have a question about CSS in splunk, I want to move these displays like this: And I can't use dashboard studio because I need some special things that dahsboard studio doesn't have ye... See more...
Hey guys I have a question about CSS in splunk, I want to move these displays like this: And I can't use dashboard studio because I need some special things that dahsboard studio doesn't have yet, like depends functionality, I'm trying with css grid but I couldn't find a way to do this yet
Hi gcusello, Thank you for your time to review this, and your quick reply! We do in fact have heavy forwarders at each tenant, so there is local control over any traffic passing through those. That ... See more...
Hi gcusello, Thank you for your time to review this, and your quick reply! We do in fact have heavy forwarders at each tenant, so there is local control over any traffic passing through those. That said though, I am told we have over 40,000 endpoints reporting directly to the cloud , so we definitely do not know the hostnames for everything. Each tenant does have their own HEC key for those submissions, so that would be about the only way to identify and separate that traffic. In my original posting, my intended means to make these changes was via a custom inputs.conf file on all the HFs and UFs rather than modifying the events on arrival, since the inputs file is already at the source. Aside from the overhead and logistics of having to deploy that file to that many workstations and servers, I was wondering if there's any downside to this approach from the perspective of the Splunk agent itself. The ability to do override some/all of any portion of any .conf files is built into the design, so it seems like a custom inputs .conf wouldn't be a problem in any way. Any feedback on that approach? Thank you!
I just posted a question that was immediately rejected. How do I get it approved please?
I use the OpenTelemetry Java agent to monitor FusionAuth in one Docker container, and send the output to the Splunk OpenTelemetry Docker container in Gateway mode.   Here's a diagram of my system a... See more...
I use the OpenTelemetry Java agent to monitor FusionAuth in one Docker container, and send the output to the Splunk OpenTelemetry Docker container in Gateway mode.   Here's a diagram of my system architecture:   ```mermaid graph LR subgraph I[Your server] direction LR subgraph G[Docker] H[(Postgresql)] end subgraph C[Docker] direction BT D(OpenTelemetry for Java) --> A(FusionAuth) end subgraph E[Docker] B(Splunk OpenTelemetry collector) end end C --> G C --> B E --> F(Splunk web server) style I fill:#111 ```   The Splunk container runs correctly and exports sample data to Splunk Observability Cloud. I can see it in the dashboard.   FusionAuth and the Java agent run correctly. But the Otel sender cannot send to the Otel collector. I get network errors: ```sh | [otel.javaagent 2024-06-07 13:52:40:936 +0000] [OkHttp http://otel:4317/...] ERROR io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export logs. The request could not be executed. Full error message: Connection reset fa | java.net.SocketException: Connection reset fa | at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:328) fa | at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:355) ...   [otel.javaagent 2024-06-07 13:52:42:847 +0000] [OkHttp http://otel:4317/...] ERROR io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export spans. The request could not be executed. Full error message: Connection reset by peer fa | java.net.SocketException: Connection reset by peer fa | at java.base/sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:425) fa | at java.base/sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:445) fa | at java.base/sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:831) fa | at java.base/java.net.Socket$SocketOutputStream.write(Socket.java:1035) ```   I'm using the standard configuration file for Splunk Linux Collector - https://github.com/signalfx/splunk-otel-collector/blob/main/cmd/otelcol/config/collector/otlp_config_linux.yaml Below is my docker compose file   ```yaml services: db: image: postgres:latest container_name: fa_db ports: - "5432:5432" environment: PGDATA: /var/lib/postgresql/data/pgdata POSTGRES_USER: ${POSTGRES_USER} POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} healthcheck: test: [ "CMD-SHELL", "pg_isready -U postgres" ] interval: 5s timeout: 5s retries: 5 networks: - db_net volumes: - db_data:/var/lib/postgresql/data fa: # image: fusionauth/fusionauth-app:latest image: faimage container_name: fa # command: "tail -f /dev/null" depends_on: db: condition: service_healthy environment: DATABASE_URL: jdbc:postgresql://db:5432/fusionauth DATABASE_ROOT_USERNAME: ${POSTGRES_USER} DATABASE_ROOT_PASSWORD: ${POSTGRES_PASSWORD} DATABASE_USERNAME: ${DATABASE_USERNAME} DATABASE_PASSWORD: ${DATABASE_PASSWORD} FUSIONAUTH_APP_MEMORY: ${FUSIONAUTH_APP_MEMORY} FUSIONAUTH_APP_RUNTIME_MODE: ${FUSIONAUTH_APP_RUNTIME_MODE} FUSIONAUTH_APP_URL: http://fusionauth:9011 SEARCH_TYPE: database FUSIONAUTH_APP_KICKSTART_FILE: ${FUSIONAUTH_APP_KICKSTART_FILE} networks: - db_net ports: - 9011:9011 volumes: - fusionauth_config:/usr/local/fusionauth/config - ./kickstart:/usr/local/fusionauth/kickstart extra_hosts: - "host.docker.internal:host-gateway" otel: image: quay.io/signalfx/splunk-otel-collector:latest container_name: fa_otel environment: SPLUNK_ACCESS_TOKEN: "secret" SPLUNK_REALM: "us1" SPLUNK_LISTEN_INTERFACE: "0.0.0.0" SPLUNK_MEMORY_LIMIT_MIB: "1000" SPLUNK_CONFIG: /config.yaml volumes: - ./config.yaml:/config.yaml networks: - db_net # no host ports are needed as communication is inside the docker network # ports: # - "13133:13133" # - "14250:14250" # - "14268:14268" # - "4317:4317" # - "6060:6060" # - "7276:7276" # - "8888:8888" # - "9080:9080" # - "9411:9411" # - "9943:9943" networks: db_net: driver: bridge volumes: db_data: fusionauth_config: ```   The FusionAuth Dockerfile starts FusionAuth like this:   ```sh exec "${JAVA_HOME}/bin/java" -javaagent:/usr/local/fusionauth/otel.jar -Dotel.resource.attributes=service.name=fusionauth -Dotel.traces.exporter=otlp -Dotel.exporter.otlp.endpoint=http://otel:4317 -cp "${CLASSPATH}" ${JAVA_OPTS} io.fusionauth.app.FusionAuthMain <&- >> "${LOG_DIR}/fusionauth-app.log" 2>&1 ``` Why can't the FusionAuth container connect to http://otel:4317 please?
Hello, I've recently tested a sourcetype for a new input via the props.conf file on my standalone dev environment, and it worked perfectly -datas were correctly parsed -. But when I put it in my pro... See more...
Hello, I've recently tested a sourcetype for a new input via the props.conf file on my standalone dev environment, and it worked perfectly -datas were correctly parsed -. But when I put it in my prod environment, the data which where attributed the sourcetype weren't parsed at all. Now, my prod environment is distributed (HFs->DS->Indexers->SH) but I've been careful to put the sourcetype both in the Heavy forwarder and in the searchhead as recommended, and i've restart both the HF and the SH but it still doesn't work. Does anyone have an idea of what I can do to fix it?
Hello, we have a Red status for Ingestion Latency,  it says the following:    Red: The feature has severe issues and is negatively impacting the functionality of your deployment. For details, see R... See more...
Hello, we have a Red status for Ingestion Latency,  it says the following:    Red: The feature has severe issues and is negatively impacting the functionality of your deployment. For details, see Root Cause.   However, I can't figure out how to see the "Root Cause".  What report should I look at, that would show me where this latency is occurring? Thanks for all of the help, Tom  
I use Eggplant https://www.keysight.com/us/en/product/EG1000A/eggplant-test.html    ( https://support.eggplantsoftware.com/downloads/eggplant-functional-downloads ) Eggplant understands GUI controls... See more...
I use Eggplant https://www.keysight.com/us/en/product/EG1000A/eggplant-test.html    ( https://support.eggplantsoftware.com/downloads/eggplant-functional-downloads ) Eggplant understands GUI controls natively - doesn't just check the HTML source - it actually checks what the user sees. Had great success testing swing clients, mobile apps across multiple platforms. You would probaly have to write some scripting to check the math.  
Hello @deepakc, thank you for your effort to respond. Maybe I was unclear with my question, let me rephrase it. TL;DR: how can changes on forwarder affect search-head's behavior during search-time? ... See more...
Hello @deepakc, thank you for your effort to respond. Maybe I was unclear with my question, let me rephrase it. TL;DR: how can changes on forwarder affect search-head's behavior during search-time? My primary goal is to understand how the extractions work particularly in this detail. Longer version: The issue here is with search-time extractions. To my knowledge, these extractions are done by search-head and/or respective indexer when the data is being searched (= after all writing to the index is finished, no matter how the data was cooked). Forwarders are not involved in searching, therefore I don't understand how forwarder can affect the search. The events themselves seem to be indexed the same way before and after upgrade, I don't see any distinction in _raw field contents and generally no reason for search-time extractions to stop working. What in my assumptions and knowledge above is not correct? Your suggestions apply - as of my knowledge - to index-time extractions, however that is not the case here.
Hello @Amadou , I believe SSE should be used in conjunction with Splunk Enterprise Security. The correlation searches within ES, ESCU, and SSE have MITRE tactics associated with each of them. The nu... See more...
Hello @Amadou , I believe SSE should be used in conjunction with Splunk Enterprise Security. The correlation searches within ES, ESCU, and SSE have MITRE tactics associated with each of them. The number of correlation searches with MITRE tactics are highlighted with dark color on the SSE dashboard and the less number would be light in shade.  I do not think this is possible to achieve on Splunk Enterprise. However, if you are able to achieve this through custom solution using .css or .js, let the community know.   Thanks, Tejas.
Hello @deangoris, The Splunk packaging toolkit will also work in the similar fashion. The packages created using the toolkit should not have the local folder within it. Otherwise it'll fail on the U... See more...
Hello @deangoris, The Splunk packaging toolkit will also work in the similar fashion. The packages created using the toolkit should not have the local folder within it. Otherwise it'll fail on the UI itself. The best way to deal with this situation is to have a Barebone app created from the UI and have all the KOs migrated to the custom private app. This way it helps modifying the objects from UI in future as well.   Thanks, Tejas.   --- If the above solution helps, an upvote is appreciated..!! 
Hi Team, Need your assistance for the configuration changes in Splunk. The requirement is to change the Timezone based on different “source” (not sourcetype). We have different sources defined in o... See more...
Hi Team, Need your assistance for the configuration changes in Splunk. The requirement is to change the Timezone based on different “source” (not sourcetype). We have different sources defined in our application. All of them are in their respective server timezone, except for the below 2 sources (these 2 are in EST timezone & our requirement is to change it into CET timezone)     source=/applications/testscan/*/testscn01/* source=/applications/testscan/*/testcpdom/*     For rest of the other sources, I do not want make any change in the Timezone. For example:     source=/applications/testscan/*/testscn02/* source=/applications/testscan/*/testnycus/* source=/applications/testscan/*/testnyus2/* source=/applications/testscan/*/testshape/* source=/applications/testscan/*/testshape2/* source=/applications/testscan/*/testshape3/*     Please note, we do not have any "props.conf" file available or configured in the server.  We are maintaining splunk configuration in only "inputs.conf" file. The present content of "inputs.conf" as below:     [monitor:///applications/testscan/.../] whitelist = (?:tools\/test\/log\/|TODAY\/LOGS\/)*\.(?:log|txt)$ index = testscan_prod sourcetype = testscan _TCP_ROUTING = in_prod [monitor:///applications/testscan/*/*/tools/test_transfer/log] index = testscan_prod sourcetype = testscan _TCP_ROUTING = in_prod [monitor:///applications/testscan/*/*/tools/test_reports/log] index = testscan_prod sourcetype = testscan _TCP_ROUTING = in_prod       Please suggest what changes to be done so that Timezone can be managed based on the "source" information provided. @ITWhisperer 
In this case a full file systems caused this file below to be empty, even after splunkd restart, it was still empty.. That was the cause of this error: [splunk@hf001 ~]$ ll /opt/splunk/quarantined_f... See more...
In this case a full file systems caused this file below to be empty, even after splunkd restart, it was still empty.. That was the cause of this error: [splunk@hf001 ~]$ ll /opt/splunk/quarantined_files/ total 8 -rwxr-x--- 1 splunk splunk 0 Jun 7 14:18 quarantine_manifest.json -rwxr-x--- 1 splunk splunk 208 Mar 16 2023 README.md Adding Enterprise default config to the file solved the issue: [splunk@hf001 ~]$ cat /opt/splunk/quarantined_files/quarantine_manifest.json {"enable_jQuery2": "not-restricted", "enable_unsupported_hotlinked_imports": "not-restricted"}
So basically I'd like to do concatenation between DeviceProcess and DeviceRegistry events in advanced hunting query | advhunt