Splunk Search

Python Custom Search Command over SSH

ephemeric
Contributor

Greetz,

For security purposes we wish to do a search from an untrusted host (could be compromised)
and therefore cannot allow the remote indexers to be search peers as if the searchhead in question is compromised, they could do searches with deletes.

I don't see any way to control what a searchhead does on an indexer? I saw something about a search peer account
possibly being introduced in a release to restrict privileges?

Can we do a custom search command that does a remote search via SSH that generates events and returns results so it looks like a "local" search?
What would the output format be on the remote side?

Any help is much appreciated!
The SSH part is tested and works.

I have the below but cannot code so it shows, I need help.
I am not sure about the remote output format, must it be parsed, processed etc.?
Can this even be done?

I get "External search command 'helloworld' returned error code 1."

And running the below hangs until interrupted:

[splunk@anomaly splunk]$ ./bin/splunk cmd python etc/apps/search/bin/helloworld.py

^CERROR
"Error : Traceback: Traceback (most recent call last):
File ""etc/apps/search/bin/helloworld.py"", line 14, in <module>
results, dummyresults, settings = splunk.Intersplunk.getOrganizedResults()
File ""/opt/splunk/lib/python2.7/site-packages/splunk/Intersplunk.py"", line 310, in getOrganizedResults
results = readResults(input_str, settings)
File ""/opt/splunk/lib/python2.7/site-packages/splunk/Intersplunk.py"", line 263, in readResults
first = False
KeyboardInterrupt
"

etc/apps/search/local/commands.conf

[helloworld]
filename = helloworld.py
generating = true
streaming = true
supports_rawargs = true

etc/apps/search/bin/helloworld.py

import re,sys,time,subprocess, splunk.Intersplunk

def hello(results, settings):
    results = []
    result = subprocess.check_output("ssh -i /home/splunk/.ssh/id_rsa 10.0.0.242 '/opt/splunkindexer/bin/splunk search 'index=test' -earliest_time -300d@d -latest_time @d -output rawdata -auth admin:changeyou -uri https://127.0.0.1:8089'", shell=True)
    results.append(result)
    return results

results, dummyresults, settings = splunk.Intersplunk.getOrganizedResults()
results = hello(results, settings)
splunk.Intersplunk.outputResults(results)
0 Karma
1 Solution

jdonn_splunk
Splunk Employee
Splunk Employee

Hello,

I just stumbled across your question, and I happen to be developing a very similar Modular Input. Basically, it uses Pexpect to SSH into a device, and executes a list of commands that you configure. The output of each command is a multiline event.

If you are interested in beta testing this App, email me at jdonn@splunk.com.

Jim

View solution in original post

0 Karma

Ayn
Legend

Not following why you need to SSH in instead of just issuing a search using directly using the REST API available on the splunkd port (8089) - firewall issues or...?

0 Karma

jdonn_splunk
Splunk Employee
Splunk Employee

Hello,

I just stumbled across your question, and I happen to be developing a very similar Modular Input. Basically, it uses Pexpect to SSH into a device, and executes a list of commands that you configure. The output of each command is a multiline event.

If you are interested in beta testing this App, email me at jdonn@splunk.com.

Jim

0 Karma

ferdydek
Path Finder

Have you managed to get the app published?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Kick the Tires Before You Commit: A Hands-On Tour of the Splunk Observability Cloud ...

Evaluating an enterprise observability platform usually goes like this: fill out a form, get a free trial with ...

Deep insights, no barriers: Splunk Observability Cloud Free Edition

As software delivery cycles continue to accelerate, observability shouldn’t be a luxury — it should be a ...

Monitoring AI Agents with Splunk Observability Cloud

Let’s say I’m running a travel planning AI app in production. A user asks for three concise hotel options in ...