Splunk Search

java curl splunk search produces error in SearchParser - missing a search command

diptij
Path Finder

In Java, I am trying to call a curl command that has a Splunk search to get contents of a lookup file.

I've used https://docs.splunk.com/Documentation/Splunk/8.0.3/RESTTUT/RESTsearches as my starting point.  Too bad they don't show how to use Java like they do for curl and python.

>>>>> The curl command works fine outside of Java.

curl -u admin:password -k https://1.2.3.4:8089/services/search/jobs/export -d output_mode=csv -d search=" | inputlookup hosts-info"

>>>>> Here is the Java program :

import java.io.IOException;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.util.stream.Collectors;

public class tstcurl {

public static void main(String[] args) {

String command = "curl -u admin:password -k https://1.2.3.4:8089/services/search/jobs/export -d output_mode=csv -d search=\" | inputlookup hosts-info\"";

try
{

System.out.println("Creating curl command: [" + command + "]");
Process process = Runtime.getRuntime().exec(command);
String result = new BufferedReader(new InputStreamReader(process.getInputStream())).lines().collect(Collectors.joining("\n"));
System.out.println(result);

}
catch (IOException e)
{

e.printStackTrace();

}

}

>>>>> Output of 'java -jar tst-curl.jar':

Creating curl command: [curl -u admin:password -k https://1.2.3.4:8089/services/search/jobs/export -d output_mode=csv -d search=" | inputlookup hosts-info"]
<?xml version="1.0" encoding="UTF-8"?>
<response>

<messages>

<msg type="ERROR">Error in 'SearchParser': Missing a search command before '"'. Error at position '0' of search query '"'.</msg>

</messages>

</response>

>>>>> Help please

I've done the following:

  1. Looked into /opt/splunk/var/log/splunkd.log for any other messages related to this.
  2. I've turned on debug and no other messages related to this issue are in the splunkd.log
  3. I've search google for anybody else having this issue
  4. I've looked into getting the Splunk SDK but seems like extra effort to just read the lookup file.

If anybody has made this work, please put share your solution.

Labels (1)
Tags (4)
0 Karma
1 Solution

diptij
Path Finder

The Java Runtime().exec(command) taking one string causes issues.

To fix, I did the following:

  1. [line 10] Made command be an array of strings and removed quotes in string after 'search=':
  2. [line 15] Output Strings in Array to see curl command:
    • System.out.println("Creating curl command: " + Arrays.toString(command));

View solution in original post

0 Karma

diptij
Path Finder

Solution with Java HttpsURLConnection and insecure (same as -k option of curl):

import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.net.URL;
import java.util.Arrays;
import java.util.Base64;
import java.util.HashMap;
import java.util.List;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

public class TstSplunkHttpUrl {

private static final String USER = "admin";
private static final String PASSWORD = "password";
private static final String USER_PASS = USER + ":" + PASSWORD;
private static final String SPLUNK_HEAD_IP = "1.2.3.4";
private static final String SPLUNK_HEAD_PORT = "8089";

private static final String SPLUNK_SEARCH_URL = "https://" + SPLUNK_HEAD_IP + ":" + SPLUNK_HEAD_PORT
+ "/services/search/jobs/export";
private static final String PARAM_SPLUNK_SEARCH_CMD = "search=|inputlookup hosts-info";
private static final String PARAM_SPLUNK_SEARCH_OUTPUT_TYPE = "output_mode=csv";
private static final String PARAMS_SPLUNK_SEARCH = PARAM_SPLUNK_SEARCH_CMD + "&" + PARAM_SPLUNK_SEARCH_OUTPUT_TYPE;

public static void main(String[] args) {

     try
     {
          // Create a trust manager that does not validate trust certificate chains
          TrustManager[] trustAllCerts = new TrustManager[]{
               new X509TrustManager()
               {
                     @Override
                     public java.security.cert.X509Certificate[] getAcceptedIssuers()
                     {
                          return null;
                     }

                     @Override
                     public void checkClientTrusted(java.security.cert.X509Certificate[] certs, String authType)
                     {
                     }

            @Override
            public void checkServerTrusted(java.security.cert.X509Certificate[] certs, String authType)
            {
            }
      }
};

                 // Install the all-trusting trust manager
                 SSLContext sc = SSLContext.getInstance("SSL");
                 sc.init(null, trustAllCerts, new java.security.SecureRandom());
                 HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

                 // Create all-trusting host name verifier
                 HostnameVerifier allHostsValid = new HostnameVerifier()
                 {
                       @Override
                       public boolean verify(String hostname, SSLSession session) {
                            return true;
                       }
                 };

                 // Install the all-trusting host verifier
                 HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);

                 // Encode User Password
                 String encodedUserPass = new String(Base64.getEncoder().encode(USER_PASS.getBytes()));
                 String basicAuth = "Basic " + encodedUserPass;

                 // Setup and open URL Connection
                 URL url = new URL(SPLUNK_SEARCH_URL);
                 HttpsURLConnection urlConn = (HttpsURLConnection) url.openConnection();

                 // Setup POST connection
                 urlConn.setRequestMethod("POST");

                 // Setup authorization to URL
                 urlConn.setRequestProperty("Authorization", basicAuth);

                 // Send parameters
                 urlConn.setDoOutput(true);
                 OutputStream outStream = urlConn.getOutputStream();
                 outStream.write(PARAMS_SPLUNK_SEARCH.getBytes());
                 outStream.flush();
                 outStream.close();
 
                 //Get connection response code
                 int urlConnRspCode = urlConn.getResponseCode();
                 if (urlConnRspCode == HttpsURLConnection.HTTP_OK)
                {
                     System.out.println("INFO: URL Connection Response Code = " + urlConnRspCode + "; OK");
                     InputStream resultStream = urlConn.getInputStream();
                     InputStreamReader resultStreamReader = new InputStreamReader(resultStream);
                     BufferedReader resultBufferedReader = new BufferedReader(resultStreamReader);

                     String line;
                     String splitBy = ",";
                     int index=0;

                     while ((line = resultBufferedReader.readLine()) != null)
                     {
                          String[] record = line.split(splitBy);
                          index++;
                          System.out.println("Line #" + index + ":" + Arrays.toString(record));
                     }
               }
               else
               {
                    System.out.println("INFO: URL Connection Response Code = " + urlConnRspCode + "; NOT OK");
               }
          }
         catch (Exception e)
         {
              e.printStackTrace();
         }
     }
}

0 Karma

diptij
Path Finder

The Java Runtime().exec(command) taking one string causes issues.

To fix, I did the following:

  1. [line 10] Made command be an array of strings and removed quotes in string after 'search=':
  2. [line 15] Output Strings in Array to see curl command:
    • System.out.println("Creating curl command: " + Arrays.toString(command));
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...