Splunk Enterprise Security

How do I converting a "curl" command to a collection configuration in "Splunk Add-On Builder" console?

yossefn
Path Finder

Hi,

I really need help with this issue.
I need to collect logs using REST from a web resource. I'm trying for a lot of time to do it by myself, unfortunately I got stock in the final step.
I have a 'curl' command I'm running against the web resource and I see the logs on my shell screen, all I'm trying to do is to convert this command to a valid REST call using the 'Add-on Builder' but I just can't finish it successfully.

Attached here is a masked version of my 'curl' command, please help me to get it done somehow so I'll be able to collect the logs.

curl -X POST 'https://api.company.webresource.com/v2/logs/audit' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Authorization:Bearer 196e6h17-798a-4e2c-64hr-xxxxxxxxxxxx' -i -d '{"query":{"from_date":1587848400000}}'

Thanks in advance!

Labels (1)
0 Karma

PavelP
Motivator

Hello @yossefn,

please try to use this config based on this example: https://answers.splunk.com/answers/736820/rest-ta-inputsconf-vs-curl.html

 [rest://test1]
 endpoint = https://api.company.webresource.com/v2/logs/audit
 activation_key = XXXACTIVATIONKEYHEREXXX
 http_method = POST
 auth_type = none
 http_header_propertys = Authorization=Bearer YYYREMOTEACCESSTMCASTOKENHEREYYY
 response_type = json
 polling_interval = 300
 index = your_index
 index_error_response_codes = 0
 sequential_mode = 0
 sourcetype = your_st
 streaming_request = 0

try to put the uploaded json in request_payload field.

Let me know if it worked.

P.S. you can try also use curl script instead of using this TA because your curl request is simple POST without auth. The only thing is the checkpointing (Typically, you store (check point) the progress of an input source so upon restart, the script knows where to resume reading data. This prevents you from reading and indexing the same data twice.)

yossefn
Path Finder

Hi @PavelP
I've created this stanza but still can't see any data coming in. How can I debug it to see the errors?

0 Karma

PavelP
Motivator

Hello @yossefn ,

I've just tested and it works:

[rest://test]
activation_key = 1yourkeyhere1
auth_type = none
endpoint = http://your-server
http_method = POST
index_error_response_codes = 0
request_payload = {"query":{"from_date":1587848400000}}
response_type = text
sequential_mode = 0
sourcetype = rest_ta_1
streaming_request = 0

just fill the UI form (Data Input > REST) with your data, choose Http Method "POST", Request Payload
- {"query":{"from_date":1587848400000}} (without quotes). After you save the configuration, splunk will request the site using API every 60 seconds (can be overwritten in Polling Interval).

0 Karma

yossefn
Path Finder

I don't understand why, but it still not working for me.

Where should I put all the headers I have in the URL? Are the should be included as part of the endpoint value?

'https://api.company.webresource.com/v2/logs/audit' -H 'Content-Type: application/json' -H 'Accept: application/json'

In your second example I can't see the Authorization part..

0 Karma

harsmarvania57
Ultra Champion

Have you looked at app https://splunkbase.splunk.com/app/4146/#/details which has curl command and you can fire POST request.

0 Karma

yossefn
Path Finder

Hi @harsmarvania57 is this app will allow me to configure a continues REST collection? I want to index events from this web resource.

0 Karma

harsmarvania57
Ultra Champion

I'll suggest to try it in test environment. To index data you need to schedule the query with curl command and store the data in index.

0 Karma

yossefn
Path Finder

Sure, I'm working on my test machine. Thanks!

0 Karma
Get Updates on the Splunk Community!

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more with ITSI’s ...

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more! Faster Time to ValueManaging and ...

New Release | Splunk Enterprise 9.3

Hi Splunky people! We are excited to share the newest updates in Splunk Enterprise 9.3!Admins and Analyst can ...

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...