How to connect Twitter data to Splunk

[jorubi]

Hi team,

With the new API for Twitter, I've been having difficulty trying to set-up a connection from my host to twitter.
QUESTION: Is there a way to connect Twitter to a server?

Every time I run the curl command on the server, I would get curl: (7) couldn't connect to host. What have I done wrong?

Thank you for helping! 🙂

Code:
*Note: I have the actual values for both: OAUTH_1_Client_Key_VALUE and OAUTH_1_Access_Token_VALUE hiding for privacy purposes

curl

--request POST --url 'https://api.twitter.com/1.1/account_activity/all/SplunkAPI/webhooks.jsonurl=https%3A%2F%2Fsplunk.yooza.tcnz.net'
 --header 'authorization: OAuth oauth_consumer_key="OAUTH_1_Client_Key_VALUE", oauth_nonce="GENERATED", oauth_signature="GENERATED", oauth_signature_method="HMAC-SHA1", oauth_timestamp="GENERATED", oauth_token="OAUTH_1_Access_Token_VALUE", oauth_version="1.0"'

Configuration:
In Splunk:
Data inputs » REST » Twitter

Endpoint URL
https://api.twitter.com/1.1/account_activity/all/SplunkAPI/webhooks.json

URL Arguments:

follow=423424432^stall_warnings=true

In Twitter:

App Name
SplunkAPI

Website URL
https://splunk.yooza.tcnz.net (Is this needed in the technical terms? because we are using a splunk server which wont be avaialble for internet connection)

[jorubi · Accepted solution]

Hi all,

I've solved the problem by opting for a full archive subscription instead.
https://api.twitter.com/1.1/tweets/search/fullarchive/dev.json

Now Splunk and Twitter speak to each other. However, this has a limitation of number of requests made.

[DavidHourani]

is it free ?

[jorubi]

Yes it is but it's not a sustainable solution because there is a threshold of 250 requests.

[evania]

Hi @jorubi ,

Thanks for sharing your answer! If it worked, please resolve this post by approving it! If your problem is still not solved, keep us updated so that someone else can help you.

Thanks for posting!

[DavidHourani]

Hi @jorubi,

The problem is with you curl command. Please use the one from this website :
https://developer.twitter.com/en/docs/accounts-and-users/subscribe-account-activity/quick-start/ente...

curl --request POST --url 'https://api.twitter.com/1.1/account_activity/webhooks.json?url=<URL>' --header 'authorization: OAuth oauth_consumer_key="<CONSUMER_KEY>", oauth_nonce="GENERATED", oauth_signature="GENERATED", oauth_signature_method="HMAC-SHA1", oauth_timestamp="GENERATED", oauth_token="<ACCESS_TOKEN>", oauth_version="1.0"'

Cheers,
David

[jorubi]

The only difference is the path file, 'https://api.twitter.com/1.1/account_activity/webhooks.json?url=' so does that mean that my Endpoint URL would change from
...account_activity/all/SplunkAPI/webhooks.json to .../account_activity/webhooks.json or remain the same?

I changed the curl command to read from https://api.twitter.com/1.1/account_activity/webhooks.json
and did another try that changed the config on Splunk to https://api.twitter.com/1.1/account_activity/webhooks.json instead of .../account_activity/all/SplunkAPI/webhooks.json but I was still getting
curl: (7) couldn't connect to host for both.

CODE:
'https://api.twitter.com/1.1/account_activity/webhooks.jsonurl=https%3A%2F%2Fsplunk.yooza.tcnz.net'
--header 'authorization: OAuth oauth_consumer_key="OAUTH_1_Client_Key_VALUE", oauth_nonce="GENERATED", oauth_signature="GENERATED", oauth_signature_method="HMAC-SHA1", oauth_timestamp="GENERATED", oauth_token="OAUTH_1_Access_Token_VALUE", oauth_version="1.0"'

[jorubi]

When I changed both
Tried doing it and would get
The 403 Forbidden error is an HTTP status code which means that accessing the page or resource you were trying to reach is absolutely forbidden for some reason.

NEW Endpoint URL
https://api.twitter.com/1.1/account_activity/webhooks.json (should this have remained as ...account_activity/all/SplunkAPI/webhooks.json)

CODE:
'https://api.twitter.com/1.1/account_activity/webhooks.jsonurl=https%3A%2F%2Fsplunk.yooza.tcnz.net'
--header 'authorization: OAuth oauth_consumer_key="OAUTH_1_Client_Key_VALUE", oauth_nonce="GENERATED", oauth_signature="GENERATED", oauth_signature_method="HMAC-SHA1", oauth_timestamp="GENERATED", oauth_token="OAUTH_1_Access_Token_VALUE", oauth_version="1.0"'

Getting these logs on Splunk: http_error_code = 401 error_message = {"errors":[{"code":32,"message":"Could not authenticate you."}]}

So sorry newb at this.

[DavidHourani]

Hello @jorubi, is this now working for you ?

[jorubi]

Unfortunately not, I would get "Received HTTP code 403 from proxy after CONNECT".
Also already included export no_proxy=************* in the command

I have a proxy running on our end.