Getting Data In

Using Splunk modular data inputs for the REST API to ingest Twitter data, how do I delete or filter out non-English events?

sunnyd
Engager

I am ingesting a lot of Twitter data for a project, and incidentally, I am ingesting Japanese and Hindi tweets along with the English ones. I do not want to collect these tweets, so is there a way to limit the collection to only English?

Or is there a way to delete the non English Twitter data?

I'm using the Splunk Modular Data inputs for the REST API.

Thanks.

0 Karma
1 Solution

gwobben
Communicator

Use a filter! Twitter has a fantastic streaming API which you can use with Splunk. Check out this great tutorial: http://discoveredintelligence.ca/stream-twitter-splunk-10-simple-steps/

Use the language filter in your endpoint (https://dev.twitter.com/streaming/overview/request-parameters#language). For example:
https://stream.twitter.com/1.1/statuses/filter.json?track=twitterapi&language=en

View solution in original post

gwobben
Communicator

Use a filter! Twitter has a fantastic streaming API which you can use with Splunk. Check out this great tutorial: http://discoveredintelligence.ca/stream-twitter-splunk-10-simple-steps/

Use the language filter in your endpoint (https://dev.twitter.com/streaming/overview/request-parameters#language). For example:
https://stream.twitter.com/1.1/statuses/filter.json?track=twitterapi&language=en

Get Updates on the Splunk Community!

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...