Splunk Dev
×

Are you a member of the Splunk Community?

Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Ask a Question

JAVA SDK is taking too long time to index data to splunk

royalhrk
Engager
‎03-22-2021 10:29 AM

Hi,

We are using custom java program to index data to splunk using Java SDK (1.6.5.0).  We have huge volume to data to index on daily bases around  aprox.100000 documents.

We used TCP method but it requires a port to open and bind with a specific source to index data. We receive data as bulk (several records) and process it and index each record seperately. to index data as unique event we need to connect TCP port each time and close it to index data. This way it is working very slow (might be  due to connecting and closing TCP port for each record).  We tried the com.splunk.Index API (submit()/ attach()), but index rate is very slow.

We have java parsers that are doing some business specific stuff with data, so dont want to go for forwards to rewrite the data parsing logic.

Can anyone help here and suggest the best practice/approach to index, using java SDK (considering high data velicity).

Thanks.

Labels (2)
Labels
Tags (2)
0 Karma
Reply
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...