Splunk Search

Search doesn't show data

ragh99
Loves-to-Learn

Hi,

I have just installed Splunk enterprise on-prem and trying to send data using HEC (port 8088). When I do a tcpdump, I do see packets coming in to splunk, but when I do search (  a basic search all using "*"), I do  not see anything. Is there anything basic I might be missing? Thanks, Raghu

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp7s0, link-type EN10MB (Ethernet), capture size 262144 bytes


01:24:01.349607 IP  ip.25648 > worker1.radan-http: Flags [P.], seq 1131684074:1131684105, ack 2035586096, win 502, options [nop,nop,TS
val 2515323145 ecr 929343968], length 31
01:24:01.349652 IP ip..25648 > worker1.radan-http: Flags [F.], seq 31, ack 1, win 502, options [nop,nop,TS val 2515323145 ecr 929343968], length 0
01:24:01.349774 IP worker1.radan-http > ip.25648: Flags [P.], seq 1:32, ack 32, win 1475, options [nop,nop,TS val 929350932 ecr 2515323145], length 31

 

ragh99_0-1609760980829.png

 

Labels (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

Please try using below search with "All Time" on time range. If you see license usage, maybe timestamps are getting wrong or your default indexes does not include that index.

index=*

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @ragh99,

Since we have only tcpdump output I can suggest checking below items;

1- If HEC is enabled on HEC data input global settings? Please check SSL also.

2- On your client do you get HTTP 200 response? If not the error message may help.

3- Firewall enabled? 8088 allowed?

 

If this reply helps you an upvote is appreciated.

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

ragh99
Loves-to-Learn

thanks @scelikok . I think they are enabled . Only thing I see on client side as well is PSH/ACK and I am assuming that the data is being exchanged.  FWIW, I am trying to send logs from a k8s infrastructure, so not sure if HEC is what I should be using? 

 

ragh99_0-1609762124756.png

# netstat -tulpn | grep 8088
tcp 0 0 0.0.0.0:8088 0.0.0.0:* LISTEN 628/splunkd. 

 

ragh99_2-1609763021044.png

 

0 Karma

ragh99
Loves-to-Learn

I do see my host sending (v-test-xxx) some traffic eventhough  its less in license usage.

Could that be reason? Thanks

 

ragh99_1-1609765369087.png

 

 

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Enhance Security Operations with Automated Threat Analysis in the Splunk EcosystemAre you leveraging ...

What Is Splunk? Here’s What You Can Do with Splunk

Hey Splunk Community, we know you know Splunk. You likely leverage its unparalleled ability to ingest, index, ...

Level Up Your .conf25: Splunk Arcade Comes to Boston

With .conf25 right around the corner in Boston, there’s a lot to look forward to — inspiring keynotes, ...