topic Re: Is there a way I can see what data is being indexed on a specific port? in Splunk Search

Is there a way I can see what data is being indexed on a specific port?

janderson19 — Wed, 15 Jun 2016 15:35:00 GMT

Hello,

In the last year, I became the manager of a Splunk system with 0 documentation. All logs were being thrown into index=main, and the only information I can find is in inputs.conf, which is not very helpful:

[splunktcp://50200]
connection_host = ip

[splunktcp://50201]
connection_host = ip

[splunktcp://50202]
connection_host = ip

[splunktcp://42500]
connection_host = ip

[splunktcp://55555]
connection_host = ip

[splunktcp://50203]
connection_host = ip
disabled = 0

[splunktcp://51225]
connection_host = ip

[splunktcp://51125]
connection_host = ip

[splunktcp://514]
connection_host = ip
disabled = 0

[splunktcp://40100]
connection_host = ip
disabled = 0

[splunktcp://50000]
connection_host = ip
disabled = 0

[splunktcp://40300]
connection_host = ip
disabled = 0

[splunktcp://41000]
connection_host = ip
disabled = 0

[splunktcp://42000]
connection_host = ip
disabled = 0

[splunktcp://50100]
connection_host = ip
disabled = 0

I would like to find what data is coming in on these ports, set them all up to come in on 9997, and send them to their own index, so that I can allow the managers of that data to securely access that data, without being able to access logs that are not theirs (via a local role that only allows one or two indexes). Is there any way I can see what data is coming in on what port, or will I have to manually go through and set each port to it's own index or sourcetype to find out?

Thanks.

Re: Is there a way I can see what data is being indexed on a specific port?

Raghav2384 — Wed, 15 Jun 2016 15:43:50 GMT

Hello,

Use this search to list all the hosts connected and sending data to your splunk instance

index=_internal source=*metrics.log group=tcpin_connections 
 | eval sourceHost=if(isnull(hostname), sourceHost,hostname) 
 | rename connectionType as connectType
 | eval connectType=case(fwdType=="uf","univ fwder", fwdType=="lwf", "lightwt fwder",fwdType=="full", "heavy fwder", connectType=="cooked" or connectType=="cookedSSL","Splunk fwder", connectType=="raw" or connectType=="rawSSL","legacy fwder")
 | eval version=if(isnull(version),"pre 4.2",version)
 | rename version as Ver 
 | fields connectType sourceIp sourceHost destPort kb tcp_eps tcp_Kprocessed tcp_KBps splunk_server Ver
 | eval Indexer= splunk_server
 | eval Hour=relative_time(_time,"@h")
 | stats avg(tcp_KBps) sum(tcp_eps) sum(tcp_Kprocessed) sum(kb) by Hour connectType sourceIp sourceHost destPort Indexer Ver
 | fieldformat Hour=strftime(Hour,"%x %H")

You can then expand this search by using sourceHost,IP and get actual "log sources"

Hope this helps!

Thanks,
Raghav

Re: Is there a way I can see what data is being indexed on a specific port?

janderson19 — Wed, 15 Jun 2016 15:47:59 GMT

Thanks! This worked perfectly!

Re: Is there a way I can see what data is being indexed on a specific port?

somesoni2 — Wed, 15 Jun 2016 15:50:28 GMT

These seems to be TCP data inputs and since there are no values explicitly defined for index/sourcetype, they are going to default places. Not sure if you can migrate all to use same port (9997), but you can keep the same port configuration, assign index/sourcetype explicitly in the inputs.conf. See this for more info on TCP data inputs

http://docs.splunk.com/Documentation/Splunk/6.4.1/Data/Monitornetworkports#Configure_a_TCP_input