Splunk Search

Why am I unable to get started with searching real-time data coming in on a TCP port?

New Member

Hi All

I may be getting old and senile, but I just can't seem to get started with searching. I have added a TCP source with a datatype (an IP port etc on which I can see incoming strings with Hyper Terminal) and then I select Search which indicates no data has been added, so I select 'please add data' which takes me straight back to Monitor which of course reports my port has already been set up. When I initially set up the port, I was prompted for 'sample data before the current datetime' . As this is real-time data, then there is no such thing as a sample (unless I am supposed to store a sample in a text file or something). I can't seem to get back to that window.

I get the impression that Splunk cannot monitor real-time incoming data on a TCP port (simple strings coming in at one per minute) search for some specific characters in it and then produce a report / chart based upon it (using a lookup table). Am I wrong?

Thanks
Active

Tags (3)
0 Karma

New Member

Hi again

I think I have answered my own question. Splunk appears to require installing under a domain account (requiring a Windows Server / Active Directory) before it's TCP Server will allow remote connections. A nice-to-have for the future would be the ability to receive data from remote devices without requiring a domain account.

Regards
Active.

0 Karma

Esteemed Legend

I don't do much work with windows but this would surprise me if it is true, especially because this is not listed as a requirement/limitation anywhere in the documentation. I would definitely submit a support ticket for clarification and inside it request that the documentation be updated to reflect this (think about the next guy).

0 Karma

Esteemed Legend

Inside your inputs.conf stanza you must specify a sourcetype= and an index= setting; have you done this? Have you restarted the server that is listening since your last configuration changes were deployed?

0 Karma

New Member

Hi Woodcock

I think I may have misunderstood what Splunk was capable of. Despite setting the source type and index (re my original post) there was still no output / sample data and I think this is because Splunk acts as a TCP server listening to the output of 'other' servers (rather than being able to launch TCP clients (similar to what Hyper Terminal does) which then listen to remote TCP servers, i.e. devices). That is why the 'suggested port' is 9997 which would generally be a single local server port and not multiple remote devices.

Regards
Active.

0 Karma

Esteemed Legend

Your term datatype is not in Spunk lexicon so nobody here will know what you mean. In any case... Yes, your TCP input creates a Splunk TCP listener on the port you specified on the Splunk server (now a Forwarder) where you edited inputs.conf. You can test the fuction of this listener by using your favorite tool (hyperterm, telnet, netcat, etc.) to connect to the port and send data. The data that you send to this port should be immediately searchable.
It does seem as though you were expecting it to work differently than it does.

0 Karma

New Member

Hi again

To prove a point I changed my devices to be TCP Clients and also altered the string outputs to be fixed length, csv. I then ran a test TCP Server on the PC (port 7000) and I can see all my clients pushing data into this TCP server. I then shutdown the TCP server rebooted the PC and ran Splunk Light with a TCP input running on port 7000, source type set to structured csv, default index. I have a utility which shows Splunk supposedly running on port 7000 yet Start Searching shows no results found (o events) despite refreshing. I then ran Hyper Terminal on the same server as Splunk and search then showed data. However running Hyper Terminal on any external server (on the same subnet as the Splunk server) search then shows nothing. It seems that Splunk cannot accept data from any 'external devices'. I have not set a host restriction. Re-running a conventional TCP server accepts inputs from both the local and remote PC Hyper Terminal and remote devices. I may be doing something fundamentally wrong but I cannot see what.

Thanks
Active

0 Karma

Esteemed Legend

Post the portions of these files which you have modified/added:

inputs.conf
props.conf
transforms.conf

Also post the output of Splunk when you restart the app on your forwarder:

$SPLUNK_HOME/bin/splunk restart

If there is a big problem with your configuration files, Splunk will complain about it when you (re)start it.

0 Karma

New Member

Hi again, note that I am using the Splunk home page to set the parameters, I am not editing the conf files directly.

There are 2 inputs.conf files 1 in etc\system\local (which has [default} and host = mypc) the other (later) in \etc\apps\search\local (which is empty).

props.conf (2 copies, this is the latest in time) -

[first_install-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[export_metrics-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

transforms.conf is rather large and I cannot see any specific areas modified when I set up the Data Input.

There does not appear to be a restart command that I can run from a command prompt.

I believe the 'restriction' is more fundamental, is the Light version of Splunk limited to only listening for local clients (and refuses external clients), whereas the Enterprise version allows external clients?

I deleted and re-established the Data Input and restarted the PC, the outcome was the same.

Regards
Active

0 Karma