How to search for data

[juanherrera]

Hello there,

In our company we've been using Splunk for a while now but I think we use it not to it's full potential.
Let me explain:

We just logged a string from our apps, the go to a web site Splunk…..:8000 and then do a search, we have a hard time understanding the way we should look for stuff and we learned that if you put things like error or * we can see what is going on, sometimes we are really wild and we do some other strange searches.

We have that single splunk web server and realized that no matter what application is sending the string to splunk we can't really differentiate what environment send it in. I've been looking for a while and I've come up with domain=domain.com but none of our sites came up, so I was wondering 2 things:

1 - Do we need to send the string to Splunk in a special way for this to work?

2 - I've read a little about indexers and I wonder if this is the way to go to differentiate different environments from sending data to a single Splunk web server, and if this is the way to go, how do I search for this particular data after it has been sent.

I'm having a hard time starting from scratch on this as I can't find a very easy tutorial that will help me get off the ground with Splunk.

ANY help will be highly appreciated!

[ChrisG]

You might also want to review the resources listed in the Hungry Newbie post on this site.

[woodcock]

Yes, and be sure to spread around the UpVotes.

[woodcock · Accepted solution]

It is very hard to give a specific answer without much more specific details. It looks like you(r team) have not taken any/enough training so I suggest that everybody go take Fundamentals 1 which is free. Also, you should have been given some free training credits when you purchased your Splunk license and these often go unused. Call Splunk, locate your sales team and ask them if your company has any unused training credits and USE THEM. Take Fundamentals 2 next and then Advanced Searching and Reporting. Also the admin classes will help because it sounds like your data was not properly curated on the way in (which is exceedingly common). You would probably benefit greatly from a Health Check which may company and many others offer. Often we allow users to shoulder surf during the Health Check process and you can learn a great deal in that process.

[juanherrera]

Thank you so much for your answer. After I posted I started searching more and more online to see ways to do what we wanted to do but pretty soon I realized that we are not sending the right data to Splunk to fully use to it's potential. First of all we send just strings of text, not key/value pairs, so we can't filter down on available information that can be useful to us (more logging type of strings "This happened" or "This didn't happen" instead) also we are sending everything to the same indexer making it very difficult to have separation among environments and everything is in one server so searches are not optimized.

I started with a free tutorial videos from splunk.com/education and while I still have a long way to go I started understanding why we were not getting what we needed from Splunk.

I tried giving you points for your answer but a message appeared telling me that if I award you points I won't be able to post any more question and no points were awarded, sorry!

[woodcock]

The "easy button" is UpVote; this does not cost you any Karma but does give Karma to the people that you `UpVote.

[juanherrera]

Yeah, I did that as soon as I ended writing my answer. Hope I did it right.

[woodcock]

You did, and I did you back.