Hi. I'm new to Splunk. I've got basic import and searching working on the windows install, but I want to get the field names too, and 9 attempts later, I still can't get it to work.
I'm starting with a tab-delimited text file as source. E.g.:
19/09/2011 20:39:30 Red Square
19/09/2011 22:04:02 Green Bob Square
19/09/2011 22:05:26 Blue Triangle
Note: some field values are empty for some events.
I did not include the field names in the first line - I read about CHECK_FOR_HEADER
causing more problems than it is worth.
I add the data via the Splunk web interface: "Add data" -> "A file or directory of files" -> "Consume any file on this Splunk server" -> "Preview data before indexing"...
Trying "Start a new source type" -> "adjust timestamp and event break settings" -> "Every line is one event ex: access logs"
Did not import the field names, which is understandable.
I've never seen my field names come in properly, but I am looking for them in the search results page below "Field discovery is: On" - when I "View all Fields", none of them have field names I care about. I also try changing to "Table View" and all of the data is still under one column called "_raw".
So I followed the instructions here and edited the files at $SPLUNK_HOME/etc/system/local
to include this in props.conf:
[My Source Type 1]
MAX_TIMESTAMP_LOOKAHEAD = 20
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
pulldown_type = 1
REPORT-myname = mydelim
And include this in transforms.conf:
[mydelim]
DELIMS = "\t"
FIELDS = "TimeStamp","Colour","First Name","Shape"
I also tried
[mydelim]
DELIMS = "\t"
FIELDS = f1,f2,f3,f4
I had to restart the splunk server to get the changes to be recognised.
Then I followed the same procedure above for adding data, but this time chose "Apply an existing source type" and chose "My Source Type 1".
Unfortunately, neither attempt worked, nor did a bunch of subtle variations. I am deleting all data between tests.
What am I doing wrong?
Can someone please confirm I am looking for field names in the correct way?
Many thanks.
I finally figured this out - I had edited the original post to use the REPORT thing. This actually worked, I just did not realise it - my two tests for looking for field names were bad tests - they did not show any field names even though there were now extracted field names!
This is pretty bizzare, and in my opinion, there should be something in the search GUI that can show all field names - maybe there is a way and I don't see it?
But oh well, but maybe I'll get used to it.
Part of what threw me is that I thought "View All 7 Fields" referred to all fields in existence, but no - it only refers to fields THAT ARE VISIBLE in the search results and you have to MAKE FIELDS VISIBLE MANUALLY. So you get into a chicken-and-egg situation where the fields you are interested in are not listed because you have not made them visible and you can't make them visible because they are not listed! I'd love to know of a good solution to this - I've found a crude hack below.
The best tests I can find now are:
Colour=Blue
sourcetype="My Source Type 1" | stats dc(*) as *
I hope that helps someone.
I finally figured this out - I had edited the original post to use the REPORT thing. This actually worked, I just did not realise it - my two tests for looking for field names were bad tests - they did not show any field names even though there were now extracted field names!
This is pretty bizzare, and in my opinion, there should be something in the search GUI that can show all field names - maybe there is a way and I don't see it?
But oh well, but maybe I'll get used to it.
Part of what threw me is that I thought "View All 7 Fields" referred to all fields in existence, but no - it only refers to fields THAT ARE VISIBLE in the search results and you have to MAKE FIELDS VISIBLE MANUALLY. So you get into a chicken-and-egg situation where the fields you are interested in are not listed because you have not made them visible and you can't make them visible because they are not listed! I'd love to know of a good solution to this - I've found a crude hack below.
The best tests I can find now are:
Colour=Blue
sourcetype="My Source Type 1" | stats dc(*) as *
I hope that helps someone.