Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- About nicholasgrabows
nicholasgrabows
Path Finder
Member since:
04-22-2011
06-05-2020
Community Statistics
| Posts | 19 |
| Solutions | 2 |
| Karma Given | 0 |
| Karma Received | 3 |
| Member Since | 04-22-2011 |
Activity Feed
- Got Karma for Re: CSV Lookup File From Share. 06-05-2020 12:46 AM
- Got Karma for Re: Issue with CSV import. 06-05-2020 12:46 AM
- Got Karma for Re: Can splunk Index data from an HTTP address. 06-05-2020 12:45 AM
- Posted Re: Issue with CSV import on All Apps and Add-ons. 01-10-2013 10:32 PM
- Posted Re: Issue with CSV import on All Apps and Add-ons. 01-08-2013 11:33 AM
- Posted Re: CSV Lookup File From Share on Splunk Search. 01-08-2013 11:18 AM
- Posted Re: Injest JSON document returned from an monitoring endpoint on Getting Data In. 12-27-2012 12:07 PM
- Posted Re: Can splunk Index data from an HTTP address on Getting Data In. 12-27-2012 12:00 PM
- Posted Re: Enable FTP on Splunk Search. 12-27-2012 11:41 AM
- Posted Re: Fetch data from FTP Server on Reporting. 12-27-2012 11:39 AM
- Posted Re: CSV Lookup File From Share on Splunk Search. 12-27-2012 11:31 AM
- Posted Re: Indexing Online CSV Files on Getting Data In. 12-27-2012 11:28 AM
- Posted Re: Splunk indexer tiers - Can we have a second tier for "cold" data only? on Getting Data In. 11-15-2012 02:51 PM
- Posted Re: Splunk indexer tiers - Can we have a second tier for "cold" data only? on Getting Data In. 11-15-2012 02:42 PM
- Posted Re: Splunk indexer tiers - Can we have a second tier for "cold" data only? on Getting Data In. 11-15-2012 11:38 AM
- Posted Re: Reduce the size of an index by eliminating parts of index on Deployment Architecture. 11-14-2012 04:47 PM
- Posted Re: Reduce the size of an index by eliminating parts of index on Deployment Architecture. 11-14-2012 04:40 PM
- Posted Re: Reduce the size of an index by eliminating parts of index on Deployment Architecture. 11-14-2012 02:59 PM
- Posted Re: Splunk indexer tiers - Can we have a second tier for "cold" data only? on Getting Data In. 11-14-2012 12:11 PM
- Posted Re: Splunk indexer tiers - Can we have a second tier for "cold" data only? on Getting Data In. 11-14-2012 11:37 AM
Topics I've Started
01-10-2013
10:32 PM
I've tested with a similar file on windows 7 with splunk 4.2.3 and it appears to work. One other question. What version of importutil are you using? 1.0 beta1 (i.e. the latest version)? If not try downloading the latest version and reinstalling. Assuming you are using the latest, can you try executing importutil from the command line:
cd %SPLUNK_HOME%\etc\apps\importutil\bin
%SPLUNK_HOME%\bin\splunk cmd python importutil.py http http://yourhost/your.csv
Email the results to splunk@ngsoft.org. Also, would you mind sending me a copy of the csv file.
... View more
01-08-2013
11:33 AM
1 Karma
paddy, The scenario you described works for me. I was able to read in and display all fields from a csv file that looks like
EventValueFilter,Duration,Earliest,Limit
"EVENTA","1000"," ","| head 5 "
"EVENTB","2000"," ","| head 5 "
"EVENTC","3000"," ","| head 5 "
I've only tested on a Mac, but it should work for splunk running on unix and windows. What OS are you running on? What version of splunk? Can you send a screen shot of the output you are seeing?
Regarding the map issue. I'm not sure I understand the question. Can you explain in more detail.
... View more
01-08-2013
11:18 AM
paddy, good luck. If this ends up working for you and if you have a moment please rate the app on splunkbase. Thanks. Good Luck!
... View more
12-27-2012
12:07 PM
You could also try this a new splunkbase app called "importutil". It lets you import input from an http, ftp, or sftp url via the splunk search command line. sftp is experimental.
http://splunk-base.splunk.com/apps/69078/importutil
Here is an http example that imports data from the federal reserve economic data website:
|importutil http http://research.stlouisfed.org/fred2/data/PAYEMS.csv
| multikv
| table DATE, VALUE
Set this up as a scheduled search. So it will hit the URL periodically.
... View more
12-27-2012
12:00 PM
1 Karma
There is a new splunkbase app called "importutil". It lets you import input from an http url via the splunk search command line. Also works for ftp. sftp is experimental.
http://splunk-base.splunk.com/apps/69078/importutil
Here is an ftp example. Pulling from the bureau of labor stats:
|importutil ftp ftp://ftp.bls.gov/pub/time.series/ce/ce.data.102.WeeklyEarningsHist
| multikv
| table series_id, year, period, value, footnote_codes
Here is an http example that imports data from the federal reserve economic data website:
|importutil http http://research.stlouisfed.org/fred2/data/PAYEMS.csv
| multikv
| table DATE, VALUE
... View more
12-27-2012
11:41 AM
There is a new splunkbase app called "importutil". It lets you import csv files (or any input) from an http url via the splunk search command line. Also works for ftp. sftp is experimental.
http://splunk-base.splunk.com/apps/69078/importutil
Here is an ftp example. Pulling from the bureau of labor stats:
|importutil ftp ftp://ftp.bls.gov/pub/time.series/ce/ce.data.102.WeeklyEarningsHist
| multikv
| table series_id, year, period, value, footnote_codes
Here is an example that imports data from the federal reserve economic data website:
|importutil http http://research.stlouisfed.org/fred2/data/PAYEMS.csv
| multikv
| table DATE, VALUE
... View more
12-27-2012
11:39 AM
There is a new splunkbase app called "importutil". It lets you import csv files (or any input) from an http url via the splunk search command line. Also works for ftp. sftp is experimental.
http://splunk-base.splunk.com/apps/69078/importutil
Here is an ftp example. Pulling from the bureau of labor stats:
|importutil ftp ftp://ftp.bls.gov/pub/time.series/ce/ce.data.102.WeeklyEarningsHist
| multikv
| table series_id, year, period, value, footnote_codes
Here is an example that imports data from the federal reserve economic data website:
|importutil http http://research.stlouisfed.org/fred2/data/PAYEMS.csv
| multikv
| table DATE, VALUE
... View more
12-27-2012
11:31 AM
1 Karma
There is a new splunkbase app called "importutil". It lets you import csv files (or any input) from an http url via the splunk search command line. Also works for ftp. sftp is experimental. Search for importutil on Splunkbase
Here is an example that imports data from the federal reserve economic data website:
|importutil http http://research.stlouisfed.org/fred2/data/PAYEMS.csv
| multikv
| table DATE, VALUE
Here is an example that imports data from the Bureau Of Labor Statistics... it uses ftp:
|importutil ftp ftp://ftp.bls.gov/pub/time.series/ce/ce.data.102.WeeklyEarningsHist
| multikv
| table series_id, year, period, value, footnote_codes
... View more
12-27-2012
11:28 AM
There is a new splunkbase app called "importutil". It lets you import csv files (or any input) from an http url via the splunk search command line. Also works for ftp. sftp is experimental.
http://splunk-base.splunk.com/apps/69078/importutil
Here is an example that imports data from the federal reserve economic data website:
|importutil http http://research.stlouisfed.org/fred2/data/PAYEMS.csv
| multikv
| table DATE, VALUE
... View more
11-15-2012
02:51 PM
One option I was considering... would appreciate some thoughts on this.
What if we created a second Index and moved all data older than 90 days into the second Index. Then all searches could just look like "(index=primary OR index=secondary) earliest=-180d blah blah blah". Is it poossible to pin certain Indexes to only certain indexer nodes?
... View more
11-15-2012
02:42 PM
We are using Summary indexing heavily and I guess we could get smarter about the older data... but the use case is a "we're not sure what to look for until the time comes" kind of thing. So SI isn't sufficient in this case.
... View more
11-15-2012
11:38 AM
sdvorak_splunk, thanks for the response. I'll leave this open for a few more days to see if anyone else has any thoughts.
... View more
11-14-2012
04:47 PM
Ayn, this is what I was looking for. Thanks so much.
... View more
11-14-2012
04:40 PM
I guess I could say the same thing about common nomenclature... At any rate, is there a name for the indexes inside a splunk Index? If there is, happy to use it.... Thanks for the answers below... I will investigate and get back to you.
... View more
11-14-2012
02:59 PM
The logs are broken by timestamp. here is an example event:
Date=11-10-2012 00:00:00, URL="http://www.someurl/path1/path2/path3/path4"
The question is how can I reduce the size/number of indexes created for this sourcetype. I'm using "index" here in the common usage (i.e. index on a relational data table), when splunk uses the word Index they mean a set of files some of which are raw data files and some of which are index files.
... View more
11-14-2012
12:11 PM
Thanks for the possible alternatives. What about the 2 questions posed in the original post?
... View more
11-14-2012
11:37 AM
We worry about the reliability of NAS. Also, we view fast access to the data as critical. Basically we need relatively fast access to a large amount of old data.
... View more
11-14-2012
11:07 AM
We have a fairly large indexing cluster. We tend to get about a 2x compression rate on our raw data. In other words, we can store about 2 GB of raw data + indexes on 1GB of disk. We were hoping to get much better compression rates. Given our data is text we believe 5 to 10x is possible. It was suggested to us that the reason we are only getting 2x is due to the amount of indexes created. Moreover, it was suggested we could reduce the number of indexes if we knew for a fact that some portion of the index is never used in searching. For example, If we have a URL like "http://www.someurl/path1/path2/path3/path4" in our data, the indexing algorithm automatically stores each path in a separate index and combinations of the paths as well. So we would end up storing path1, path2, path1/path2, etc, in indexes. Yet if we know our logs are well defined then folks won't be searching for these types of things, rather they will search for URL=path1... hence we could, if splunk let's us, significantly reduce the size of the index. Is this possible? If so is their any documentation on the same?
... View more
11-14-2012
10:53 AM
We have 10+ splunk indexer cluster. However, the disk capacity on these machines is not large enough to hold all our data for longer than 90 days. We'd prefer 180 days. As such we're looking into several options. Purchasing larger disks and adding SAN storage are two areas we are exploring. Another option is simply to add more machines. As luck would have it we have quite a few spare machines. However these machiens have very limited CPU power and the disks are not very fast. So we are concerned that adding them to the indexer tier might slow down most searches. As an alternative someone suggested adding a second indexer tier. This second tier of low performing servers would hold only cold data, meaning most searches would hit the main indexer tier, but searhces that hit the cold data would enage this secondary tier. It was explained that splunk can handle the movement of hot/warm data from the main tier to cold storage on the secondary tier. Does anyone know if this option is real? If so is there any documentation on the same?
... View more
