Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Are you a member of the Splunk Community?
Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
- About rhaarmann
rhaarmann
Engager
Member since:
08-05-2015
06-05-2020
Community Statistics
| Posts | 6 |
| Solutions | 0 |
| Karma Given | 1 |
| Karma Received | 0 |
| Member Since | 08-05-2015 |
Activity Feed
- Karma Re: How to extract fields from a log that is comma delimited? for woodcock. 06-05-2020 12:47 AM
- Posted Re: Is it possible to have a script run on a Heavy Forwarder to process and convert logs to CSV and forward results to the indexer? on Getting Data In. 02-22-2016 02:58 PM
- Posted Re: Is it possible to have a script run on a Heavy Forwarder to process and convert logs to CSV and forward results to the indexer? on Getting Data In. 02-22-2016 07:28 AM
- Posted Re: Is it possible to have a script run on a Heavy Forwarder to process and convert logs to CSV and forward results to the indexer? on Getting Data In. 02-22-2016 04:52 AM
- Posted Is it possible to have a script run on a Heavy Forwarder to process and convert logs to CSV and forward results to the indexer? on Getting Data In. 02-19-2016 11:33 AM
- Tagged Is it possible to have a script run on a Heavy Forwarder to process and convert logs to CSV and forward results to the indexer? on Getting Data In. 02-19-2016 11:33 AM
- Tagged Is it possible to have a script run on a Heavy Forwarder to process and convert logs to CSV and forward results to the indexer? on Getting Data In. 02-19-2016 11:33 AM
- Tagged Is it possible to have a script run on a Heavy Forwarder to process and convert logs to CSV and forward results to the indexer? on Getting Data In. 02-19-2016 11:33 AM
- Tagged Is it possible to have a script run on a Heavy Forwarder to process and convert logs to CSV and forward results to the indexer? on Getting Data In. 02-19-2016 11:33 AM
- Tagged Is it possible to have a script run on a Heavy Forwarder to process and convert logs to CSV and forward results to the indexer? on Getting Data In. 02-19-2016 11:33 AM
- Posted Re: How to extract fields from a log that is comma delimited? on Splunk Search. 02-15-2016 03:11 PM
- Posted How to extract fields from a log that is comma delimited? on Splunk Search. 08-05-2015 01:59 PM
- Tagged How to extract fields from a log that is comma delimited? on Splunk Search. 08-05-2015 01:59 PM
- Tagged How to extract fields from a log that is comma delimited? on Splunk Search. 08-05-2015 01:59 PM
- Tagged How to extract fields from a log that is comma delimited? on Splunk Search. 08-05-2015 01:59 PM
- Tagged How to extract fields from a log that is comma delimited? on Splunk Search. 08-05-2015 01:59 PM
Topics I've Started
02-22-2016
02:58 PM
As stated before there is already something doing clean up of the script on the production server, but I wish to move it off the production server to reduce processing on the production server as clients are connected to it. Also, according to documentation Heavy forwarders do have the ability to index. I can introduce another process to get around this, but I want to see if the current technology (Splunk) that we have the server is capable of forwarding the data to another server and execute the post-process after it is received by the Intermediate server. Also, the file unable to be tailed, since it is in a proprietary compressed file from the application, which is why we have a special script from that application that can uncompress the file. I would like to setup a Intermediate forwarder that captured these files and preprocess and forward the results to the indexers. If this is not possible then that's fine, just need to know.
My only other method to solve this is to share SAN space for all the servers and have the logs dumped on the SAN and have the Intermediate server just watch for new logs and do scripted inputs.
... View more
02-22-2016
07:28 AM
I updated the Tags to properly reflect the type of Forwarder I want to achieve.
... View more
02-22-2016
04:52 AM
Yes, that is what I want to accomplish but not sure how to achieve it with a script.
... View more
02-19-2016
11:33 AM
Looking to set up a Heavy Forwarder as a data processing server. We get data logs in a specific format dropped on our production machines, but it needs to be opened and converted to CSV by a special script. Since the data originally drops on the production servers, there is also a process that follows it on the same server and converts it to CSV, which is then picked up by the Universal Forwarder and sent to the Indexer.
I would like to move the conversion process from the production server and save potential performance impact and push these to a Heavy Forwarder, which would then execute the script and send the results forward to the Indexer. Is it possible to do this with Splunk Forwarders, or do I need to look into an automated copy process from the production servers to the processing server?
... View more
02-15-2016
03:11 PM
This worked for the case of when the raw data is already indexed, but I would like to setup a transform that converts it to a better format then index the transformed data. I am currently testing out the solution you provided with searches and summary indexes and it is very expensive on the search heads, especially since the amount of the log data is around 25GB/day. I tried doing a transform line that contained SEDCMD, but it would not work, not sure why yet.
... View more
08-05-2015
01:59 PM
Ok, complex extraction. I have a log that is comma delimited, but they have key,value,key,value,key,value, etc. It's key-value structured like JSON, but in CSV format. I have figured out how to work with this in Python, but not familiar with writing splunk apps and scripts that will do a stream on a file.
I found this Splunk Answers post (http://answers.splunk.com/answers/204994/complex-kv-extraction.html ), but it doesn't completely answer my problem as I will have to hardcode each field, but all my fields are unknown and can change as developers make changes.
Log Example:
Key.Value,Timestamp,$timestamp,ms,$ts_ms,key1,value1,key2,value2,key3,value3
Key.Value,Timestamp,$timestamp,ms,$ts_ms,key1,value1,key4,value4
Key.Value,Timestamp,$timestamp,ms,$ts_ms,key2,value2,key3,value3,key5,value5,key6,value6,key4,value4
... View more
