Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Join the Conversation
Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
- Find Answers
- :
- About lukejadamec
lukejadamec
Super Champion
Member since:
03-26-2013
06-07-2021
Community Statistics
| Posts | 1291 |
| Solutions | 120 |
| Karma Given | 420 |
| Karma Received | 471 |
| Member Since | 03-26-2013 |
Activity Feed
- Got Karma for Re: Windows Events Message field. 02-11-2025 09:01 AM
- Got Karma for Re: Lots of old bundle files on Deployment Server - Safe to delete?. 12-10-2024 06:46 AM
- Got Karma for Re: Keyboard Shortcut to Format Search. 07-23-2024 11:39 AM
- Got Karma for Re: Index Strategy - Single index with multiple sourcetypes vs Multiple indexes with dedicated sourcetype. 01-05-2022 11:52 AM
- Got Karma for Re: Referencing Multiple hosts in Props.conf. 07-27-2021 08:34 AM
- Got Karma for Re: Need to return a field in a search even if it doesn't exist. 03-24-2021 10:28 AM
- Got Karma for Re: Rawdata may be corrupt. 02-27-2021 07:05 AM
- Karma Re: Why am I unable to index contents of a text file being monitored by universal forwarder? for lguinn2. 06-05-2020 12:48 AM
- Karma Re: What volume(s) is indexerWeightByDiskCapacity based on? for esix_splunk. 06-05-2020 12:48 AM
- Karma Re: How do I extract two different variations of a timestamp from the same sourcetype? for sowings. 06-05-2020 12:48 AM
- Karma Re: Search formatting in Splunk 6.5.0 for easier readability for lquinn. 06-05-2020 12:48 AM
- Karma Re: Is it safe to delete .bundle files ? for ddrillic. 06-05-2020 12:48 AM
- Karma Re: How to edit my regular expression to extract a string between percentages and other characters? for govindsinghrawa. 06-05-2020 12:48 AM
- Karma Re: How to convert a string value in the format HH:mm:ss to usable seconds for a graph? for sundareshr. 06-05-2020 12:48 AM
- Karma Re: Which instance is installed on a server? for gcusello. 06-05-2020 12:48 AM
- Karma Re: Creating a timeline showing when someone log out and login? for DEAD_BEEF. 06-05-2020 12:48 AM
- Karma Re: CSV Field Extraction with spaces in field name for lguinn2. 06-05-2020 12:48 AM
- Karma Re: Hi i need to do splunk up gradation. My splunk version is 6.3.1 i need to upgrade to 6.5. what procedure i need to follow. for inventsekar. 06-05-2020 12:48 AM
- Karma Re: Why is one of my blacklists on inputs.conf not working to filter events from Windows Event Logs? for gokadroid. 06-05-2020 12:48 AM
- Karma Re: How to control splunk logs splunkd_stderr.log & splunkd-utility.log filling up disk space for ddrillic. 06-05-2020 12:48 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
10-21-2016
04:47 PM
@googlelogzilla it looks like your math is wrong. 25K events/second works out to 207 GB/day at 1 byte/event. We all know events are often larger than 1 byte, and the question referenced 500GB of indexed/searchable events/day.
I respect anyone that has worked anywhere and has succeeded with anything and am also willing to learn from your experience.
I'm expecting that this environment will accept input from X number of systems and will be tasked by Y number of users with search requests for A number of reports, B number of ad hock searches, C number of summary indexed searches/reports, and D number of alerts (based on real time searches).
In my real world you need superior processing power and I/O for both indexers and search heads to achieve this.
As a side note, the number of search threads is dependent on the number of cores, so be sure to balance the search load with core speed.
... View more
10-21-2016
12:03 PM
You don't need a new case for this. Just contact your Splunk rep. The turn around time for a license reset even after hours is typically pretty quick. If you're going through a third party, then by pass them as quickly as possible - they are only useful for sales.
Regardless, Do Not turn off your indexers because you may end up losing data. The indexes will continue to index data - it is the search capability that is stopped when you have license violations.
... View more
10-21-2016
07:47 AM
1 Karma
Here are some real world comments:
1) Manufacturers make high core slow speed processors cheap, and they are slow. Reduce the cores and get 3.5GHz processors with high buss speeds and match the buss speeds with the ram. You'll end up with much better performance even with fewer cores and less RAM - every CPU action will be 30% faster.
2) The separation of Hot and Warm buckets based on SSD or Not might not make sense. A Hot bucket is an active indexing bucket that is searchable that is rolled to Warm once a day or on restart, and a Warm bucket is non-indexing recent searchable bucket that was rolled from Hot.
a. Hot bucket needs fast Read and Write, so SSD makes sense on the surface.
b. Warm bucket needs fast Read speed, so HDD makes sense on the surface.
Real World:
a. With 5TB of SSD (~20 days in your environment) it makes more sense to set Hot and Warm based on primary indexes, i.e. the indexes that will be the targets of the most common searches/alerts/reports in your environment, and use the HDDs for Cold buckets for those indexes.
b. Use the HDDs for Hot, Warm, and Cold buckets for nice to have data indexes.
3) RAID 6 vs RAID 10.
a. RAID 6 will give you more space but you take a big hit on Write performance and lower Read performance. Write = ((Disks = 2) * Speed)/6 and Right = ((Disks +2) * Speed)
b. RAID 10 is as close to SSD as you can get with HDDs, but it takes more disks. Write = Disks * Speed and Read = 2*Disks*Speed
4) Network speeds need to consider the Whole network. You can only get as much at a server as the network infrastructure can send you, and you want to get the data into the indexer as fast as possible. You should look at two things, 1) NIC performance specs and 2) network infrastructure throughput potential.
... View more
10-21-2016
06:14 AM
It would be helpful to post some example events also.
... View more
10-20-2016
04:12 PM
You might be able to use the SEDCMD. Can you post an example event or event number?
... View more
10-20-2016
11:07 AM
No. Once the data is passed the parsing phase it cannot go back. Even worse, you could end up with a situation where the events from a search show the SEDCMD data, but the interesting fields and _raw show the original data.
... View more
10-20-2016
07:18 AM
You can see the logins from any device that logs the login data with usernames provided those logs are indexed by Splunk.
If you need help building a search, then you should post some sample events that contain the login data from the various devices.
... View more
10-19-2016
10:39 AM
Without knowing the error it is hard to say.
If you compare the service properties between the local and remote install, is the Log On user the same?
Does it work after the remote system is rebooted?
... View more
10-19-2016
09:16 AM
I did not have luck using rex with orderDetails, but _raw worked:
... | rex field=_raw "\"Account\:(?<Account>[^,]+).*Name\:(?<AccountName>[^,]+).*OrderIpAddress\:(?<OrderIpAddress>[^\"]+)" | Table Account,AccountName,OrderIpAddress,orderDetails
... View more
10-19-2016
06:30 AM
I agree. What is the point of decreasing the size of the log files if they are all indexed with a max size of 1/2 TB anyway?
... View more
10-18-2016
11:57 AM
Rockondog.
... View more
10-18-2016
11:11 AM
I did not know that Splunk could index data without an inputs.conf stanza for that data. I guess I'll have to study more. What is the network component manufacturer/model?
... View more
10-18-2016
10:47 AM
Does Account Name appear twice in the Message of the events you're trying to block?
Since both of these are Message text regexes, have you tried removing #4 to test #5?
... View more
10-18-2016
09:53 AM
You did try putting HealthMailbox. in parens right? (HealthMailbox.)
... View more
10-18-2016
09:36 AM
Regardless of whether you use the GUI or not, there is an inputs.conf file for these inputs (the GUI creates it).
If they are coming from different IPs or the same IP and different ports, then the index= parameter can be added to those inputs.conf stanzas.
If all sources are coming in on the same IP and port, then doing this in props/transforms is the way to go.
... View more
10-18-2016
09:16 AM
Destination indexes are typically specified in the inputs.conf
Can you post your inputs.conf entries for these sources?
... View more
10-17-2016
10:57 AM
It sounds like you should be extracting these fields at search time and using dedup, but it is difficult to be sure without seeing your search. Can you post your search?
... View more
10-17-2016
09:43 AM
Not I.
If Splunk does not support installation of the forwarder on Windows 7, then I'm not sure it is wise to find a way to force it.
On a side note, since security updates will be available for Windows 7 until about 2020, I wonder why Splunk stopped supporting it. Perhaps it is because Windows 7 tech support ended in 2015.
The workaround is, as @richgalloway noted, is to use 6.4.1 or older forwareders that do install.
... View more
10-17-2016
03:38 AM
What are you trying to transform?
... View more
10-16-2016
04:35 AM
Can you list the values you are trying to extract for 'steps' from the event you posted?
... View more
10-14-2016
12:55 PM
1 Karma
Yes you can, but you should be very familiar with how the deployment server works first.
I once used the deployment server to reconfigure some Splunk apps and other system changes outside of Splunk, but the content pushed to the servers was designed to self-destruct so that it would not get re-deployed and not continually trigger once deployed.
So, Yea you can do it, just make sure you know what you're doing.
One suggestion, you might want to have the script copy the installer to a temp directory before the script executes the installer because your system may prohibit installers from running in the program files directory.
... View more
10-14-2016
09:31 AM
If you use rex to extract id from _raw, and then run the two searches from the results of rex what do you get? If you don't know how to use rex, then post an example event.
Without rex, after you run these searches, what id count to you see in Interesting Fields?
Have you identified the 4 combinations of id and _time that are getting missed - is it a difference in id or _time?
... View more
10-14-2016
06:57 AM
If you are using the Splunk UI to configure your inputs, then the location of the config files is based on the setting you select while creating the input. In particular, the setting for App tells Splunk which App to use when storing configs. The Apps are located in Splunk_Home\etc\apps. In the App that was selected when creating the input you will see a "local" folder which will contain the config files.
If you are using apps downloaded from Splunk, then the data used those Apps are typically configured inside that App, which can be found typically by name in Splunk_Home\etc\apps.
If you don't remember which App was selected when the input was created than you can use the Splunk UI to find it. Go to Settings, Data Inputs, Files and Directories for a list of file inputs. This list will also tell you which index is used for that input. It is possible that you are using inputs other than "File and Directories", so review the list of possible local inputs. There is a column for the number of inputs to give you a clue as to what type of inputs you have for those input types.
If you have custom field extractions or other UI generated transforms, then you may file those settings in the user context in Splunk_Home\etc\users under the user that created them.
... View more
10-14-2016
05:07 AM
Yes. But I don't think those are the times used for _time. In the example logs there is "date" at the beginning which I assumed was the timestamp for _time - not sure why examples of the timestamp were left out.
If those are the timestamps, then it should be fairly easy to construct a time format so that they can be read.
Regardless, I did not see your answer until I posted mine, and I came up with my answer after giving up on trying to figure out how to do it they way you illustrated. Your answer is probably the way it should be done.
... View more
10-13-2016
02:21 PM
This may seem too easy, but using your logs it appears to work.
Use SEDCMD in props.conf to replace the spaces with underscores.
inputs.conf
[monitor://c:\temp\sed-csv.log]
disable=false
index=sedcsv
sourcetype=sedcsv
props.conf
[sedcsv]
SEDCMD-replacespace = s/ /_/g
This replaces all spaces in your events with underscores. Without any other specialized config Splunk appears to be picking out all of the fields with the correct = value.
Since you did not specify a date/time format I just put one in in the beginning of each event.
The only field that is not automatically extracted is the server field, but that is because it is not a field=value combination. However, pulling that out at search time should not be hard.
If I understand the pipeline correctly, this config will need to be added at the forwarder, and will only affect newly ingested logs after a restart.
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
06-07-2021
10:52 AM
|
Karma from
Karma given to
