About adamw

adamw · ‎09-09-2012

Marc, Removed from queue typically means that Splunk has reached an EOF and is finished reading the file. Thanks, --adam

adamw · ‎09-09-2012

Instance 1: rsync -av $SPLUNK_HOME/etc/passwd /sharedfolder/splunk/passwd Instance 2/3/...: rsync -av /sharedfolder/splunk/passwd $SPLUNK_HOME/etc/passwd Put these both in cron every 5 minutes (or so) and you should be good to go.

adamw · ‎08-17-2011

Can you set maxTotalDataSizeMB to 0, or optionally set it to an incredibly high number (90000000, or 90TB) in order to hopefully make the cold-to-frozen process only happen based on frozenTimePeriodInSecs. Basically time based retention instead of the maxTotalDataSizeMB? Thanks, --adam

adamw · ‎08-17-2011

We have a considerable amount of searches on the jobs page that have an expires time of "Saved". Many of these are from a few months ago, and appear to be searches done by end users, and they may have saved the search, or shared a link to the results with someone. Is there a way to auto-expire these after a certain amount of time? Thanks, --adam

adamw · ‎08-16-2011

What about putting an outputs.conf on the indexers to forward data out VIA syslog. The universal forwarders don't do any filtering or anything, and might not even be capable of doing syslog output.

adamw · ‎06-16-2011

For various reasons (performance, not picking up files, etc) we are looking to migrate our syslog receiver from a universal forwarder to a lightweight forwarder. I have installed the lightweight forwarder package and migrated configurations over, but I want to make sure I grab the checkpoint (place-in-the-file-marker) files, so that no repeat data is indexed. Where can I find these files? As well, are there any additional considerations before I make the switch, anything else that should be copied over or potential pitfalls? Thanks, --adam

adamw · ‎04-22-2011

I agree with piebob, we've found in our deployment that adding indexers and doing load balancing and distributed search significantly improves index and search performance. Basically search heads are CPU intensive, while indexers are memory intensive. So plan accordingly, but scaling indexers will give you your most bang for your buck, imho.

adamw · ‎03-29-2011

Hey guys, So we just installed 4.2 on our searchhead, and loving it so far! We ran into a bit of a snag though. Our searchhead normally sits behind a server that does SSO and passes the logged on user back with the REMOTE_USER variable like specified in the docs. In 4.1, there was still a "Logout" link in the top right corner of the Splunk page. This appears to have been removed in 4.2 when using SSO. Even when not going through the SSO reverse proxy, the "Logout" link is still not there (although I can then login, which is a plus). I know it sounds strange to need to login with a different account than you are SSO'd in with, but we actually do it quite a bit for various reasons. Is there a parameter in web or server.conf that could re-enable the logout link, even though SSO is enabled? Thanks, --adam

adamw · ‎03-24-2011

I noticed while comparing the default configs for WMI and Perfmon that there's a LocalProcesses query in WMI that lacks a corresponding entry in Perfmon. It seems unique in this regard and I'm wondering if there's a reason it's not there. Is there something I'm missing?

adamw · ‎01-26-2011

This still seems to add the port at the end of it.

adamw · ‎12-13-2010

We use the host_segment option in inputs.conf, in order to pull the hostname straight off of the path on the file system, as written by syslog-ng. in inputs.conf: [monitor:///opt/syslog] disabled = true host_segment = 3 index = default and syslog-ng configuration: destination hosts { file("/opt/syslog/$HOST/$YEAR/$MONTH/$DAY/$FACILITY$YEAR$MONTH$DAY")

adamw · ‎12-09-2010

Krusty, What about configuring syslog-ng to write files out with hostname, then have splunk index that with regex on path for hostnames. This way you also don't lose syslog entries when you restart splunk.

adamw · ‎12-08-2010

It's going to end in less tears if you do what balbano suggested, and have splunk index the files that syslog-ng writes out. This was strongly recommended by all of the Splunk folks we've ever talked to.

adamw · ‎12-08-2010

what version of Splunk are you running, I know they made some fixes to PDF in 4.1.6.

adamw · ‎09-29-2010

We seem to be having an issue with the postfix_syslog sourcetype (that came as a default sourcetype in Splunk) and its date extractions. I posted this at 8:20am on 9/29, and did a search of events that take place between 15:00 and 23:59 on 9/29 and come back with the following results. As you can see, the date_hour is set up as 18 on one of these events, which translates to 6pm, but the original event actually took place at 5am. I am not overriding any of the default postfix_syslog stuff, and these events are getting sourcetyped properly, as shown below. [postfix_syslog] TIME_FORMAT = %b %d %H:%M:%S According to the docs, %H is the 24 hour time, even though Splunk seems to believe it is not. Any help is appreciated. Thanks, --adam (EDIT) This is Splunk 4.1.4, data is coming in with syslog-ng.

adamw · ‎09-28-2010

Thats what I assumed as well, but it didn't seem to work after a splunk restart. I had to overwrite the default folder's directory for my changes to take effect.

adamw · ‎09-28-2010

So we have the default download of the Unix app, and we moved all of our unix stuff into the unix_os index, instead of the os index as suggested by the app. I've already modified the saved searches in the app to reflect this change, but a lot of the dashboard views are still not working. I noticed they are defined in the default/data/ui/ folder. Can I copy this data folder to the local/data/ folder and have it override the default ones. I know this works for configs, but what about dashboard views and other UI changes? Thanks, --adam

adamw · ‎09-16-2010

I'm sensing an insurmountable amount of pain from using summary indexing to accomplish this, plus I was wondering if it was something simple and I just hadn't realized it yet. Thanks!

adamw · ‎09-14-2010

I know it has been mentioned various times that transactions are the way to go when searching through postfix transaction logs with separate queue ID's, and joining all queue ID's together, but this can take an insane amount of time if I wanted to get a record of all email to and from a particular person for the past year (yes, I have been asked for this before). Sep 13 23:55:41 mailhost1 postfix/qmgr[16069]: [ID 197553 mail.info] 44B09122E5: removed Sep 13 23:55:42 mailhost1 postfix/smtpd[15824]: [ID 197553 mail.info] connect from example.com[1.1.1.1] Sep 13 23:55:42 mailhost1 postfix/smtpd[15824]: [ID 197553 mail.info] 3042911BC9: client=example.com[1.1.1.1] Sep 13 23:55:42 mailhost1 postfix/cleanup[15826]: [ID 197553 mail.info] 3042911BC9: message-id=<01010101010101@example.com> Sep 13 23:55:42 mailhost1 postfix/qmgr[16069]: [ID 197553 mail.info] 3042911BC9: from=, size=8033, nrcpt=1 (queue active) Sep 13 23:55:42 mailhost1 postfix/smtpd[15824]: [ID 197553 mail.info] disconnect from example.com[1.1.1.1] Sep 13 23:55:42 mailhost1 postfix/cleanup[15835]: [ID 197553 mail.info] 4A870122E5: message-id=<01010101010101@example.com> Sep 13 23:55:42 mailhost1 postfix/local[15812]: [ID 197553 mail.info] 3042911BC9: to=, relay=local, delay=0.16, delays=0.1/0/0/0.05, dsn=2.0.0, status=sent (forwarded as 4A870122E5) Sep 13 23:55:42 mailhost1 postfix/qmgr[16069]: [ID 197553 mail.info] 4A870122E5: from=, size=8165, nrcpt=1 (queue active) Sep 13 23:55:42 mailhost1 postfix/qmgr[16069]: [ID 197553 mail.info] 3042911BC9: removed Sep 13 23:55:42 mailhost1 postfix/smtp[15718]: [ID 197553 mail.info] 4A870122E5: to=, relay=mymailserver.mydomain.com[192.168.1.2]:25, delay=0.08, delays=0.07/0/0/0, dsn=2.0.0, status=sent (250 Message accepted for delivery) That's what the log looks like, I basically want a transaction based on the SMTP Transaction ID, except I would like it done at index time. I've already played with the breakafter stuff in props.conf, but if a message sits in the queue for a long period of time, that won't work. Any ideas, I know this is a pretty complex operation? If there is no good way to do it, I'm okay with that, just thought I'd ask. Thanks, --adam

adamw · ‎09-02-2010

So I have an application that auto-rotates its config files every time it is changed, and uses the following structure for filenames across changes. config.2010-09-01T12:00:00 config.2010-09-02T13:53:00 I'd like to use fschange to diff these config files. The problem is that Splunk expects a path (or filename) that is to be monitored and to diff changes on one file. Is there any way to have Splunk basically perform the tasks of the fsmonitor input type, except across these multiple files, treating them as one file? Thanks, --adam

adamw · ‎08-19-2010

What type of input is this. We have noticed a slowdown in monitors where there are hundreds (even thousands) of files being monitored. Solution was to remove some of the files being monitored, because they were old rotated log files, and once splunk has them,we don't really care about the source file anymore.

adamw · ‎08-19-2010

I actually filed an enhancement request for this exact thing at the conference, but the more you file, the more attention it gets :). Also put in your business case when you put it in, it helps them know what it is used for.

adamw · ‎08-19-2010

I heard this was some type of feature, so that a malicious person cod not make a new app with a malicious script inside it that could be deployed onto a deployment client and run to do malicious tasks on your server.

adamw · ‎08-11-2010

What about using Apache in front of Splunkweb. I would very much recommend it. There are other solutions on answers using Mod_proxy in Apache to do this.

adamw · ‎08-11-2010

There used to be a pageSize setting back in the 3.x days (still lives in some of the docs), but it doesnt exist in 4.x, any chance of this being addeed back in?

Posts	54
Solutions	12
Karma Given	10
Karma Received	37
Member Since	‎05-19-2010

Online Status	Offline
Date Last Visited	‎06-05-2020 02:02 AM

maxTotalDataSizeMB max value or 0

Expire saved jobs

Convert universal forwarder TO lightweight forward...

4.2 SSO removes "Logout" button

WMI vs Perfmon: No Local Processes?

postfix_syslog time extraction inaccuracies

unix app with local data folder

Postfix logs index time transaction

fschange on auto-rotated config files

Splunk audit log in syslog output

Re: Q: BatchReader - Removed from queue WHY????

Re: 2 Splunk instance on Linux - passwd sync

maxTotalDataSizeMB max value or 0

Expire saved jobs

Re: Universal Forwarder send syslog to a thrid par...

Convert universal forwarder TO lightweight forward...

Re: concurrent users and hardware scaling

4.2 SSO removes "Logout" button

WMI vs Perfmon: No Local Processes?

Re: Email Alerts behind proxy not sending proper l...

Re: Problems with syslog and splunk

Re: Problems with syslog and splunk

Re: Retain "Facility" via TCP/UDP:514 Direct Index...

Re: Schedule PDF 404 error.

postfix_syslog time extraction inaccuracies

Re: unix app with local data folder

unix app with local data folder

Re: Postfix logs index time transaction

Postfix logs index time transaction

fschange on auto-rotated config files

Re: Indexer is slowing down

Re: Audit modifications to a user account

Re: Application scripts not executable when deploy...

Re: how do I define an intermediate certificate au...

Re: What is LDAP error: Size Limit Exceeded ?

Are you a member of the Splunk Community?