About melonman

melonman · ‎04-19-2012

Around me Isilon is mainly used for media/video file storage server, and Isilon provides very good performance. I was wondering if Splunk, which mainly does small and fast I/O operations, would be also on isilon to get good performance.

melonman · ‎04-19-2012

Hi, Is there any hardware limitation for the storage to putting index on? Is Isilon (as NFS) OK to use with Splunk? Thanks in advance,

melonman · ‎04-18-2012

Thanks Ayn, great hint you gave me! Appreciated!

melonman · ‎04-18-2012

That's nice, thank Ayn!!! I just had to edit some search, also I compared 'search' and query' sourcetype=score earliest=-1m [search sourcetype=score earliest=-1h@h latest=@h | chart avg(score) as score by server | eval query="server=" + server + " AND score>=" + score | fields query] | stats avg(score) as query by server DEBUG: Subsearch evaluated to the following search expression: ( ( server=node0 AND score>=35.717718 ) OR ( server=node1 AND score>=35.464630 ) OR ( server=node2 AND score>=36.445820 ) OR ( server=node3 AND score>=35.307061 ) OR ( server=node4 AND score>=35.291473 ) ) DEBUG: [subsearch]: base lispy: [ AND sourcetype::score ] DEBUG: base lispy: [ AND sourcetype::score [ OR node0 node1 node2 node3 node4 ] ] DEBUG: search context: user="admin", app="search", bs-pathname="/opt/splunk/etc" INFO: Your timerange was substituted based on your search string INFO: [subsearch]: Your timerange was substituted based on your search string sourcetype=score earliest=-1m [search sourcetype=score earliest=-1h@h latest=@h | chart avg(score) as score by server | eval search="server=" + server + " AND score>=" + score | fields search] | stats count by server DEBUG: Subsearch evaluated to the following search expression: server=node0 AND score>=35.717718 DEBUG: [subsearch]: base lispy: [ AND sourcetype::score ] DEBUG: base lispy: [ AND node0 sourcetype::score ] DEBUG: search context: user="admin", app="search", bs-pathname="/opt/splunk/etc" INFO: Your timerange was substituted based on your search string INFO: [subsearch]: Your timerange was substituted based on your search string So I will use query in my case. Really appreciated.

melonman · ‎04-18-2012

Hi, Can I change the operator in the result of format command for subsearch? I actually want to pass the subsearch format result with "greater than (>) operator" in a particular field. Example: Calculate average score using last 1 hour events, then compare average score with each score in the last 1 minute to get the events with score more than average of the server. sourcetype=score earliest=-1m@m latest=@m [search sourcetype=score earliest=-1h@h latest=@h | chart avg(score) as score by server | format] Subsearch creates a result like this: ( ( score="36.019553" AND server="node0" ) OR ( score="34.130435" AND server="node1" ) OR ( score="36.378066" AND server="node2" ) OR ( score="36.503577" AND server="node3" ) OR ( score="35.652893" AND server="node4" ) ) However, what I really want is: ( ( score>"36.019553" AND server="node0" ) OR ( score>"34.130435" AND server="node1" ) OR ( score>"36.378066" AND server="node2" ) OR ( score>"36.503577" AND server="node3" ) OR ( score>"35.652893" AND server="node4" ) ) And pass this to parent search to retrieve events more than averaged score. How can I do this? Any comment appreciated. Thanks

melonman · ‎04-15-2012

This is usually the question my team asks, and this is not documented in Doc or WiKi as long as I know. As a Splunk user, I am curous about this type of things.

melonman · ‎04-04-2012

Hi, Splunk can monitor a file like a tail -f command. I would like to know how actually Splunk sees the file change, how often Splunk see if the file has changed, does Splunk actually open a monitored file to check if the file is updated...etc I have read the doc about crc of first/last 256bytes and crcSalt contents, and also looked at Wiki page. However, I could not find the information I want to know. I would like to know tailing mechanism so I can understand how well Splunk can handle log file change for very heavy traffic logging. I hope anyone could answer my question. Thank you in advance

melonman · ‎03-23-2012

With splunk team, I could get the following result, but I am still looking for the search to get the result based on actual time stamp like above. # splunk search 'sourcetype="sendmail" | convert dur2sec(delay) dur2sec(xdelay) | transaction queueid | search msgid=*ChangeMe from=testuser1* to=testuser2* | where delay>=0 | stats last(_time) as time1 first(_time) as time2 list(delay) as delay(sec) values(host) list(stat) by from,to,msgid | convert ctime(time1) ctime(time2)' from to msgid time1 time2 delay(sec) values(host) list(stat) ---------------------- ---------------------- ----------------------------------------- ------------------- ------------------- ---------- ------------ ---------- testuser1@mydomain.poc testuser2@mydomain.poc 012B28991FA3471F9929FEA3977B7C8A@ChangeMe 12/20/2010 16:31:00 12/20/2010 16:31:08 5 mail1 Sent 7 mail2 Sent 5 mail3 Sent testuser1@mydomain.poc testuser2@mydomain.poc 1EE3A51D02FD4EF58488BEA27F452324@ChangeMe 12/20/2010 16:39:11 12/20/2010 16:39:13 0 mail1 Sent 1 mail2 Sent 1 mail3 Sent testuser1@mydomain.poc testuser2@mydomain.poc 24FC6F57221C4FFA9BC1B5E5E0676519@ChangeMe 12/16/2010 14:06:18 12/16/2010 14:06:18 0 mail1 Sent testuser1@mydomain.poc testuser2@mydomain.poc 282997A8AE804969B7AEDEF9E8C357F6@ChangeMe 12/16/2010 17:14:08 12/16/2010 17:14:11 0 mail1 Sent 2 mail2 Sent 1 mail3 Sent testuser1@mydomain.poc testuser2@mydomain.poc 2FB15937BFF349B8BEA52BC33776C122@ChangeMe 12/20/2010 16:29:13 12/20/2010 16:29:16 0 mail1 Sent 2 mail2 Sent 2 mail3 Sent testuser1@mydomain.poc testuser2@mydomain.poc 99A437DA859C4EA4B58702961591692C@ChangeMe 12/16/2010 14:27:31 12/16/2010 14:27:33 0 mail1 Sent 1 mail2 Sent 2 mail3 Sent testuser1@mydomain.poc testuser2@mydomain.poc 9E3FCD84A0524ACFB0F7AF4FD2FA7B68@ChangeMe 12/16/2010 15:02:35 12/16/2010 15:02:37 1 mail1 Sent 1 mail2 Sent 1 mail3 Sent testuser1@mydomain.poc testuser2@mydomain.poc FA1F6FD73ED347CB8F3B5451C59750CA@ChangeMe 12/16/2010 17:10:52 12/16/2010 17:10:55 1 mail1 Sent 2 mail2 Sent 1 mail3 Sent

melonman · ‎03-22-2012

I am not sure this is right way, but Icame up with the following: Create summary index for 7 day average index=test earliest=-7d@d latest=@d | stats avg(number) as mean | addinfo | collect index=summary addinfo marker="info_search_name=\"7day_avg_num\"" Search from summary index and put the average value using eval + subsearch index=test | eval 7day_avg=([search index=summary info_search_name="7day_avg_num" | head 1 | rename mean as search]) | timechart avg(7day_avg), avg(number) Any comment appreciated.

melonman · ‎03-15-2012

Hi I am trying to create a timechart report that displays both average of a numeric value of last 7 days and real time value in 30sec realtime window. The data looks like this: 2012/03/16 15:29:44 number=2090 uname=uid19 loc=area1_section8 ipaddr=10.1.1.31 result=success 2012/03/16 15:29:44 number=4816 uname=uid44 loc=area0_section4 ipaddr=10.1.1.19 result=success 2012/03/16 15:29:44 number=7542 uname=uid5 loc=area0_section9 ipaddr=10.1.1.7 result=success 2012/03/16 15:29:44 number=269 uname=uid30 loc=area2_section5 ipaddr=10.1.1.32 result=failed I came up with the following search and set the time range to 30 sec window. index=test sourcetype=sample2 | eventstats mean(number) as mean | timechart avg(mean) max(number) However, the average, mean value changes as events get indexed because the eventstats calculates only mean in recent 30 sec window. Is there search that calculate last 7 days mean and use the mean as fixed value while realtime search is running so the mean value does not get calculated in realtime with recent 30 sec realtime value? Thanks

melonman · ‎02-19-2012

Thanks for your comment! I need to confirm SP1 is tested and supported by Splunk because we plan to upgrade to SP1 in production deployment. But your comment makes scene, I will test on SP1 on my local machine as well as I ask support for this.

melonman · ‎02-17-2012

Hi, I am running Splunk4.1.4 on windows 2008 R2, and planning to apply SP1 to current environment. Is Splunk4.1.4 supported on Windows 2008 R2 + SP1, specifically SP1. I looked at doc, but there was no mention about win 2008 R2 Service Pack 1 specific information. Anyone have information? Thanks,

melonman · ‎02-16-2012

Yes, Iguinn's comment is correct. There is a Splunk doc about RDM http://docs.splunk.com/index.php?title=Community:SplunkOnVirtualMachines -- Copied from SplunkOnVirtualMachines Raw Device Mapping (RDM) is a technique by which a raw Logical Unit Number (LUN), local or remote, can be aliased to a VMDK file on a VMFS partition. The net effect is direct access to the LUN being aliased. Think of this as literally creating a symlink on a VMFS filesystem that points to raw storage. RDM can deliver sequential read and write benefits that include slightly greater IOps, lower overhead, and also benefits when working with block sizes smaller than 32kb. For indexing volumes < 25 GB per day, indexing to VMDK should function well For indexing volumes > 25 GB per day, RDM should be used.

melonman · ‎02-16-2012

Thans, dwaddle!

melonman · ‎01-31-2012

Hi I have configured the monitor path of inputs.conf. /nfsmount/log/log.d*_vd*hoge*/*/*/aaa_*_bb*-ccc*.log My question is how many wildcard characters I can put in the path. Is there any limitation of use of wildcard in monitor path? I have checked the following articles, but still can not find the answer... http://splunk-base.splunk.com/answers/13613/use-of-wild-card-character-in-monitor-path http://www.splunk.com/base/Documentation/latest/Data/Specifyinputpathswithwildcards Any thought? Thank you in advance!

melonman · ‎01-31-2012

Thanks, Brian!

melonman · ‎01-23-2012

Hi When I update props.conf and/or other .conf files, I usually issue 'extract reload=t' to reload configurations without restarting Splunk. Seems like this command only reloads search time extraction confguration. Intex time configuration and other input or output configuration is not reloaded by this command. I would like to know exactly what configuration file or properties can I reload when I do '| extract reload=t'? Thank you!

melonman · ‎01-20-2012

Thanks for helpful comment!

melonman · ‎01-20-2012

Thank you for your help! Now I can get what I wanted. I added "?" as you pointed out.

melonman · ‎01-20-2012

Thanks, I tried to change to multi-line mode, but still no luck. With EXTRACT-ipaddress = (?m)^IPADDRESS: (?P .+)$, "sourcetype="mytype" | head 1 | table ipaddress" still returns: 1 10.1.1.21 Result: success 2 10.1.1.34 Result: fail PID: 9360 3 10.1.1.9 Result: fail PID: 663410.1.1.21 Result: success With EXTRACT-ipaddress = (?m)^IPADDRESS: (?P .+)[\r\n], "sourcetype="mytype" | head 1 | table ipaddress" still returns: 1 10.1.1.21 2 10.1.1.34 Result: fail 3 10.1.1.9 Result: fail While I am reading the regex website, I would like to know how to get this right.

melonman · ‎01-20-2012

How do I configure regex to get only test after each line's : in the following log? I have a log file containing events like this: PID: 3047 CurrentTime: 2012/01/20 16:23:55 Username: username45 Floor: floor7 IPADDRESS: 10.1.1.4 Result: success CurrentTime: 2012/01/20 16:23:54 Username: username51 Floor: floor3 IPADDRESS: 10.1.1.32 Result: fail PID: 8020 CurrentTime: 2012/01/20 16:23:53 Username: username67 Floor: floor8 IPADDRESS: 10.1.1.24 Result: success Additional: Some more information and props.conf includes the following configuraion. [mytype] SHOULD_LINEMERGE = true BREAK_ONLY_BEFORE = ^PID: EXTRACT-result = ^Result: (?P<result>.+)$ EXTRACT-ipaddress = ^IPADDRESS: (?P<ipaddress>.+)$ EXTRACT-floor = ^Floor: (?P<floor>.+)$ In order to get value sucha as floor8 for floor and 10.1.1.24 for ipaddress after :, I ran the search for each field, but still getting unwanted information too. sourcetype="mytype" | table result result --- ---------------------------------------- 1 success 2 fail PID: 9360 3 fail PID: 6634 4 fail PID: 3908 5 fail PID: 1183 6 success PID: 8456 7 success PID: 5730 8 fail PID: 3004 9 fail PID: 278 10 fail PID: 7551 sourcetype="mytype" | table result ipaddress --- ---------------------------------------- 1 10.1.1.21 Result: success 2 10.1.1.34 Result: fail PID: 9360 3 10.1.1.9 Result: fail PID: 6634 4 10.1.1.21 Result: fail PID: 3908 5 10.1.1.33 Result: fail PID: 1183 6 10.1.1.8 Result: success PID: 8456 7 10.1.1.20 Result: success PID: 5730 8 10.1.1.32 Result: fail PID: 3004 9 10.1.1.8 Result: fail PID: 278 10 10.1.1.20 Result: fail PID: 7551 sourcetype="mytype" | table result floor --- ---------------------------------------- 1 floor7 IPADDRESS: 10.1.1.21 Result: success 2 floor1 IPADDRESS: 10.1.1.34 Result: fail PID: 9360 3 floor5 IPADDRESS: 10.1.1.9 Result: fail PID: 6634 4 floor9 IPADDRESS: 10.1.1.21 Result: fail PID: 3908 5 floor3 IPADDRESS: 10.1.1.33 Result: fail PID: 1183 6 floor8 IPADDRESS: 10.1.1.8 Result: success PID: 8456 7 floor2 IPADDRESS: 10.1.1.20 Result: success PID: 5730 8 floor6 IPADDRESS: 10.1.1.32 Result: fail PID: 3004 9 floor0 IPADDRESS: 10.1.1.8 Result: fail PID: 278 10 floor4 IPADDRESS: 10.1.1.20 Result: fail PID: 7551 How do I configure regex in props.conf to get only test after each line's : ? Thank you in advance

melonman · ‎01-19-2012

Thanks for answers and tips!

melonman · ‎01-18-2012

Hi I was trying to configure email alert to deliver scheduled saved search result. The SMTP server I was trying to use is gmail SMTP. The configuration in Email alert settings -> Mail Server Settings is: Mail Host -> smtp.gmail.com:465 # Enable SSL checked Username -> @gmai.com Password -> ... Is there any misconfiguration or limitation of SMTP server? Does anyone know how to use gmail SMTP with splunk email alering? Thank you!

melonman · ‎01-06-2012

That's great, thanks!

melonman · ‎01-05-2012

Hi, I have a logfile containing data that looks like the below: Nov 21 13:59:41 hostname1 data1 data2 data3 Nov 21 13:59:42 hostname1 data1 data2 data3 Nov 21 13:59:43 hostname1 data1 data2 data3 I would like to extract hostname, and each data in a separate line. How can I configure the regex to get such fields? Thanks in advance,

Posts	438
Solutions	92
Karma Given	100
Karma Received	189
Member Since	‎06-07-2010

Online Status	Offline
Date Last Visited	‎06-05-2020 02:02 AM

MLTK with autoML features?

dbxlookup with where function()

Fixed Issues listed for 6.6.4 doc in HTML is diffe...

Splunkに関する日本語の技術情報について

How to export/import/share ML models between Splun...

TensorFlow Integration

How to set initial value of text input in a form u...

Can I connect to Impala (SQL engine on Hadoop) fro...

Can I configure Splunk to export a PDF as displaye...

How to track memory/cpu usage per search execution...

Re: Splunk index on isilon?

Splunk index on isilon?

Re: Can I change the operator in the result of for...

Re: Can I change the operator in the result of for...

Can I change the operator in the result of format ...

Re: Tailing Mechanism

Tailing Mechanism

Re: MTA Relay transaction tracking search

Re: How do I show a numeric average of last X days...

How do I show a numeric average of last X days wit...

Re: Is Splunk 4.1.4 supported on Windows 2008 R2 +...

Is Splunk 4.1.4 supported on Windows 2008 R2 + SP1...

Re: Using RDM vs VMFS for Virtual Machines

Re: How many wild cards (*) can I put in monitorin...

How many wild cards (*) can I put in monitoring pa...

Re: Exactly what configuration file or properties ...

Exactly what configuration file or properties can ...

Re: Regex for multiline events

Re: Regex for multiline events

Re: Regex for multiline events

Regex for multiline events

Re: How to configure email alert using gmail smtp?

How to configure email alert using gmail smtp?

Re: How do I extract fields in line separated data...

How do I extract fields in line separated data wit...

Join the Conversation