Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Are you a member of the Splunk Community?
Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
- Find Answers
- :
- About theouhuios
theouhuios
Motivator
Member since:
10-05-2012
06-05-2020
Community Statistics
| Posts | 339 |
| Solutions | 15 |
| Karma Given | 19 |
| Karma Received | 61 |
| Member Since | 10-05-2012 |
Activity Feed
- Got Karma for Can Splunk 5 modules like Converttodrilldownsearch be used in 4.3?. 09-29-2022 01:31 AM
- Got Karma for How to chart values over time. 06-16-2021 12:53 PM
- Got Karma for Create new dimensions from collectd. 06-05-2020 12:50 AM
- Got Karma for How do you CIDR Match a subnet in a list of subnets?. 06-05-2020 12:50 AM
- Karma Re: How to display values in the columns for a chart? for Venkat_16. 06-05-2020 12:47 AM
- Karma Re: REST API Modular Input: Is there a way to have a REST API call retrieve only new events, not all data? for martin_mueller. 06-05-2020 12:47 AM
- Got Karma for Re: Why Splunk DB Connect is not properly importing data in Splunk after configuring an Oracle database input?. 06-05-2020 12:47 AM
- Got Karma for Re: Why custom drilldown XML in pie chart doesn't populate target field?. 06-05-2020 12:47 AM
- Got Karma for Re: Why am I unable to edit or create lookups that are local to an app using Lookup File Editor?. 06-05-2020 12:47 AM
- Got Karma for Re: How to split results which are combined in a table and export to a csv?. 06-05-2020 12:47 AM
- Got Karma for Re: How to split results which are combined in a table and export to a csv?. 06-05-2020 12:47 AM
- Got Karma for How to split results which are combined in a table and export to a csv?. 06-05-2020 12:47 AM
- Got Karma for How to split results which are combined in a table and export to a csv?. 06-05-2020 12:47 AM
- Got Karma for REST API Modular Input: Is there a way to have a REST API call retrieve only new events, not all data?. 06-05-2020 12:47 AM
- Got Karma for Re: Timestamp problem with DB Connect. 06-05-2020 12:47 AM
- Got Karma for Re: Permission change detected for Informatica data directory/files. 06-05-2020 12:47 AM
- Got Karma for Re: Why Splunk DB Connect is not properly importing data in Splunk after configuring an Oracle database input?. 06-05-2020 12:47 AM
- Got Karma for Re: Why Splunk DB Connect is not properly importing data in Splunk after configuring an Oracle database input?. 06-05-2020 12:47 AM
- Got Karma for Re: Why Splunk DB Connect is not properly importing data in Splunk after configuring an Oracle database input?. 06-05-2020 12:47 AM
- Got Karma for Re: Filter the source-file of logs with "rex" command to produce fields. 06-05-2020 12:47 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 1 | |||
| 0 | |||
| 1 | |||
| 1 | |||
| 1 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
03-25-2015
09:57 AM
Thanks for the answer. We don't have this issue now. This ticket is old.. about a year old.
... View more
03-25-2015
09:56 AM
yup. Thats how it is now.
... View more
03-23-2015
06:01 AM
You should be able to install the agent on the EC2 instance. Everything else is same as if you would it On Prem.. Open up firewall ports to DS and Indexers.
... View more
03-23-2015
05:59 AM
@martin_mueller
I added the filename as a global variable
FETCH_INTERVAL = 600 # in seconds
STEP = 15
CONSOLIDATION_FUNCTION = "AVERAGE"
TIME_FORMAT = "%Y-%m-%d %H:%M:%S"
filename = "/mnt/var/log/ganglia/rrds/__SummaryInfo__/dfs.namenode.BlockReportNumOps.rrd"
and commented the arg part. Not sure if this is right though.
def main(args):
#if (len(args) != 2):
# sys.exit(1)
#filename = args[1]
dump_rrd(filename, STEP, CONSOLIDATION_FUNCTION, None, FETCH_INTERVAL, TIME_FORMAT)
#if (__name__ == '__main__'):
# main(sys.argv)
Any suggestions?
... View more
03-21-2015
02:30 PM
Hello
I am trying to get ganglia data into splunk and I see error in _internal. The error which is I see is fetch() takes exactly 2 arguments (3 given)
[script:///mnt/splunk/etc/apps/search/bin/readrrd.sh /mnt/var/log/ganglia/rrds/__SummaryInfo__/dfs.namenode.BlockReportNumOps.rrd]
interval = 600
sourcetype = collectd
source = readrrd.sh
#disabled = false
#index=test
I changed the step Value for this in the script based on the rrdtool's output.
rrdtool info /mnt/var/log/ganglia/rrds/__SummaryInfo__/dfs.namenode.BlockReportNumOps.rrd
filename = "/mnt/var/log/ganglia/rrds/__SummaryInfo__/dfs.namenode.BlockReportNumOps.rrd"
rrd_version = "0003"
step = 15
Errors in _internal logs are:
03-21-2015 17:26:44.643 -0400 ERROR ExecProcessor - message from "/mnt/splunk/etc/apps/search/bin/readrrd.sh /mnt/var/log/ganglia/rrds/AMZN-EMR/ip-10-162-5-172.ec2.internal/dfs.datanode.ReplaceBlockOpNumOps.rrd" TypeError: fetch() takes exactly 2 arguments (3 given)
03-21-2015 17:26:44.643 -0400 ERROR ExecProcessor - message from "/mnt/splunk/etc/apps/search/bin/readrrd.sh /mnt/var/log/ganglia/rrds/AMZN-EMR/ip-10-162-5-172.ec2.internal/dfs.datanode.ReplaceBlockOpNumOps.rrd" return self.backend.fetch(*data)[returnStyle]
03-21-2015 17:26:44.643 -0400 ERROR ExecProcessor - message from "/mnt/splunk/etc/apps/search/bin/readrrd.sh /mnt/var/log/ganglia/rrds/AMZN-EMR/ip-10-162-5-172.ec2.internal/dfs.datanode.ReplaceBlockOpNumOps.rrd" File "/usr/lib/python2.6/site-packages/PyRRD-0.1.0-py2.6.egg/pyrrd/rrd.py", line 226, in fetch
03-21-2015 17:26:44.643 -0400 ERROR ExecProcessor - message from "/mnt/splunk/etc/apps/search/bin/readrrd.sh /mnt/var/log/ganglia/rrds/AMZN-EMR/ip-10-162-5-172.ec2.internal/dfs.datanode.ReplaceBlockOpNumOps.rrd" data = rrd.fetch(cf=cf, resolution=resolution, start=start, end=end)
03-21-2015 17:26:44.643 -0400 ERROR ExecProcessor - message from "/mnt/splunk/etc/apps/search/bin/readrrd.sh /mnt/var/log/ganglia/rrds/AMZN-EMR/ip-10-162-5-172.ec2.internal/dfs.datanode.ReplaceBlockOpNumOps.rrd" File "/mnt/splunk/etc/apps/search/bin/readrrd.sh", line 53, in dump_rrd
03-21-2015 17:26:44.643 -0400 ERROR ExecProcessor - message from "/mnt/splunk/etc/apps/search/bin/readrrd.sh /mnt/var/log/ganglia/rrds/AMZN-EMR/ip-10-162-5-172.ec2.internal/dfs.datanode.ReplaceBlockOpNumOps.rrd" dump_rrd(filename, STEP, CONSOLIDATION_FUNCTION, None, FETCH_INTERVAL, TIME_FORMAT)
03-21-2015 17:26:44.643 -0400 ERROR ExecProcessor - message from "/mnt/splunk/etc/apps/search/bin/readrrd.sh /mnt/var/log/ganglia/rrds/AMZN-EMR/ip-10-162-5-172.ec2.internal/dfs.datanode.ReplaceBlockOpNumOps.rrd" File "/mnt/splunk/etc/apps/search/bin/readrrd.sh", line 100, in main
03-21-2015 17:26:44.643 -0400 ERROR ExecProcessor - message from "/mnt/splunk/etc/apps/search/bin/readrrd.sh /mnt/var/log/ganglia/rrds/AMZN-EMR/ip-10-162-5-172.ec2.internal/dfs.datanode.ReplaceBlockOpNumOps.rrd" main(sys.argv)
03-21-2015 17:26:44.643 -0400 ERROR ExecProcessor - message from "/mnt/splunk/etc/apps/search/bin/readrrd.sh /mnt/var/log/ganglia/rrds/AMZN-EMR/ip-10-162-5-172.ec2.internal/dfs.datanode.ReplaceBlockOpNumOps.rrd" File "/mnt/splunk/etc/apps/search/bin/readrrd.sh", line 104, in <module>
03-21-2015 17:26:44.643 -0400 ERROR ExecProcessor - message from "/mnt/splunk/etc/apps/search/bin/readrrd.sh
Any help would be great. Thanks
... View more
03-17-2015
10:48 AM
Sorry about that. Been a while since I posted here. Looks like formatting got messed up.
... View more
03-17-2015
10:45 AM
1 Karma
http://docs.splunk.com/Documentation/Splunk/5.0.4/Deploy/Configureforwarderswithoutputs.confd#Types_of_outputs.conf_files
You should look at the Data Cloning part in the link.
... View more
03-17-2015
10:34 AM
rex field=_raw "*\s+(?P<Start_date>\S+)\s+"
should give you the time value.
rex field=_raw "(?P<ROLLS>num_rows:\s+\d+)"
This will get the ROLLS.
But you would want num_rows = 65931447 right? for that..
rex field=_raw "num_rows:\s+(?P<num_rows>\d+)\,"
Didn't test these. Should be good though.
... View more
03-17-2015
10:26 AM
You want to index them at oregon? Or do you want oregon to just act as forwarder and push those logs to sanDiego?
I do this in our environment where I route prod server logs to dev environment through forwarders in the middle which have ports and fw rules open to dev. Not sure if this is same type of configuration that you are trying to achieve.
... View more
02-26-2015
07:55 AM
Hello
Mcafee HIPS logs are written on all laptops and basically contain firewall like data. The issue is HIPS logs are written in unicode text format which Splunk sees as a binary file. Has anyone had any success in trying to index hips logs into splunk?
Thanks
... View more
01-13-2015
11:48 AM
Yup. They are applied on SH
... View more
01-13-2015
10:22 AM
They are actually being indexed on the SH itself. There is no forwarder passing data to indexer here. SH itself does this job and just sends the events to indexers. The .txt files are being downloaded from a S3 bucket onto the SH directly.
... View more
01-13-2015
09:39 AM
Added props and transforms to the first post. No, this event is happening on just one file each day. Its a random file. Even when I re index the file it still does the same. I tried it in our dev environment with the file and it give correct count. This is all proxy data so size of single event vary a lot. But is there a way that I can set it to ignore that size and just index it?
... View more
01-13-2015
08:36 AM
Hello
Splunk is dropping lot of data from a log file in our Prod environment.
It errors out as below in _internal logs.
The data is pretty straight forward with single line events. Has anyone seen this happen before?
Dropping data without forwarding since the size is greater than 67108864, event size=95536310,
Props:
[scansafe]
INDEXED_EXTRACTIONS = TSV
TRUNCATE=0
SHOULD_LINEMERGE=False
disabled=false
FIELD_DELIMITER=\t
TRANSFORMS-null= setnull
Transforms:
[setnull]
REGEX = ^#
DEST_KEY = queue
FORMAT = nullQueue
Events look like
#Fields: datatime c-ip cs(X-Forwarded-For) cs-username cs-method cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs(User-Agent) cs(Content-Type) cs-bytes sc-bytes sc-status sc(Content-Type) s-ip x-ss-category x-ss-last-rule-name x-ss-last-rule-action x-ss-block-type x-ss-block-value x-ss-external-ip x-ss-referer-host
2015-01-13 15:05:05 GMT 10.1.1.1 WinNT://NASDCORP\abcd CONNECT https 0009b601.pphosted.com 10000 / Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; ) - 0 0 0 10.1.1.1 c:busi default do_not_inspect 10.1.1.1
2015-01-13 15:05:58 GMT 10.1.1.1 10.1.1.1 CONNECT https 10.1.1.1 443 / Mozilla/4.0 (compatible) - 0 0 0 10.1.1.1 c:comp default do_not_inspect 10.1.1.1
2015-01-13 15:05:34 GMT 10.1.1.1 WinNT://NASDCORP\gghjhh GET http 421-vt.c3tag.com 80 / iN=456678&cid=421&nid=x-nid:Display<-ch-nid->Armonix-RT¶m4=300x250-female.jpg¶m1=300x250&w=1440&h=900&sT=5 Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; ) - 0 27 200 text/html 10.1.1.1 c:busi default allow 10.1.1.1 bossip.com
... View more
12-30-2014
12:27 PM
Hello
We are trying to get Cisco ScanSafe data into splunk and it looks like I need to make calls to the s3 bucket on scansafe using s3cmd. Has anyone tried this method or is there another way to get the data?
Thanks
... View more
12-23-2014
08:23 AM
Hello
I am trying to duplicate the values of status and user for all rows below so that I can use them in my search later. Is there a way to have all rows filled with the status and user automatically ?
... View more
- Tags:
- search
12-10-2014
08:33 AM
You can make it as an automatic lookup. It will work in the search once it an Automatic Lookup. If not.. you can still make it work by doing something like below in the search Criteria.
Sourcetype=ABC|lookup user.csv ID AS ID OUTPUTNEW DEPT AS DEPT|where DEPT=XYZ
Search terms in basically another search which will be the first part of the search when a user runs it. What ever that user executes will be appended to this search.
... View more
12-10-2014
08:02 AM
1 Karma
Splunk wouldn't really be an ideal tool for this from my view. But you can do this with splunk. Write a simple script which will basically do an ls -l and run the script every 10/15 mins. Once the data is in splunk you can write searches which will check its previous state and the present state and if it changes send an alert.
Now the reason why Splunk isn't the right tool to do it.
You will be ingesting pretty decent amount of data to find a change of permissions
... View more
12-10-2014
07:53 AM
Handling this would be a mess in a long run if its done on the user by user basis. To solve this what I did at my place was to integrate the data which Managers would need with the AD data. We basically pull the information on employee, Employee number, Manager, TeamName and use a lookup which will be populated once every week. So using this lookup we attached it to the userid in the data to the respective team and then assign each manager their respective teams. This solves the issue of you as an admin trying to add user Z later when a new employee comes under that team. Instead with the lookup being populated with a new user whenever they join the manager will automatically be able to see the data for that user.
... View more
12-10-2014
07:28 AM
Hello
Has anyone figured out a way to send Mcafee's HIPS firewall logs which are written to event.log file on the agent to be indexed to Splunk? It writes in binary so splunk can't just read the file. Any ideas on how to do it? It looks like the ePO doesn't get the firewall event logs back to itself so I can't get them to write it as a text file for splunk to read it. And it doesn't get them to its database too so can't use DB Connect to pull them in.
Any ideas?
... View more
- Tags:
- mcafee
12-04-2014
12:07 PM
Yup. Thanks Martin
... View more
12-04-2014
05:02 AM
Yeah, did the same. I was thinking if I could do it in the input itself, but since that didn't work used the transforms
... View more
12-03-2014
08:27 AM
When I place a datetime as a token it gets empty events which is actually right. But is it possible to not index the json data if there is no value in the items ?
... View more
11-25-2014
12:06 PM
But it only gets the date but not the time right? Each time it runs its getting everything for today instead of getting only the changes which happened.
... View more
11-25-2014
08:38 AM
TIME_PREFIX=^\<\d+\>|^
TIME_FORMAT = %Y:%m:%d-%H:%M:%S
Try this
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
06-05-2020
02:03 AM
|
Karma from
