Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- About michael_sleep
michael_sleep
Communicator
Member since:
01-22-2016
02-24-2023
Community Statistics
| Posts | 52 |
| Solutions | 3 |
| Karma Given | 0 |
| Karma Received | 11 |
| Member Since | 01-22-2016 |
Activity Feed
- Got Karma for Re: Attempt to thaw data slow and ended unexpectedly. 10-28-2023 12:36 PM
- Got Karma for Re: INGEST_EVAL replace changes the visible _raw shown in search results but does not impact license/ingestion. 02-22-2023 09:29 AM
- Posted Re: INGEST_EVAL replace changes the visible _raw shown in search results but does not impact license/ingestion on Getting Data In. 02-22-2023 09:21 AM
- Posted INGEST_EVAL replace changes the visible _raw shown in search results but does not impact license/ingestion? on Getting Data In. 02-22-2023 08:22 AM
- Posted Is it possible to remove unnecessary JSON wrapping before it's ingested to save license? on Getting Data In. 02-02-2023 07:16 AM
- Posted How to display specific timezone in table results instead of user preference timezone? on Splunk Search. 02-23-2022 11:45 AM
- Got Karma for JMX add-on failing once a day at same time, every day: java.net.SocketException: Socket is closed. 06-05-2020 12:49 AM
- Got Karma for How can I remove the error "Unable to initialize modular input "jmx" defined inside the app "SPLUNK4JMX"" received on our search heads?. 06-05-2020 12:48 AM
- Got Karma for How can I remove the error "Unable to initialize modular input "jmx" defined inside the app "SPLUNK4JMX"" received on our search heads?. 06-05-2020 12:48 AM
- Got Karma for Re: How can I remove the error "Unable to initialize modular input "jmx" defined inside the app "SPLUNK4JMX"" received on our search heads?. 06-05-2020 12:48 AM
- Got Karma for Re: How to extract MAC addresses from a log that has all values delimited by a comma, but the order of fields can change?. 06-05-2020 12:48 AM
- Got Karma for Re: How to extract MAC addresses from a log that has all values delimited by a comma, but the order of fields can change?. 06-05-2020 12:48 AM
- Got Karma for Re: Why is one transform overriding the other with my current configuration?. 06-05-2020 12:48 AM
- Got Karma for Re: TA-juniper "Bad regex value" error after upgrade from Splunk 6.2.6 to 6.3.5. 06-05-2020 12:48 AM
- Got Karma for Re: Ignoring header in the csv file. 06-05-2020 12:48 AM
- Posted Re: Will Splunk App for Auth0 ever support Splunk 7.x? on All Apps and Add-ons. 10-17-2019 10:10 AM
- Posted Re: Will Splunk App for Auth0 ever support Splunk 7.x? on All Apps and Add-ons. 09-21-2018 12:19 PM
- Posted Re: JMX add-on failing once a day at same time, every day: java.net.SocketException: Socket is closed on All Apps and Add-ons. 01-25-2018 06:24 AM
- Posted JMX add-on failing once a day at same time, every day: java.net.SocketException: Socket is closed on All Apps and Add-ons. 01-24-2018 09:04 AM
- Posted Re: Sub-search doesn't work in some apps/dashboards but works in others and as a separate search on Splunk Search. 01-22-2018 11:20 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 1 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 2 | |||
| 0 | |||
| 0 |
02-22-2023
09:21 AM
1 Karma
I may have come upon the solution on my own already... seems this may be caused by INDEXED_EXTRACTIONS = JSON being defined in another props.conf. Removing this definition seems to be fixing the issue so far. UPDATE: It turns out this did NOT fix the issue, this actually made everything else worse... for some reason we started dropping about 95% of the events coming in.
... View more
02-22-2023
08:22 AM
We have been trying to address a problem that exists between our Splunk deployment and AWS Firehose, namely that Firehose adds 250 bytes of useless JSON wrapper to all log events (which when multiplied by millions/billions events increases our storage and license costs enormously).
In order to address this we turned to a combination of INGEST_EVAL on our heavy forwarders which will: 1. Strip the JSON envelope from the event 2. Unescape all of the JSON quotes in the actual log data, making it parse-able JSON once again 3. Assign the logStream/logGroup values to host/source respectively This is somewhat working and when we look in Splunk it appears our events are showing up with all the appropriate fluff removed... so for example this is what our events used to look like (logGroup, logStream, message and timestamp are all added values from AWS Firehose):
After the processing they now look like this:
As you can see, the event is much much smaller without losing any necessary information. However, to our surprise this has not had any impact on ingestion levels. It seems to be exactly the same. We also noticed that all of these fields, even though they do not appear in the event view, are actually available and indexed in the "interesting fields" area, which seems to explain why our ingestion/storage has not decreased at all:
For reference, these are the props/transforms I'm using to accomplish this: Props.conf:
[source::http:AWS2Splunk]
TRANSFORMS-hostname = changehost
TRANSFORMS-sourceinfo = changesource
priority = 100
[aws:firehose:json]
priority = 1000
TRANSFORMS-stripfirehosewrapper = stripfirehosewrapper
Transforms.conf:
[changehost]
DEST_KEY = MetaData:Host
REGEX = \,.logStream...([^\"]+)\"\,\"timestamp
FORMAT = host::$1
[changesource]
DEST_KEY = MetaData:Source
REGEX = \,.logGroup...([^\"]+)\"\,\"logStream
FORMAT = source::$1
[stripfirehosewrapper]
INGEST_EVAL = _raw=replace(replace(replace(replace(replace(_raw,"\{\"message\"\:\"",""),"..\"logGroup\"\:\".*",""),"\\\\\"","\""),"\\\{2}","\\"),"\"stream\":\"\w+\"\,","")
Anyone have any thoughts as to what we're doing wrong? Is this possibly a conflict with doing a DEST_KEY before INGEST_EVAL? Will these two not necessarily play nice together? UPDATE: I changed from DEST_KEY to using INGEST_EVAL completely... still seems to be the same issue
[changehost]
#DEST_KEY = MetaData:Host
#REGEX = \,.logStream...([^\"]+)\"\,\"timestamp
#FORMAT = host::$1
INGEST_EVAL = host=replace(replace(_raw,".*\"\,\"logStream\"\:\"",""),"\"\,\"timestamp\".*","")
[changesource]
#DEST_KEY = MetaData:Source
#REGEX = \,.logGroup...([^\"]+)\"\,\"logStream
#FORMAT = source::$1
INGEST_EVAL = source=replace(replace(_raw,".*logGroup\"\:\"",""),"\"\,\"logStream\"\:.*","")
... View more
Labels
02-02-2023
07:16 AM
Hey there, we have a large volume (about 500-600gb) of data coming in daily but about 200gb of this is a JSON wrapper from Amazon Firehose. The data essentially looks like this: {
"message": "ACTUAL_DATA_WE_WANT",
"logGroup": "/use1/prod/eks/primary/containers",
"logStream": "fluent-bit/cross-services/settings-7dbb9dbdb4-qjz5b/settings-api/81d3685eaaeae0effab5931590784016ce75a8171ad7e3e76152e30bd732a739",
"timestamp": 1675349068034
} As you can see, ACTUAL_DATA_WE_WANT is what we need. This contains everything including timestamp and application information. The JSON wrapper is added by Firehose and makes up at least 250 bytes of every event. Is it possible to remove all of this unnecessary data so that we can save ingestion for more useful things? I have heard that the SEDCMD can do this but it is resource intensive and we ingest almost a billion events a day.
... View more
Labels
02-23-2022
11:45 AM
I spent a fair amount of time perusing Google and Splunk Answers but couldn't seem to find a solution that made sense... essentially the requirement I have is to display a timestamp in a Splunk dashboard in a specific timezone, regardless of what user preferences people have configured. The reason for this requirement is that we have several members located globally that have a legitimate/more frequent need to have their own timezone (so we can't ask them to change to Eastern) but the dashboard in question specifically needs to report on issues using Eastern time (they need to look the same for everyone). I feel like there must be some simple way to do this that I just haven't found.
I'm not doing anything complicated right now, I'm just converting a UNIX timestamp with strftime:
| eval openTime=strftime(openTime,"%m/%d/%Y %H:%M:%S")
| eval closedTime=strftime(closedTime,"%m/%d/%Y %H:%M:%S")
When I display them in a table they display in whatever the user preference is for timezone. Every solution I've tried doesn't really seem to be a solution. It is easy to convert a timestamp with a timezone to unix. It is also easy to convert unix to a timestamp that shows your local timezone... but so far it seems impossible to convert a unix timestamp to a specific timezone (and have it display in that timezone instead of whatever the user has configured). Thoughts? To re-iterate, we cannot ask these users to change their timezone preference but these times MUST be shown in Eastern.
... View more
Labels
- Labels:
-
eval
10-17-2019
10:10 AM
A year and a half later and I've actually cracked this nut... you can "update" the app by replacing the Splunk SDK located here:
\etc\apps\splunk-auth0\bin\app\node_modules\splunk-sdk
With the latest Javascript version downloaded from:
http://dev.splunk.com/sdks
Once this has been updated the app should start working with the latest versions of Splunk. Make sure you restart Splunk.
... View more
09-21-2018
12:19 PM
Hello @dgwhjj, I was curious how you managed to get this to work... I understand, I think, the directory you're talking about... in my case, with Windows, it's this folder within the app itself:
However I'm not entirely understanding your fix... are you installing a new/separate Node JS to this folder location? This almost makes sense to me because there is a node_modules folder there (it's the only other folder there actually) but I wouldn't think that updating this would make the necessary updates to server.js (which the app needs to run). The Node JS in Splunk is already updated, so I'm assuming you are some how updating Node JS within the Auth0 app?
... View more
01-25-2018
06:24 AM
Updated the tag.
... View more
01-24-2018
09:04 AM
1 Karma
Been wrestling with this one for a while. Every day at the same time the JMX add-on stops working with the following error in the jmx.log:
2018-01-24 05:27:01,638 - com.splunk.modinput.ModularInput -60118777 [main] ERROR - Error executing modular input : java.lang.RuntimeException: java.net.SocketException: Socket is closed : java.lang.RuntimeException: java.lang.RuntimeException: java.net.SocketException: Socket is closed
at com.splunk.Entity.refresh(Unknown Source)
at com.splunk.Entity.refresh(Unknown Source)
at com.splunk.Resource.validate(Unknown Source)
at com.splunk.Entity.validate(Unknown Source)
at com.splunk.Entity.validate(Unknown Source)
at com.splunk.Resource.getTitle(Unknown Source)
at com.splunk.Resource.getName(Unknown Source)
at com.splunk.jmx.JMXModularInputV3.doRun(Unknown Source)
at com.splunk.modinput.ModularInput.init(Unknown Source)
at com.splunk.jmx.JMXModularInputV3.main(Unknown Source)
Caused by: java.lang.RuntimeException: java.net.SocketException: Socket is closed
at com.splunk.AtomObject.createReader(Unknown Source)
at com.splunk.AtomFeed.parseStream(Unknown Source)
... 10 more
Caused by: javax.xml.stream.XMLStreamException: java.net.SocketException: Socket is closed
at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.setInputSource(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.<init>(Unknown Source)
at com.sun.xml.internal.stream.XMLInputFactoryImpl.getXMLStreamReaderImpl(Unknown Source)
at com.sun.xml.internal.stream.XMLInputFactoryImpl.createXMLStreamReader(Unknown Source)
... 12 more
Caused by: java.net.SocketException: Socket is closed
at sun.security.ssl.SSLSocketImpl.checkEOF(Unknown Source)
at sun.security.ssl.AppInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.MeteredStream.read(Unknown Source)
at java.io.FilterInputStream.read(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startEntity(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startDocumentEntity(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.setInputSource(Unknown Source)
... 16 more
I've tried updating to the latest JMX app. The latest version of the JDK/JRE, etc. Based on the majority of the 'working' log it looks like JMX refreshes it's JMX server information every minute:
2018-01-24 05:20:46,081 - org.exolab.castor.mapping.Mapping -59743220 [Thread-2] INFO - Loading mapping descriptors from jar:file:/C:/Splunk/etc/apps/Splunk_TA_jmx/bin/lib/jmxmodinput.jar!/mapping.xml
2018-01-24 05:20:46,137 - com.splunk.modinput.ModularInput -59743276 [Thread-2] INFO - 552 servers found in stanza jmx://websitejava
2018-01-24 05:20:50,777 - com.splunk.modinput.ModularInput -59747916 [main] INFO - stanza count:1
2018-01-24 05:21:01,352 - com.splunk.modinput.ModularInput -59758491 [main] INFO - stanza count:1
2018-01-24 05:21:11,914 - com.splunk.modinput.ModularInput -59769053 [main] INFO - stanza count:1
2018-01-24 05:21:22,496 - com.splunk.modinput.ModularInput -59779635 [main] INFO - stanza count:1
2018-01-24 05:21:33,084 - com.splunk.modinput.ModularInput -59790223 [main] INFO - stanza count:1
2018-01-24 05:21:43,644 - com.splunk.modinput.ModularInput -59800783 [main] INFO - stanza count:1
2018-01-24 05:21:54,209 - com.splunk.modinput.ModularInput -59811348 [main] INFO - stanza count:1
2018-01-24 05:21:54,796 - org.exolab.castor.mapping.Mapping -59811935 [Thread-2] INFO - Loading mapping descriptors from jar:file:/C:/Splunk/etc/apps/Splunk_TA_jmx/bin/lib/jmxmodinput.jar!/mapping.xml
2018-01-24 05:21:54,853 - com.splunk.modinput.ModularInput -59811992 [Thread-2] INFO - 552 servers found in stanza jmx://websitejava
2018-01-24 05:22:04,806 - com.splunk.modinput.ModularInput -59821945 [main] INFO - stanza count:1
2018-01-24 05:22:15,387 - com.splunk.modinput.ModularInput -59832526 [main] INFO - stanza count:1
2018-01-24 05:22:26,009 - com.splunk.modinput.ModularInput -59843148 [main] INFO - stanza count:1
2018-01-24 05:22:36,567 - com.splunk.modinput.ModularInput -59853706 [main] INFO - stanza count:1
2018-01-24 05:22:47,138 - com.splunk.modinput.ModularInput -59864277 [main] INFO - stanza count:1
2018-01-24 05:22:57,720 - com.splunk.modinput.ModularInput -59874859 [main] INFO - stanza count:1
2018-01-24 05:23:03,632 - org.exolab.castor.mapping.Mapping -59880771 [Thread-2] INFO - Loading mapping descriptors from jar:file:/C:/Splunk/etc/apps/Splunk_TA_jmx/bin/lib/jmxmodinput.jar!/mapping.xml
2018-01-24 05:23:03,692 - com.splunk.modinput.ModularInput -59880831 [Thread-2] INFO - 552 servers found in stanza jmx://websitejava
2018-01-24 05:23:08,317 - com.splunk.modinput.ModularInput -59885456 [main] INFO - stanza count:1
2018-01-24 05:23:18,895 - com.splunk.modinput.ModularInput -59896034 [main] INFO - stanza count:1
2018-01-24 05:23:29,520 - com.splunk.modinput.ModularInput -59906659 [main] INFO - stanza count:1
2018-01-24 05:23:40,109 - com.splunk.modinput.ModularInput -59917248 [main] INFO - stanza count:1
2018-01-24 05:23:50,668 - com.splunk.modinput.ModularInput -59927807 [main] INFO - stanza count:1
2018-01-24 05:24:01,390 - com.splunk.modinput.ModularInput -59938529 [main] INFO - stanza count:1
2018-01-24 05:24:03,634 - org.exolab.castor.mapping.Mapping -59940773 [Thread-2] INFO - Loading mapping descriptors from jar:file:/C:/Splunk/etc/apps/Splunk_TA_jmx/bin/lib/jmxmodinput.jar!/mapping.xml
2018-01-24 05:24:03,693 - com.splunk.modinput.ModularInput -59940832 [Thread-2] INFO - 552 servers found in stanza jmx://websitejava
2018-01-24 05:24:11,950 - com.splunk.modinput.ModularInput -59949089 [main] INFO - stanza count:1
2018-01-24 05:24:22,509 - com.splunk.modinput.ModularInput -59959648 [main] INFO - stanza count:1
2018-01-24 05:24:33,110 - com.splunk.modinput.ModularInput -59970249 [main] INFO - stanza count:1
2018-01-24 05:24:43,696 - com.splunk.modinput.ModularInput -59980835 [main] INFO - stanza count:1
2018-01-24 05:24:54,316 - com.splunk.modinput.ModularInput -59991455 [main] INFO - stanza count:1
2018-01-24 05:25:03,721 - org.exolab.castor.mapping.Mapping -60000860 [Thread-2] INFO - Loading mapping descriptors from jar:file:/C:/Splunk/etc/apps/Splunk_TA_jmx/bin/lib/jmxmodinput.jar!/mapping.xml
2018-01-24 05:25:03,787 - com.splunk.modinput.ModularInput -60000926 [Thread-2] INFO - 552 servers found in stanza jmx://websitejava
2018-01-24 05:25:05,076 - com.splunk.modinput.ModularInput -60002215 [main] INFO - stanza count:1
2018-01-24 05:25:15,652 - com.splunk.modinput.ModularInput -60012791 [main] INFO - stanza count:1
2018-01-24 05:25:26,285 - com.splunk.modinput.ModularInput -60023424 [main] INFO - stanza count:1
2018-01-24 05:25:36,869 - com.splunk.modinput.ModularInput -60034008 [main] INFO - stanza count:1
2018-01-24 05:25:47,514 - com.splunk.modinput.ModularInput -60044653 [main] INFO - stanza count:1
2018-01-24 05:25:58,144 - com.splunk.modinput.ModularInput -60055283 [main] INFO - stanza count:1
2018-01-24 05:26:03,739 - org.exolab.castor.mapping.Mapping -60060878 [Thread-2] INFO - Loading mapping descriptors from jar:file:/C:/Splunk/etc/apps/Splunk_TA_jmx/bin/lib/jmxmodinput.jar!/mapping.xml
2018-01-24 05:26:03,807 - com.splunk.modinput.ModularInput -60060946 [Thread-2] INFO - 552 servers found in stanza jmx://websitejava
2018-01-24 05:26:08,929 - com.splunk.modinput.ModularInput -60066068 [main] INFO - stanza count:1
2018-01-24 05:26:19,499 - com.splunk.modinput.ModularInput -60076638 [main] INFO - stanza count:1
2018-01-24 05:26:30,047 - com.splunk.modinput.ModularInput -60087186 [main] INFO - stanza count:1
2018-01-24 05:26:40,639 - com.splunk.modinput.ModularInput -60097778 [main] INFO - stanza count:1
2018-01-24 05:26:51,213 - com.splunk.modinput.ModularInput -60108352 [main] INFO - stanza count:1
2018-01-24 05:27:01,638 - com.splunk.modinput.ModularInput -60118777 [main] ERROR - Error executing modular input : java.lang.RuntimeException: java.net.SocketException: Socket is closed : java.lang.RuntimeException: java.lang.RuntimeException: java.net.SocketException: Socket is closed
at com.splunk.Entity.refresh(Unknown Source)
at com.splunk.Entity.refresh(Unknown Source)
at com.splunk.Resource.validate(Unknown Source)
at com.splunk.Entity.validate(Unknown Source)
at com.splunk.Entity.validate(Unknown Source)
at com.splunk.Resource.getTitle(Unknown Source)
at com.splunk.Resource.getName(Unknown Source)
at com.splunk.jmx.JMXModularInputV3.doRun(Unknown Source)
at com.splunk.modinput.ModularInput.init(Unknown Source)
at com.splunk.jmx.JMXModularInputV3.main(Unknown Source)
Caused by: java.lang.RuntimeException: java.net.SocketException: Socket is closed
at com.splunk.AtomObject.createReader(Unknown Source)
at com.splunk.AtomFeed.parseStream(Unknown Source)
... 10 more
Caused by: javax.xml.stream.XMLStreamException: java.net.SocketException: Socket is closed
at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.setInputSource(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.<init>(Unknown Source)
at com.sun.xml.internal.stream.XMLInputFactoryImpl.getXMLStreamReaderImpl(Unknown Source)
at com.sun.xml.internal.stream.XMLInputFactoryImpl.createXMLStreamReader(Unknown Source)
... 12 more
Caused by: java.net.SocketException: Socket is closed
at sun.security.ssl.SSLSocketImpl.checkEOF(Unknown Source)
at sun.security.ssl.AppInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.MeteredStream.read(Unknown Source)
at java.io.FilterInputStream.read(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startEntity(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startDocumentEntity(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.setInputSource(Unknown Source)
... 16 more
but for whatever reason after 24 hours it fails with a "socket closed" and doesn't recover. I'm sure it's a configuration issue somewhere but haven't been sure where to look
... View more
01-22-2018
11:20 AM
Nevermind, I figured this out coincidentally shortly after posting... it appears the problem is with using ADFS as SSO into Splunk. The created SIDs for search are too large and cause problems when attempting to create sub-searches. I resolved the issue by making a generic "admin" account owner of the dashboard panels, instead of my ADFS account.
... View more
01-22-2018
10:54 AM
Been wrestling with this issue for a while now... I have a search like the below (sensitive information redacted). This works when using it in a regular search. When adding it as a dashboard panel it doesn't work. The weird thing is, it works in some dashboards/apps but not others:
index=xxxx_logs container_name="organization-api" namespace=pvcs "update" UserController
| spath input=log
| search action=update
| eval corrId=spath(log,"request_id")
| eval params=spath(log,"params")
| rex field=params "(?P<parameters>[^\"]+\"\:\"[^\"]+)" max_match=0
| rex field=parameters mode=sed "s/\"\:\"/: /g"
| eval userID=spath(params,"id")
| eval username=spath(params,"login_name")
| join corrId
[search index=xxxx_logs container_name="lbo" audit
| spath input=log
| eval corrId=spath(log,"corrId")
| eval byUser=spath(log,"byUserId")]
| table _time,controller,action,userID, username,parameters,request_id,byUser
| sort _time
The only error that comes up in the job inspector is:
"No matching fields exist"
This says to me that the search can't or doesn't run spath or for one reason or another can't extract the fields necessary to do the sub-search. The weird thing is, I can copy/paste this into another dashboard or run it separately as a search and it works fine. I've gone through the app and the dashboard source but I don't see anything that would be different between them configuration wise.
I've tried including | fields=* to try and force verbose mode. I've also tried setting permissions as high as they'll go on the app/dashboard hoping this would fix it but nada... thoughts?
... View more
06-02-2017
08:15 AM
If you use a custom datetime config XML it will probably work... this may do what you're looking for:
<datetime>
<define name="_xmldateformat" extract="year, month, day, hour, minute, second">
<text>\<year\>(\d+)\<\/year\>[\s\n\r]+\<month\>(\d+)\<\/month\>[\s\n\r]+\<day\>(\d+)\<\/day\>[\s\n\r]+\<hour\>(\d)\<\/hour\>[\s\n\r]+\<minute\>(\d+)\<\/minute\>[\s\n\r]+\<second\>(\d+)\<\/second\></text>
</define>
<timePatterns>
<use name="_xmldateformat"/>
</timePatterns>
<datePatterns>
<use name="_xmldateformat"/>
</datePatterns>
</datetime>
... View more
06-02-2017
07:50 AM
1 Karma
I think the problem is that you don't need to use INDEXED_EXTRACTIONS at all if you don't intend to have Splunk extract the data based on the field headers at the top of the CSV file. I think DELIMS is also a search-time extraction and it doesn't have any reliance on INDEXED_EXTRACTIONS. So you can use PREAMBLE_REGEX but just don't use INDEXED_EXTRACTIONS. Don't treat it like a CSV file, just treat it like a log file.
... View more
12-09-2016
11:41 AM
This does appear to get the job done. Upvoted/accepted.
... View more
11-29-2016
08:45 AM
Neither of these work unfortunately because they give the percentage spread across all months, rather than the percentage per month. For example, I would expect Windows 7 usage to be around 70% each month but using the above searches over a period of 3 months shows the percentage at around 23% per month (so it is giving a percentage per month across all months, rather than per month).
... View more
11-29-2016
08:12 AM
Hey there, I'm scratching my head trying to figure out how to do this.
Basically, I want to run a report on 6 months of browsing data and be able to view the percentage of usage on different operating systems (Windows 7, OS X, etc) per month over several months. It would hopefully look something like this:
OS August September October
Windows 7 68% 70% 70%
Windows 10 3% 5% 6%
Linux 10% 8% 7%
Windows 8.1 9% 8% 8%
MacOS 4% 4% 4%
Windows XP 2% 2% 2%
iPad 1% 1% 1%
Chrome OS 0% 0% 0%
Android OS 0% 0% 0%
Windows Vista 1% 1% 1%
Windows 8 1% 1% 1%
iPhone 0% 0% 0%
The closest I can get is this:
index=marketing_summary_sp | timechart span=1mon count by ua_os_family limit=0
That basically gives me the "count" of individual OS sessions per month but not the percentages. The only fields I think that would be involved are date_month and ua_os_family. I just can't figure out how to give per month percentages while also showing several other months. Thoughts?
... View more
11-10-2016
07:20 AM
Can you provide an example? . would indeed match any character.
... View more
11-10-2016
07:16 AM
1 Karma
The solution we used for this was actually to break it up into several Powershell scripts (5-10) and run them all concurrently. This didn't seem to impact performance on the indexer at all and each script ran at the same speed. So I guess if you need to thaw a few thousand buckets, do it in several concurrent scripts, unless you want to leave something running for several days.
... View more
11-09-2016
07:57 AM
Hello, we have a clustered index that basically has one master indexer and two child indexers. Our data moves to a frozen state after 3 months and we needed to run a report on 6 months worth of data. I moved buckets from the necessary index from the "frozendb" folder to the "thaweddb" folder and created a script to issue the following command to each bucket (bucket names being whatever the bucket was obviously):
splunk rebuild R:\splunkdb\sp_72_logs\thaweddb\rb_1469724228_1469715286_114_CACEB811-4B3C-4B60-AE46-A061185F4F10
This process took over 2 days and was still running when my Powershell session was abruptly ended. I was curious what people would recommend doing in this case or if anyone has ever noted the thawing process to be extremely slow. There doesn't appear to be a way for me to know which buckets are and aren't thawed or where the data actually ends to try and sort it out. Should I just run the process again and hope for the best? Will this duplicate the data? Why are rebuilds so slow?
... View more
09-22-2016
07:17 AM
1 Karma
Enjoy your karma.
... View more
09-21-2016
08:47 AM
2 Karma
Hey there, we're using Monitoring of Java Virtual Machines with JMX (SPLUNK4JMX) to capture JMX data from some of our applications. Right now we're getting the following error message from the SPLUNK4JMX deployment on our search heads:
Unable to initialize modular input "jmx" defined inside the app "SPLUNK4JMX": Introspecting scheme=jmx: script running failed (exited with code 1).
The solution to this error is to install the JDK and make sure the Java home path is set but I don't believe this should apply to the search heads because they will not be making any JMX calls (we have SPLUNK4JMX on a heavy forwarder with the JDK).
The app itself actually works perfectly well on the search heads and displays the JMX data without any problems. The only issue is that the error message keeps appearing. I've tried removing various files from the app, including the bin directory but it keeps trying to initialize the JMX modular input... I can't see where that information is coming from. Anyone know the solution to this?
... View more
09-09-2016
12:56 PM
I've been racking my brain over multi-searches, subsearches, and a few other methods I harvested from Google and Splunk Answers, but so far, I've not been able to find a solution for this.
Scenario:
I have two separate indexes. In both indexes, I have a field called "site" which contains a domain URL. In either one of these indexes, the site value could appear anywhere from not at all to hundreds of times. I need to construct a search that will boil down each index to the unique site URLs that appear in each, then compare the results of both indexes to see if there are URLs that appear in one index and not the other.
The search query does not necessarily need to do show what's missing from both at once, it could be either or, as long as I can modify the search to get the other results I'm looking for. Any idea how to do this?
... View more
08-23-2016
07:03 AM
That actually all looks good. I was going to suggest that possibly an EOF was causing Splunk to split the event. I've had something similar happen before. I think a good test would be taking that log file (the one with 5000 events) uploading it directly to your indexer through the GUI with the "Add Data" feature. Configure everything the same and see if the event is still breaking weird in the middle. I use this method sometimes if it seems like it should be working based on the config. If it works there then it means its something else.
... View more
08-23-2016
06:13 AM
Is there any reason to use index time extractions over search-time? Search-time extractions are better in 99% of cases. This is an overly complicated solution for something that could be done with just the search time extraction?
^Remark\=\"(?P<Remark>[^\s]+)
... View more
08-23-2016
06:06 AM
See this is answered now but I didn't put my regex in code format so it was all messed up... if you use the above as a search-time extraction it will work fine though.
... View more
08-22-2016
10:04 AM
If you are trying to capture everything after Remark=" and before the first space then you could use the following search time extraction:
^Remark\=\"(?P<Remark>[^\s]+)
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
02-24-2023
09:53 AM
|
Karma from
