About Paolo_Prigione

Paolo_Prigione · ‎07-19-2011

You might want to use the map command to take the "Start" and "End" timestamps from your events and run sub-searches using them as parameters... ... | ... get the start and end timestamps as fields... | map search="search earliest::$Start$ latest::$End$ sourcetype=.... | transaction hostname" maxsearches=10

Paolo_Prigione · ‎07-15-2011

I have seen the same problem with Splunk 4.2.1 32bit on a Windows 2k3 32 bit with underlying Intel Xeon CPU

Paolo_Prigione · ‎07-12-2011

I want to group consecutive lines starting with the same pattern. I know the TRANSACTION command can be used as well, but having it already indexed correctly would be very handy... Something like LINE_BREAKER = ([\n\r]+)((\S+\s+){4}).*([\n\r]+\2.*)* Thanks, Paolo

Paolo_Prigione · ‎07-08-2011

Hi all, I am reasoning about the *nix app sourcetype=ps' pctCPU metric and how to plot it correctly. I see Splunk's nix app generally plots it by doing *... | timechart avg(pctCPU) by ...**. This would be perfectly fine if pctCPU expressed the instantaneous usage of cpu (as top does). Instead, per "man ps" definition (RHEL 5.5): %cpu %CPU cpu utilization of the process in "##.#" format. Currently, it is the CPU time used divided by the time the process has been running (cputime/realtime ratio), expressed as a percentage. It will not add up to 100% unless you are lucky. (alias pcpu). pctCPU expresses the average cpu used by the process since its startup (cpu time / total run time)! Say the process has long been running with low usage, then it has a burst for some minutes, then usage drops again. Ps' pctCPU would not reflect this behaviour as the total cpu time over which it has been computed did not increment that much with respect to total runtime. pctCPU is smoothed in this case. Does my reasoning make any sense to you? I have a quite complex solution under work which involves computing deltas of CPUTIME and ELAPSED (splunk's ps.sh definitions) for any multikv'ed ps execution, then compute "instantaneous" pctCPU, then average and plot it. However, this is a fairly slow (requires to use | sort +host +PID +COMMAND +_time with all its limits) and complex solution. Has anybody came up with something better?

Paolo_Prigione · ‎06-29-2011

Summary indexes are not a sort of excel file in which you populate some cells of the same lines at different times. Deduping works at the "row" level, not column. However, it all depends on the search you use to read summary data and present it to the user, so fields on different summary index events might become part of the same row in a table (transactions, | bucket _time span=1h | stats first(field1), first(field6) by _time, ....) I think you might want to create a test summary index and test this a little bit. I'm not sure I got your usecase right

Paolo_Prigione · ‎06-29-2011

Well, technically a scheduled search CAN backfill data, provided you execute it from the Command line interface. All the explanations are in the docs. There is a python script which will take care of running a saved search for over period of time in the past (-et and -lt) running it as many times as it would have run for real. ./splunk cmd python fill_summary_index.py -app search -name *your_saved_seach* -et -2d@d -lt @d -j 8 -owner admin -auth admin:changeme The -dedup param might suit your need not to override data that is already there. I think this might be very helpful in your case.

Paolo_Prigione · ‎06-29-2011

The defaultGroup setting should specify the stanza name, "cm-processing", and not the server name, "cm". As the forward server setting is always tricky to set, I'd suggest you to log in onto a vanilla universal forwarder and run splunk add forward-server cm:9997 then copy over the $SPLUNK_HOME/etc/system/local/outputs.conf settings generated by the command in your configuation distribution system. In any case, the final configuration of your outputs.conf should be (untested): [tcpout] defaultGroup=cm-processing [tcpout:cm-processing] server=cm:9997 [tcpout-server://cm:9997] Cheers, Paolo

Paolo_Prigione · ‎06-27-2011

Here you go: "Drive Type: (?:SATAII |FC )*(?<drive_tier>Fibre|SATAII(?! SSD)|SSD)" Results: Drive Type: SATAII SSD --> SSD Drive Type: SATAII --> SATAII Drive Type: Fibre Channel --> Fibre Drive Type: FC SSD --> SSD There's no way to have it output an uppercase "FIBRE" as regexes only output what was in the original text.

Paolo_Prigione · ‎06-21-2011

Hi RNB, some things I've noticed: Any Splunk chart expects data in a specific format, which is specified in the Charting Reference page. A <chart> panel won't generally plot a list of events, as nick noted. You should have a command such as stats or timechart to prepare data for plotting. Maybe this is the reason why you don't get any chart. In case you have missing data points and you are using line or area charts, splunk will not show you those points in a line chart. You've got to tell it how to deal with those gaps, which is the following option: <option name="charting.nullValueMode">(accepted values: gaps, zero, and connect)</option>, to be put between the <chart> and </chart> tags. If possible, add markes to line charts in oder to visualize data points which are not connected to anything else: <option name="charting.showMarkers ">true</option> As a personal suggestion, try to manually run your saved search and see whether, on the rightmost part of the screen over the list of returned events/results, you have a "show report" or "build report" button: the former means your data can aleady be plotted: open the report viewer and play with the settings at wish the latter means your data is not yet suitable for charting, so you will need some repoting command (top, stats, timechart,... therefore to post process the search results) Cheers! Paolo

Paolo_Prigione · ‎06-21-2011

You won't need the UF to forward to the indexers, that can be done by the SH itself if you configure it from the Manager's forwarding/receiving configuration panel: you might want to set "index and forward" to false in the "Forwarding defaults" page (no indexing happening on the SH) You'll then need to add a forward server by specifying your indexer's dns name and 9997 port (if you are using default configs) You'll also need to create the "os" index on the indexer as those data won't be stoerd on the SH anymore. Also, override the default system/default/outputs.conf configuation in case you want to ship your SH splunk logs to the indexer as well (index=_intenal is not forwarder by default). But that is another topic...

Paolo_Prigione · ‎06-09-2011

This is a known bug, which is present in the Release Notes' Known Issues page. There is no way to escape an asterisk (*) in the search language. (SPL-30079) So you should go for the suggested workarounds...

Paolo_Prigione · ‎06-03-2011

I like the ongoing theme of spelunking, but IMHO that would be too native-english-speaker specific?

Paolo_Prigione · ‎05-12-2011

Hi James, neither of them if you talk about hot/warm buckets. Those are created and rotated solely based on the size and number of those already in these statuses. Older hot buckets are rolled to warm when they are more than maxHotBuckets, each as big as maxDataSize specifies (plus some other idle data rules...). Older warm buckets are rolled to cold when they are more than maxWarmDBCount. Older cold buckets are rolled to frozen when their latest event timestamp is older than frozenTimePeriodInSecs. Also, if the index grows more than maxTotalDataSizeMB, the oldest data is rolled to frozen. Some more settings may be applied, please check the Index.conf docs for that. Paolo

Paolo_Prigione · ‎05-11-2011

Have you tried to turn the last comma into optional? REGEX= (?<t>[^,]*),?

Paolo_Prigione · ‎05-09-2011

If you enable the app on the search head (SH) and the SH already lists all the indexers as search peers, then you should be able to get the aggregate view from the SH itself. However, in case your SH does not store the summary indexes locally but rather forwards everything to the indexers themselves, then you'll have to manually create the Deployment Monitor's specific indexes on the indexers, too. EDIT: in case you have multiple search heads, you'd better follow these docs.

Paolo_Prigione · ‎05-09-2011

servername is used to identify the Splunk instance for features such as distributed search and defaults to <hostname>-<user running splunk>. You set this in server.conf. default-hostname is used as default "host" field on all events coming from that Splunk instance. You set this in inputs.conf.

Paolo_Prigione · ‎04-19-2011

No, I don't. But might be worth the test. Was it a search command, you could make it "streaming", so that it executes while the search is still running. Don't know how lookups are managed.

Paolo_Prigione · ‎04-19-2011

Why not a scripted python lookup, then? Would be included automatically for that sourcetype (or source) and (I think) it might overwrite the destination fields themselves. It would totally go unnoticed.

Paolo_Prigione · ‎04-19-2011

Even if you coded this into props.conf, the evaluation would still run at search time so I see no benefit there. But that's not possible anyway 🙂 There are, however, alternatives: If you want to report on long periods of time, you might wish to use summary indexing to pre-compute every sort of metric... If you have these data show up in dashboards, you could use scheduled searches to feed the dashboards so that splunk won't have to run a search every time somedody loads the page. A walkthrough here. I'm short of other options...

Paolo_Prigione · ‎04-19-2011

MAX_TIMESTAMP_LOOKAHEAD gives the last char position before which the timestamp have to be found, not it's starting point.

Paolo_Prigione · ‎04-05-2011

You're welcome!

Paolo_Prigione · ‎04-04-2011

Yes, it is possible. But if your search performs data aggregation (timechart, stats, chart,...) you might want to use summary indexing instead... Which kind of search is that? To re-use a search results in a dashboard you basically have to: build a saved search that will feed your view (it will gather 1 week worth of data), schedule that search to run weekly (use the CRON syntax, read up the cron_schedule param in the conf), build an advanced view which uses that saved search, configure the HiddenSavedSearch module to use the saved search history instead of re-running it (set "useHistory" param to "True").

Paolo_Prigione · ‎04-04-2011

If you are using Splunk 4.2 you'd need to update the stylesheet file: $SPLUNK_HOME/etc/apps/maps/appserver/modules/GoogleMaps/GoogleMaps.css Everywhere you see this: url(img/... you should update it with: url(GoogleMaps/img/... Save, restart Splunk (might not be necessary) and delete your browser's cache. More info on: http://answers.splunk.com/questions/12836/custom-modules-css-images-not-working-in-4-2

Paolo_Prigione · ‎04-01-2011

Yeah, if you know the sourcetype in advance, always force it manually. Or devise some sourcetype assignment rules in props.conf based on the source field. This way you'll never get any unwanted surprise.

Paolo_Prigione · ‎03-28-2011

Thanks for the suggestion. At the time I didn't have the skills to set that up. I had solved the problem at search time: the volume was not that bad.

Posts	203
Solutions	46
Karma Given	52
Karma Received	224
Member Since	‎03-30-2010

Online Status	Offline
Date Last Visited	‎06-05-2020 02:02 AM

Splunk DB Connect: extremely long SQL query doesn'...

Splunk DB Connect and Neo4j anyone?

WMI route to nullQueue based on host. Is it possib...

Integration of Imperva Database Activity Monitor (...

How to get data data from ArcSight Connectors

Form-view and flashtimeline displaying different r...

SimpleXML row grouping="x,y" not working on form d...

Can I use backreferences in LINE_BREAKER?

Is *nix sourcetype=ps' pctCPU really suitable for ...

Custom module's css images not working in 4.2

Re: Transactions/Stats?

Re: Error starting Splunkweb Service (HELP!!)

Can I use backreferences in LINE_BREAKER?

Is *nix sourcetype=ps' pctCPU really suitable for ...

Re: Two questions: API and Backfill?

Re: Two questions: API and Backfill?

Re: forward-server listed as inactive

Re: Specific rex doesnt seem to work

Re: Chart not plotting in dashboard

Re: UniversalForwarder and Splunk 4.2.x on Search ...

Re: field value *

Re: What should we name Splunk User Groups? How ab...

Re: What initiates the data rotation between bucke...

Re: Regex for transforms.conf for splitting string

Re: How can I forward all "Splunk Deployment Monit...

Re: Difference between 'show default-hostname' and...

Re: Putting Calculations in .conf Files

Re: Putting Calculations in .conf Files

Re: Putting Calculations in .conf Files

Re: Why are my events splitting by second and not ...

Re: How to configure Static weekly report on dashb...

Re: How to configure Static weekly report on dashb...

Re: Not Displaying Google Event Icons

Re: Problem with data parsing

Re: Routing to nullQueue based on extracted field

Join the Conversation