License Usage past 30 days does not work. (Distributed Management Console)

[pattokt]

Overview of Issue

I've seen a few discussions on this topic, but none of them seem clear to me. We have two indexers configured as license slaves that report back to our license master. The license master runs on the same server as the Distributed Management Console. The indexers are also setup as search peers from the DMC console. I'm not sure why I would want to forward my DMC _internal events to the indexers when the indexers specifically state that they report licensing info back to the DMC.

The default search returns no results:

index=_internal host=lspldmc source=*license_usage.log type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(b) AS b by slave, pool, _time | timechart span=1d sum(b) AS "volume" fixedrange=false | join type=outer _time [search index=_internal host=lspldmc source=*license_usage.log type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(stacksz) AS "stack size" by _time] | fields - _timediff | foreach * [eval <>=round('<>'/1024/1024/1024, 3)]

Returns accurate license data

index=_internal splunk_server=local source=*license_usage.log type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(b) AS b by slave, pool, _time | timechart span=1d sum(b) AS "volume" fixedrange=false | join type=outer _time [search index=_internal host=lspldmc source=*license_usage.log type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(stacksz) AS "stack size" by _time] | fields - _timediff | foreach * [eval <>=round('<>'/1024/1024/1024, 3)]

Troubleshooting

Running the below search from the dmc returns the event below.

index=_internal source=*license_usage.log

03-03-2016 19:08:01.766 -0600 INFO LicenseUsage - type=Message - License usage logging not available for slave licensing instances, please see license_usage.log on license master=https://lspldmc:8089 for usage breakdown

The two hosts searched are my indexers.

Links already researched:

1. http://docs.splunk.com/Documentation/Splunk/latest/Admin/LicenseUsageReportViewexamples
2. https://answers.splunk.com/answers/113466/license-usage-past-30-days-dont-work.html

[stevepraz · Accepted solution]

I was having a similar issue but I think I figured it out. So, my setup was a license server/DMC server. My last 30 days license reports weren't working. I did a bunch of digging on answers and found some ideas like yours. It was working in a similar setup in non-prod. By chance I looked at my distsearch.conf and noticed that my DMC/license server was categorized as an indexer (it was not setup to forward the data to the other indexers) while in prod that was not the case.

I went into the DMC roles in prod and made my DMC/License server also have the indexer role and my reports started working again.

[pattokt]

As soon as I changed my DMC role to also be an indexer like you suggested it started working!!!

[pattokt]

Your search does work, but it goes back to the original issue where I have to specify splunk_server=local in order to make it work. I do like your search though for license usage.

[polymorphic]

I use this search and i believe that it should work in your environment as well (I might be wrong 😉 )

index=_internal source=license_usage (type=Usage OR type=RolloverSummary) earliest=-30d
| timechart span=1d sum(eval(if(type="Usage", b, 0))) AS volume max(eval(if(type="RolloverSummary", stacksz, 0))) AS limit
| eval volume=round((volume / 1024 / 1024 / 1024), 2)
| eval limit=round((limit / 1024 / 1024 / 1024), 2)
| eval free=(limit - volume)
| fields - limit
| rename volume as "Used GB" free as "Free GB"

[pattokt]

Your search does work, but I arrive at the same issue where I need to specify splunk_server=local in the search. I do like the search you provided though.

[polymorphic]

Maybe if you added splunk_server=* instead?

Unfortunately i don't have a setup similar to yours, so i'm unable to test.

Some sample data might be helpfull