All Apps and Add-ons

Shared memory between separate stanza for login token

shreyans
Path Finder

Hi,

I am using rest modular input app from quite a while but this time I have a new but very much valid scenario to address

In my inputs.conf I have around 200 stanza (Rest api to call) and fact is all the api are using same login method and same custom auth handler

Now with default setting I see that rest.py process is newly created for every stanza and that leads to separate authentication for every stanza (It uses same object for subsequent call after polling interval) . As I have 200 Stanza in inputs.conf, REST api is calling auth for 200 times for separate API which I wanted to avoid as I know that one time login and generating login token would be suffice for all the other rest api until that token is expired

My question is,
Is there any way I can share the token between stanza or any build in functionality in rest modular input which I am not aware of and can be used in this scenario

Best Regards
- Shreyans Soni

0 Karma

Damien_Dallimor
Ultra Champion

In inputs.conf you can define a parent stanza for common shared values that gets inherited by child stanzas.

[rest]
sourcetype = abc
index=main
...

[rest://foo]
endpoint=http://foo.com
...

[rest://goo]
endpoint=http://goo.com
...

As for 200 connections sharing 1 single authentication flow , this feature is not in the App.You would have to customize the code yourself for your specific requirement to achieve this functionality.

The only work around I can think of is to have some other auth process that acquires your tokens and writes them to the parent [rest] stanza shown above so that all the children can inherit those auth settings and not perform individual authentication flows.

shreyans
Path Finder

Adding to that I have customize my code to
1. Write token to local file and use it from there but since it is login token... it's not secure enough to write the token on a file and use it
2. Write token to environment variable but since rest.py is separate process, environment variable set in one process will not be accessible by other process
3. Other shared memory concept of python is not useful as we have to have process array to shared memory between those process and i guess all the process creation of rest.py is happening in .egg file kept inside app
Are you suggesting here to change .egg file? to have that customization of shared memory between objects of rest.py?

kindly help clarify

0 Karma

shreyans
Path Finder

Thanks Damien for quick reply appreciated.....

But if I understand the rest modular app correctly it will inherit the base stanza and it's value to sub stanza correct?

so if I have custom auth handler written as some python class for base stanza which was inherited by all sub stanza
then also my auth will be called multiple times for all sub stanza, and will not server purpose.....

as rest.py itself is separate for separate stanza. Please correct me if I am making any mistake here

0 Karma

Damien_Dallimor
Ultra Champion

I think you need to re-read my answer.

0 Karma

shreyans
Path Finder

Thanks Damien you are very prompt and quick in replying 🙂 thanks for your help

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!