How to change the index at index time

[mataharry]

I have data coming from syslog udp:514
but I want to send some events to a different index.

Depending of :

• the host they are coming from : ftp or vpn should go to indexB
• some keywords in the events. "AUDIT" or "ALERT" should go to indexC

[mataharry]

For retention mostly, the vpn logs are voluminous, and I want to keep only 1 month of them.
But sometime for permissions, I want only my security team to be able to see the "ALERT" events.

[gkanapathy]

I would also ask why think you want to send them to different indexes? is it for security reasons? retention reasons?

[yannK · Accepted solution]

Method 1 :
The optimal solution is to open several ports, one for each index and have your syslog routed to the proper port.

example :

[udp:514]
index=myindexA
[udp:515]
index=myindexB
[udp:516]
index=myindexC
#etc...

It is also possible to then enforce a sourcetype or a timezone per source .

---

Method 2: If you cannot separate your inputs, then you have to create props/transforms rules to change the index based on : the content of the events, or the host.

see
in inputs.conf

[udp:514]
index=myindexA
sourcetype=syslog


in props.conf
To apply only for your input

[source::udp:514]
TRANSFORMS-changeindex=routeIndexbasedonhost, routeIndexbasedonevent

OR to apply to all syslog sourcetype


[syslog]
TRANSFORMS-changeindex=routeIndexbasedonhost, routeIndexbasedonevent


in transforms.conf

`
[routeIndexbasedonhost]
SOURCE_KEY = MetaData:Host
REGEX=(ftp|vpn)
DEST_KEY=_MetaData:Index
FORMAT=myindexB
WRITE_META=true

[routeIndexbasedonevent]
REGEX=(AUDIT|ALERT)
DEST_KEY=_MetaData:Index
FORMAT=myindexC
WRITE_META=true
`

The order of the transforms is important:
TRANSFORMS-changeindex=routeIndexbasedonhost, routeIndexbasedonevent
will have : myindexA for all, then some will become myindexB, and finally some will become myindexC
so the last one will overwrite the previous ones.