Beyond Trust logs are not getting parsed

[js1235]

Hi,

The beyond trust log fields are not getting extracted. I tried both Index time field extraction and Search time field extractions to extract the fields. Below are the sample logs.

Dec 5 13:12:06 192.168.251.35 Agent Desc: "",Agent ID: "AppAudit",Agent Ver: "",Category: "PMM API SignAppIn",Source Host: "",Event Desc: "",Event Name: "Login",OS: "",Event Severity: "0",Source IP: "192.168.251.35",Event Subject: "106",Event Type: "0",User: "VPN-OTSA-EDMS-HANU",Workgroup Desc: "",Workgroup ID: "",Workgroup Location: "",AuditID: "2200680",ActionType: "Login",SystemName: "PMM API SignAppIn",AppUserID: "106",CreateDate: "12/5/2023 1:11:34 PM",UserName: "VPN-OTSA-EDMS-HANU",IPAddress: "192.168.251.35",Mapped Credential: "Primary",Mapped Credential Id: "2",Mapped Credential Description: "OFID-PS-Usersync",Mapped Credential Platform: "ActiveDirectory",Mapped Credential Domain/Server: "opecfund.org",Authenticate Credential Id: "2",Authenticate Credential UserName: "opecfund.org\OFID-PS-Usersync@opecfund.org",Authenticate Credential Description: "OFID-PS-Usersync",Authenticate Credential Platform: "ActiveDirectory",Domain Name: "opecfund.org",SAM Account Name: "VPN-OTSA-EDMS-HANU",Group: "opecfund.org\OFID-BTPRAPS-Vendor",Authentication Type: "Active Directory via API"
Dec 5 13:12:06 192.168.251.35 Agent Desc: "",Agent ID: "AppAudit",Agent Ver: "",Category: "PMM API SignAppIn",Source Host: "",Event Desc: "",Event Name: "Login",OS: "",Event Severity: "0",Source IP: "192.168.251.35",Event Subject: "38",Event Type: "0",User: "SA-BTPSGlobalRequestor4SRAAPI",Workgroup Desc: "",Workgroup ID: "",Workgroup Location: "",AuditID: "2200678",ActionType: "Login",SystemName: "PMM API SignAppIn",AppUserID: "38",CreateDate: "12/5/2023 1:11:23 PM",UserName: "SA-BTPSGlobalRequestor4SRAAPI",IPAddress: "192.168.251.35",Authentication Type: "API"
Dec 5 13:12:06 192.168.251.35 Agent Desc: "",Agent ID: "AppAudit",Agent Ver: "",Category: "PMM API Authentication Rule Failure",Source Host: "",Event Desc: "",Event Name: "Login",OS: "",Event Severity: "0",Source IP: "192.168.251.35",Event Subject: "-1",Event Type: "0",User: "opecfund.org\SA-BTPSGlobalRequestor4SRAAPI",Workgroup Desc: "",Workgroup ID: "",Workgroup Location: "",AuditID: "2200677",ActionType: "Login",SystemName: "PMM API Authentication Rule Failure",AppUserID: "-1",CreateDate: "12/5/2023 1:11:23 PM",UserName: "opecfund.org\SA-BTPSGlobalRequestor4SRAAPI",IPAddress: "192.168.251.35",Host Name: "SVR-BTPS01",User Name: "opecfund.org\SA-BTPSGlobalRequestor4SRAAPI",API Key: "****************************************************************************************************************************4416",IP Address: "192.168.251.35",Authentication Rule: "API Key",Message: "Invalid RunAs - UserNameOrPasswordAreIncorrect"
Dec 5 13:11:06 192.168.251.35 Agent Desc: "",Agent ID: "AppAudit",Agent Ver: "",Category: "PMM API SignAppIn",Source Host: "",Event Desc: "",Event Name: "Login",OS: "",Event Severity: "0",Source IP: "192.168.251.35",Event Subject: "38",Event Type: "0",User: "SA-BTPSGlobalRequestor4SRAAPI",Workgroup Desc: "",Workgroup ID: "",Workgroup Location: "",AuditID: "2200675",ActionType: "Login",SystemName: "PMM API SignAppIn",AppUserID: "38",CreateDate: "12/5/2023 1:10:28 PM",UserName: "SA-BTPSGlobalRequestor4SRAAPI",IPAddress: "192.168.251.35",Authentication Type: "API".

Can someone help here

Below are the props and transform which i tried on Index time field extraction

[beyondtrust]
KV_MODE = none
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(\w{3} \d{1,2} \d{2}:\d{2}:\d{2} \d+\.\d+\.\d+\.\d+)
NO_BINARY_CHECK = true
REPORT-keyvaluepairs = keyvalue


[keyvalue]
REGEX = (\w+\s?\w+): "[^"]*"
FORMAT = $1::$2
MV_ADD = true

Search time field extraction:

[beyondtrust]
EXTRACT-AgentDesc = Agent Desc: "(?P<Agent_Desc>[^"]+)"
EXTRACT-AgentID = Agent ID: "(?P<Agent_ID>[^"]+)"
EXTRACT-AgentVer = Agent Ver: "(?P<Agent_Ver>[^"]+)"
EXTRACT-Category = Category: "(?P<Category>[^"]+)"
EXTRACT-SourceHost = Source Host: "(?P<Source_Host>[^"]+)"
EXTRACT-EventDesc = Event Desc: "(?P<Event_Desc>[^"]+)"
EXTRACT-EventName = Event Name: "(?P<Event_Name>[^"]+)"
EXTRACT-OS = OS: "(?P<OS>[^"]+)"
EXTRACT-EventSeverity = Event Severity: "(?P<Event_Severity>\d+)"
EXTRACT-SourceIP = Source IP: "(?P<Source_IP>[^"]+)"
EXTRACT-EventSubject = Event Subject: "(?P<Event_Subject>[^"]+)"
EXTRACT-EventType = Event Type: "(?P<Event_Type>\d+)"
EXTRACT-User = User: "(?P<User>[^"]+)"
EXTRACT-WorkgroupDesc = Workgroup Desc: "(?P<Workgroup_Desc>[^"]+)"
EXTRACT-WorkgroupID = Workgroup ID: "(?P<Workgroup_ID>[^"]+)"
EXTRACT-WorkgroupLocation = Workgroup Location: "(?P<Workgroup_Location>[^"]+)"
EXTRACT-AuditID = AuditID: "(?P<Audit_ID>\d+)"
EXTRACT-ActionType = ActionType: "(?P<Action_Type>[^"]+)"
EXTRACT-SystemName = SystemName: "(?P<System_Name>[^"]+)"
EXTRACT-AppUserID = AppUserID: "(?P<App_User_ID>[^"]+)"
EXTRACT-CreateDate = CreateDate: "(?P<Create_Date>[^"]+)"
EXTRACT-UserName = UserName: "(?P<UserName>[^"]+)"
EXTRACT-IPAddress = IPAddress: "(?P<IPAddress>[^"]+)"
EXTRACT-AuthenticationType = Authentication Type: "(?P<Authentication_Type>[^"]+)"
EXTRACT-HostName = Host Name: "(?P<Host_Name>[^"]+)"
EXTRACT-APIKey = API Key: "(?P<API_Key>[^"]+)"
EXTRACT-IPAddress2 = IP Address: "(?P<IP_Address2>[^"]+)"
EXTRACT-AuthenticationRule = Authentication Rule: "(?P<Authentication_Rule>[^"]+)"
EXTRACT-Message = Message: "(?P<Message>[^"]+)"

[PickleRick]

Where do you have those settings applied? Remember that index-time settings (like line-breaking, timestamp recognition/parsing) go to indexing tier (HFs/indexers) while search-time settings are needed on the search tier (it doesn't hurt to have the full set of settings on both tiers - unneeded settings are just not used there).

[js1235]

For Index time i have applied in Heavy Forwarders and for Search time i tried in Search Head.

[js1235]

I tried both index time and search time but nothing got worked.

[PickleRick]

OK, first things first - are you searching in fast or verbose mode?

Did you try to search for a value (even any value like something=*) in any of those fields?

[js1235]

I'm searching in verbose mode. Yes i tried searching for a filed and value.. the events are filtering 

[PickleRick]

That's.... wierd.

If you search, for example, for UserName=*, you get events but those events don't show the UserName field?

[js1235]

Yes, if i search for any field and value, the events are filtering based on my search, but the fields are not getting extracted.