We're using the fieldsummary function in splunk to return the list of fields (as it was designed) for each of our indexes. This works great for almost all our indexes except for our windows snare index. When fieldsummary is run on this index we get all the fields plus each individual log line being returned.
Does anyone know how fieldsummary works and if the query can be run manually? and/or if there's some sort of character limitation on fieldsummary that our windows event logs are tripping causing it spew all the log lines when the command is executed?
e.g
index=snare | fieldsummary | table field
field
Account_Domain
Account_For_Which_Logon_Failed
Account_Name
Account_Whose_Credentials_Were_Used
Additional_Information
Authentication_Package
Caller_Domain
Caller_Logon_ID
Caller_Process_ID
Caller_Process_Name
Caller_User_Name
CategoryString
Certificate_Information
Certificate_Issuer_Name
Certificate_Serial_Number
Certificate_Thumbprint
Client_Address
Client_Port
ComputerName
Creator_Process_ID
Criticality
DataString
Detailed_Authentication_Information
Domain
EventCode
EventLog
EventLogType
Event_Log
ExpandedString
Failure_Code
Failure_Information
Failure_Reason
Image_File_Name
Key_Length
Logon_Account
Logon_GUID
Logon_ID
Logon_Process
Logon_Type
Mar_11_20_08_03_10_200_12_14_YYY0029_xxxxx_xxxxx_com_MSWinEventLog_0_Security_200054_Mon_Mar_11_16_06_18_2013_4624_Microsoft_Windows_Security_Auditing_Ixxxxx_smith_N_A_Success_Audit_YYY0029_xxxxx_xxxxx_com_Logon__An_account_was_successfully_logged_on_____Subject
Mar_11_20_08_03_10_200_12_14_YYY0029_xxxxx_xxxxx_com_MSWinEventLog_0_Security_200056_Mon_Mar_11_16_06_18_2013_4624_Microsoft_Windows_Security_Auditing_Ixxxxx_smith_N_A_Success_Audit_YYY0029_xxxxx_xxxxx_com_Logon__An_account_was_successfully_logged_on_____Subject
Mar_11_20_08_03_10_200_70_45_YYY45_xxxxx_xxxxx_com_MSWinEventLog_0_Security_123596_Mon_Mar_11_16_06_06_2013_540_Security_somesrvacct_User_Success_Audit_YYY45_Logon_Logoff__Successful_Network_Logon
Mar_11_20_08_03_10_200_70_45_YYY45_xxxxx_xxxxx_com_MSWinEventLog_0_Security_123597_Mon_Mar_11_16_06_06_2013_538_Security_somesrvacct_User_Success_Audit_YYY45_Logon_Logoff__User_Logoff
Mar_11_20_08_03_10_200_70_45_YYY45_xxxxx_xxxxx_com_MSWinEventLog_0_Security_123598_Mon_Mar_11_16_06_06_2013_576_Security_somesrvacct_User_Success_Audit_YYY45_Logon_Logoff__Special_privileges_assigned_to_new_logon
Mar_11_20_08_03_10_200_70_45_YYY45_xxxxx_xxxxx_com_MSWinEventLog_0_Security_123599_Mon_Mar_11_16_06_06_2013_540_Security_somesrvacct_User_Success_Audit_YYY45_Logon_Logoff__Successful_Network_Logon
Mar_11_20_08_03_10_200_70_45_YYY45_xxxxx_xxxxx_com_MSWinEventLog_0_Security_123600_Mon_Mar_11_16_06_06_2013_538_Security_somesrvacct_User_Success_Audit_YYY45_Logon_Logoff__User_Logoff
Mar_11_20_08_03_10_200_86_180_YYY5686_xxxxx_xxxxx_com_MSWinEventLog_0_Security_2585_Mon_Mar_11_16_06_18_2013_4673_Microsoft_Windows_Security_Auditing_NT_AUTHORITY_LOCAL_SERVICE_N_A_Failure_Audit_YYY5686_xxxxx_xxxxx_com_Sensitive_Privilege_Use__A_privileged_service_was_called_____Subject
Mar_11_20_08_03_10_202_105_17_YYY101155_xxxxx_xxxxx_com_MSWinEventLog_0_Security_230359_Mon_Mar_11_16_06_17_2013_4624_Microsoft_Windows_Security_Auditing_NT_AUTHORITY_ANONYMOUS_LOGON_N_A_Success_Audit_YYY101155_xxxxx_xxxxx_com_Logon__An_account_was_successfully_logged_on_____Subject
Mar_11_20_08_03_10_202_105_17_YYY101155_xxxxx_xxxxx_com_MSWinEventLog_0_Security_230360_Mon_Mar_11_16_06_17_2013_4624_Microsoft_Windows_Security_Auditing_NT_AUTHORITY_ANONYMOUS_LOGON_N_A_Success_Audit_YYY101155_xxxxx_xxxxx_com_Logon__An_account_was_successfully_logged_on_____Subject
....
whereas
index=dns | fieldsummary | table field
field
Context
Direction
InternalPktID
Protocol
Thread_ID
date_hour
date_mday
date_minute
date_month
date_second
date_wday
date_year
date_zone
dest_domain
eventtype
host
index
linecount
product
punct
source
sourcetype
splunk_server
src_ip
vendor
xid
Did you try something like:
index=snare |field - Mar_* | fieldsummary | table field
Also, it may help to add a cluster command in the middle to reduce the load.
index=snare |fields - Mar_* | cluster | fields - cluster_* | fieldsummary | table field
It's not a maxval issue. I'm just trying to get the field names, not the values in the fields.
manually.. running the query I mean generate the same data without using the "fieldsummary" command, some other method of generating the same data.
All the lines.. that start with "Mar_11_20_08_03_10_202_" are gibberish.. not actual fields.
not sure if this is what the problem is, but are you explicitly setting the maxvals argument? it has a default value of 100 distinct values to return for each field if you don't set it explicitly.
also, can you provide the search string you're using and a sample of the data that is working and a sample of what's not?
also, what do you mean by running the query manually? it's a search command, so you can run it on the commandline if you have the necessary permissions.