All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Have you checked from mongodb.log why this is not starting? There is one another case where Windows OS was not supported by Splunk 9.4.0 version. https://community.splunk.com/t5/Splunk-Enterprise/KVs... See more...
Have you checked from mongodb.log why this is not starting? There is one another case where Windows OS was not supported by Splunk 9.4.0 version. https://community.splunk.com/t5/Splunk-Enterprise/KVstore-unable-to-start-after-upgrade-to-Splunk-Enterprise-9-4/m-p/708264#M21264
I did some reading of the documentation and realized that underlying Mongo DB was upgraded to 7. I figured out that Mongo DB 5+ requires AVX instruction set.  So time to check if CPU supports AVX in... See more...
I did some reading of the documentation and realized that underlying Mongo DB was upgraded to 7. I figured out that Mongo DB 5+ requires AVX instruction set.  So time to check if CPU supports AVX instruction set - in my case the CPU model did support this instructions. But running the lscpu command didnt show AVX flags. It turned out that AVX instructions were not available, because the VM had Processor compatibility mode enabled. In hyper-v we had to remove "Allow migration to a virtual machine host with a different processor version" checkbox.  After VM was restarted, AVX appeared in CPU flags and Splunk KV Store was operational. Lession learned: before upgrading  to 9.4 (or making fresh install), check if AVX flag is available. If it isn't, it is about time to upgrade your hardware and in stick to Splunk 9.3.
I am posting this to maybe save you from few hours of troubleshooting like I did. I did clean install of Splunk 9.4 in small customer environment with virtualized AIO instance. After the installat... See more...
I am posting this to maybe save you from few hours of troubleshooting like I did. I did clean install of Splunk 9.4 in small customer environment with virtualized AIO instance. After the installation there was an error notifying that KV Store can not start and that mongo log should be checked. The following error was logged:   ERROR KVStoreConfigurationProvider [4755 KVStoreConfigurationThread] - Could not start mongo instance. Initialization failed.     Mongod.log was completely empty.  So there was no clues in the log files about what is wrong and what can I do to make KVStore operational. Time to start Googling. Solution will be posted in the next post.
How about ceil for diffTime? https://docs.splunk.com/Documentation/Splunk/9.4.0/SearchReference/MathematicalFunctions#ceiling.28.26lt.3Bnum.26gt.3B.29_or_ceil.28.26lt.3Bnum.26gt.3B.29
If you can do you can try to standardise your company log messages like Mandatory part Here are some fields/information that every event must contain, regardless of the service. It doesn’t matter i... See more...
If you can do you can try to standardise your company log messages like Mandatory part Here are some fields/information that every event must contain, regardless of the service. It doesn’t matter if those fields are KV, JSON, or just placement formatted. More important is that these are always present and easily identified in the log event.       Field   Content   Example   Purpose   timestamp RFC-3339 formatted timestamp 2024-07-01T12:13:15.123+03:00 When an event occurs in this service. log_type audit/apps/trace/metric audit What is the event type from a security content perspective? source_ip IP of logging systems / service 10.11.22.123 Where did the event occur from an IP perspective? source_system Source System of log event aa.bb.local Host or service where event was created. process Process / service which has processed event app_abc Which application / service processed the event. sessionId Session where event belongs 82B98B54-9553-43CD-A5AB-E6F45656CD95 e.g. GUID to identify the entire session. requestId Request where event belongs 9DF09DE7-4061-487B-953C-49B73C000E2C e.g. GUID to identify individual request within session. userId User’s identification on service. a12345 Pseudonyms should be used instead of real user IDs to avoid exposing PII. outcome Status of action Error Did the action succeed or fail? errorDetails Details for action result Not authorized A more detailed error message, including the full message, could be part of the service-based payload payload Application / service specific parts { “as”:23, { “aa”:”bb”, “cc”:12}} A separate payload based on the real audit trail needs.   In that way you could do some kind of DM based on this, but as I said usually the payload is the interesting part and this is different by every subsystem / service etc. And there are lot equipments which have their own log format.
Thank you for the input everyone.  @isoutamo - you are correct in that each data source I'm looking at has vastly different data available... Some sources come from endpoint agents which have userna... See more...
Thank you for the input everyone.  @isoutamo - you are correct in that each data source I'm looking at has vastly different data available... Some sources come from endpoint agents which have username, endpoint name, ip address (local/public), url, url ip, etc.  Other sources from network devices and might track users by local IP only, but also might have which FW the request goes through, etc.  I have one source which only lists a single field to identify the user.... the MAC address... really not helpful without an additional lookup. I ended up using a number of macros and lots of coalesces to make my field names consistent.
I have this search, where I get the duration and I need to convert it to integer: Example: Min:Sec to Whole 00:02      to   1 00:16      to   1 01:53      to  2 09:20      to  10 ...etc S... See more...
I have this search, where I get the duration and I need to convert it to integer: Example: Min:Sec to Whole 00:02      to   1 00:16      to   1 01:53      to  2 09:20      to  10 ...etc Script: index="cdr" | search "Call.TermParty.TrunkGroup.TrunkGroupId"="2811" OR "Call.TermParty.TrunkGroup.TrunkGroupId"="2810" "Call.ConnectTime"=* "Call.DisconnectTime"=* |lookup Pais Call.RoutingInfo.DestAddr OUTPUT Countrie | eval Disctime=strftime('Call.DisconnectTime'/1000,"%m/%d/%Y %H:%M:%S %Q") | eval Conntime=strftime('Call.ConnectTime'/1000, "%m/%d/%Y %H:%M:%S%Q") | eval diffTime=('Call.DisconnectTime'-'Call.ConnectTime') | eval Duracion=strftime(diffTime/1000, "%M:%S") | table Countrie, Duración Spain 00:02 Spain 00:16 Argentina 00:53 Spain 09:20 Spain 02:54 Spain 28:30 Spain 01:18 Spain 00:28 Spain 16:40 Spain 00:03 Chile 00:25 Uruguay 01:54 Spain 01:54  Regards.  
This is actually quite simple question but give a good answer is extreme hard My answer or actually thinking is based on what I have learn e.g. in finance sector. I suppose that on almost all en... See more...
This is actually quite simple question but give a good answer is extreme hard My answer or actually thinking is based on what I have learn e.g. in finance sector. I suppose that on almost all enterprise grade business URL is just a start/execution point for real application. This means that when this endpoint is called it's just e.g. API Gateway request which are forwarder to one (usually several) backend(s) which are processing real request and then return needed response to client.  In tecnical point of view this means that there are a session (what user are doing in real life transaction) and this contains several request (those individual URLs) which are processing e.g. individual dashboard or step in real process. Usually there should be sessionId which are fixed for one real life transaction e.g. login into web bank and do what ever you are doing in one login (e.g. check balance, pay some invoices, transfer money etc.). Then there is requestId which is execution for one individual URL / process step (like see our account amount, check invoice, modify invoice, accept it into pay etc.). When you are think this workflow and which kind of event all those tens of subsystems are generating for click one entry point UR it's quite obviously that. you cannot define any DM which can describe this bunch of events. I suppose that you can do some DM for base audit data, but as payloads of different requests for backend systems are totally different it will be extremely hard to create generic DM for this. If/when needed you can do it by yourself, but quite probably it will be different for every customer or at least for every entry  point Just some thoughts not any real answer. r. Ismo.  
Hi I propose that you set up a test/lab system to test and document this change. There are some things which you need check and test. Are you using REST api for current users? This works different... See more...
Hi I propose that you set up a test/lab system to test and document this change. There are some things which you need check and test. Are you using REST api for current users? This works differently with SAML users How you avoid that used user accounts / userIDs didn't change or if those have change, how to migrate users private KOs users schedules something else which are depending on userID Are you need CLI access with your old LDAP users. This didn't work with SAML account or at least it needs some additional scripts or something else? Probably something else depending on which SAML idP you are using? r. Ismo
Hi I think that this is place for sub query like index=lalala source=lalala EventID=4728 AND PrimaryGroupId IN (512,516,517,518,519) AND [ search index=lalala source=lalala EventID=4720 | fields... See more...
Hi I think that this is place for sub query like index=lalala source=lalala EventID=4728 AND PrimaryGroupId IN (512,516,517,518,519) AND [ search index=lalala source=lalala EventID=4720 | fields UserName | dedup UserName | format ] In this way it first look those UserNames which has created and then that "outer" base search this those (UserName = "xxx" OR UserName = "yy"....) If you are looking for long period then maybe there is better options too. r. Ismo 
All fields should be there if those contains some values. You could debug it e.g. comment dedup away comment stats away and replace it with table Also if/when you are using verbose mode you can... See more...
All fields should be there if those contains some values. You could debug it e.g. comment dedup away comment stats away and replace it with table Also if/when you are using verbose mode you can see what values you have in Events tab. With Smart or Fast mode this tab is not available. r Ismo
One old post which do exactly what you are doing. https://community.splunk.com/t5/Getting-Data-In/sending-specific-events-to-nullqueue-using-props-amp-amp/m-p/660688
Here is the other one https://conf.splunk.com/files/2021/slides/PLA1410C.pdf
Yes, I want to take all logs but events with envoy in it. I am only using universal forwarder which I believe cant parse any data like a heavy forwarder. Am I mistaken? I made the transformation name... See more...
Yes, I want to take all logs but events with envoy in it. I am only using universal forwarder which I believe cant parse any data like a heavy forwarder. Am I mistaken? I made the transformation names unique and restarted splunk via GUI but still no discards. 
Please write all SPL etc. inside </> tags. That way those are easier to take into use. It also ensure that we can get the same SPL what you have write into your example.
Hi even you can mask that data in GUI, it didn't mean that you have really masked that data in Splunk. You must remember that after you have write it into bucket then it there and there is always a ... See more...
Hi even you can mask that data in GUI, it didn't mean that you have really masked that data in Splunk. You must remember that after you have write it into bucket then it there and there is always a way to get it out in plain text if/when you have access to GUI and can write SPL. Even you are using search time props.conf and transforms.conf. The only way is remove that data from index and reindex it again. And even the delete command is not enough if you have access to buckets on CLI level, you could get thet data back. The only way is let it go away with set frozen time enough low, then wait and then reindex it. r. Ismo
Thanks @bowesmana @yuanliu @gcusello for your help and input.   One thing I'm still missing is being able to populate values in all the fields listed. Let me explain with some screenshots for bette... See more...
Thanks @bowesmana @yuanliu @gcusello for your help and input.   One thing I'm still missing is being able to populate values in all the fields listed. Let me explain with some screenshots for better context -   The output of just corelight query shows values for the fields -   The output of just the firewall query shows values for all related fields -   However, the output of the suggested OR queries does not populate the values of many of the fields (highlighted in red)- eg (1).    eg (2).  I only see 5 values when using stats values (*) by * in the OR query (seems like it's just the fields that are common to both indices and none of others listed will be displayed?)     Any suggestions on this?   Thanks!
One way to do it is this | makeresults format=csv data="location, name location A, name A2 location B, name B1 location B, name B2 location C, name C1 location C, name C2 location C, name C3" | se... See more...
One way to do it is this | makeresults format=csv data="location, name location A, name A2 location B, name B1 location B, name B2 location C, name C1 location C, name C2 location C, name C3" | search name != "*2*" | stats count by location | append [| makeresults format=csv data="location, name location A, name A2 location B, name B1 location B, name B2 location C, name C1 location C, name C2 location C, name C3" | eval count=0 | fields location count | dedup location] | stats sum by location but as @PickleRick said, Splunk is not good with non existent values. r. Ismo
Hi bishida, I appreciate your response. Thanks for pointer to the development guide. I'll give it a try.
Hi Based on log you are running unsupported OS.   CONTROL [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2   On Windows operating systems oldest supported version is Win 2019 or W... See more...
Hi Based on log you are running unsupported OS.   CONTROL [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2   On Windows operating systems oldest supported version is Win 2019 or Win 10. r. Ismo