All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Ended up going with this that works pretty good. [My Search] | timechart span=$span$d dc(userSesnId) as count_of_user_sessions | trendline sma$sma$(count_of_user_sessions) as "Moving Average" |... See more...
Ended up going with this that works pretty good. [My Search] | timechart span=$span$d dc(userSesnId) as count_of_user_sessions | trendline sma$sma$(count_of_user_sessions) as "Moving Average" | rename count_of_user_sessions AS "Disctinct Count of User Sessions"
Hi All, I have a main search where name1 filed will have multiple values I need to run sub search based on the value of name1.  The structure goes like this: mail_search which has name1=a sub... See more...
Hi All, I have a main search where name1 filed will have multiple values I need to run sub search based on the value of name1.  The structure goes like this: mail_search which has name1=a sub search if name1=a   then run search1 if name1=b   then run search2 I have tried this with the following code:     | makeresults | eval name1="a" | eval condition=case(name1="a", "index=_internal | head 1 | eval val=\"Query for a1\" | table val", name1="b", "index=_internal | head 1 | eval val=\"Query for b\" | table val", 1=1, "search index=_internal | head 1 | eval val=\"Default query\" | table val") |table condition | map search=$condition$     I am getting the following error Unable to run query '"index=_internal | head 1 | eval val=\"Query for a1\" | table val"'.
It's exactly like @PickleRick said. There is no way to ensure that you can do that all in one phase without data leaks. Especially when you didn't know where and which kind of data you will receive f... See more...
It's exactly like @PickleRick said. There is no way to ensure that you can do that all in one phase without data leaks. Especially when you didn't know where and which kind of data you will receive from source systems. Then only way try to get this working even somehow it strict data onboarding and change management process with separate dev/test environment where all data and changes are first integrated. Then you need some way to analyze that data and ensure that there haven't been any data leaks which contains e.g. SSN, IBAN, or other secure PII data. And as it said the format options for those are actually countless (believe or not, but you will see it). After you are absolutely sure that your masking etc. is working then you can do production integration. But you will see that earlier or later someone will make some "emergency" change or something and then you have those events in your production After when this has happened you have unmasked data on Splunk indexes and there is no way to mask it in search time so that no-one cannot see it. Event using delete command is not enough as those events are still on buckets and if you have access to your storage layer you can get that data out there. Then only way is delete the index ensure that those are overwritten enough many times and then reingest that data. r. Ismo
Apart from what @gcusello and @PickleRick have said (which I agree with), this "problem" should be fixed at source. Your organisation should fully justify why sensitive data such as credit card numbe... See more...
Apart from what @gcusello and @PickleRick have said (which I agree with), this "problem" should be fixed at source. Your organisation should fully justify why sensitive data such as credit card numbers and social security numbers are being saved unobfuscated in logs for Splunk to ingest in the first place. Ideally, all such data should be stored in a secure place which is only retrievable by a few trusted people who access it for legitimate reasons. Anything else, is possibly a violation of your customer privacy protection rights.
OK. Several things here. 1. For a question starting with "what is the best way", especially if no boundary conditions are given, the answer is usually "it depends". 2. From my experience - the wors... See more...
OK. Several things here. 1. For a question starting with "what is the best way", especially if no boundary conditions are given, the answer is usually "it depends". 2. From my experience - the worse problem definition - the less reliable outcome. I've dealt with customers who wanted something just "configured so it works" (we're not necessarily talking about Splunk, just a general idea) and the result was usually less than stellar. Your problem is rooted in the compliance but it's also equally common in DLP areas - just find something. We don't know what/where/if it is but we want you to find it. While for some types of identifiers you can distinguish them because they are in a particular format _and_ they have some internal integrity which you can check (like IBAN numbers has control digits), others do not have it and there is either a fat chance of false positives or false negatives, depending on how creative you are with finding - for example - all those possible ways of writing a phone number. And don't even get me started on trying to find names or addresses. Of course, you can try to "use AI" to guess what ad where constitutes sensitive data but this will only add another layer to already excruciating headache. Even a human, having a relatively good understanding of a context, could make mistakes here now and then. So even without getting into the gory technical details of how to implement such stuff with/around Splunk, I'd say if you want to do something like that without proper data classification and well-defined things to filter/mask you're in for a treat - a neverending project of tweaking your detection engine and dealing with stakeholders' complaints about false positives and negatives.
Hi @Richy_s , to mask sensitive data, you can follow the instructions at https://docs.splunk.com/Documentation/Splunk/9.4.0/Data/Anonymizedata The main issue, if I correctly understood, is to ident... See more...
Hi @Richy_s , to mask sensitive data, you can follow the instructions at https://docs.splunk.com/Documentation/Splunk/9.4.0/Data/Anonymizedata The main issue, if I correctly understood, is to identify PII and sensitive information in your data. The best approach, in myexperiesce, is to ingest data in a temporary index (so you can delete it when you will end the analysis) and identify all the data and the regexes to filter them, then you have to apply these filters using the approach in the below link. I don't understand what you mean whan you say "the props need to be dynamic": filter rules must be defined and used, if you have new rules, you have to add them. Ciao. Giuseppe
Hi, I have a requirement to mask any sensitive data, such as credit card numbers or Social Security Numbers, that might be ingested into Splunk. I can write the props to handle data masking, but the... See more...
Hi, I have a requirement to mask any sensitive data, such as credit card numbers or Social Security Numbers, that might be ingested into Splunk. I can write the props to handle data masking, but the challenge is that I do not know where or if the sensitive data will appear. Although the data we currently have doesn't contain any sensitive information, compliance mandates require us to implement controls that detect and mask such data before it is ingested into Splunk. Essentially, the props need to be dynamic. Is there a way to achieve this?   Thanks.
If I create a role test_role which inherits from the can_delete role, it will show in configtracker - I created a test_role inheriting from can_delete. But if I now create a test2_role which inherits... See more...
If I create a role test_role which inherits from the can_delete role, it will show in configtracker - I created a test_role inheriting from can_delete. But if I now create a test2_role which inherits from test_role, the configtracker entry will only show that it inherits from test_role but it won't show the underlying inheritance from can_delete.
What you are meaning with this "It will also not work if your inheritance is nested." ?
_configtracker index is currently excellent place to found those changes. It logging those even those are done when splunk is down. Just add changes when splunk is starting. Unfortunately there could... See more...
_configtracker index is currently excellent place to found those changes. It logging those even those are done when splunk is down. Just add changes when splunk is starting. Unfortunately there could be some differences on SCP side? At least earlier it didn't log all SCP platform changes or at least even sc_admin cannot see those, but I suppose that Splunk own SREs can see also those.
It will also not work if your inheritance is nested. Unfortunately, there is no good way of tracking all inheritances for a role except for listing effective capabilities for a given user.
When you are replacing :: in _meta fields then receiving splunk instance don't recognize it any more as _meta data. And if there is no those mandatory meta fields then splunk cannot guess those and do... See more...
When you are replacing :: in _meta fields then receiving splunk instance don't recognize it any more as _meta data. And if there is no those mandatory meta fields then splunk cannot guess those and do what is needed for those events. Then based on receiver side configuration this data goes to default index or it will dropped.
I'm assuming you're receiving this on SC4S. So as you've changed the format of sent data, the receiving end probably doesn't know what to do with that. First thing to check would be to sniff the tra... See more...
I'm assuming you're receiving this on SC4S. So as you've changed the format of sent data, the receiving end probably doesn't know what to do with that. First thing to check would be to sniff the traffic to see whether the data is being sent and what it looks like.
I checked it, but unfortunately it does not seem to work. Now I can't seem to find logs that contain any metadata, so I assume they are being dropped due to some problem. Where should I look for cl... See more...
I checked it, but unfortunately it does not seem to work. Now I can't seem to find logs that contain any metadata, so I assume they are being dropped due to some problem. Where should I look for clues?
Valid point. No its quite limited and logs only GUI edits (still could be useful in cloud). Just tested editing authorize.conf in system and via new and existing app. Everything can be seen in _confi... See more...
Valid point. No its quite limited and logs only GUI edits (still could be useful in cloud). Just tested editing authorize.conf in system and via new and existing app. Everything can be seen in _configtracker with following simple search index="_configtracker" data.path="*authorize.conf" GUI Edits to existing role are as expected logged under configtracker.
1. You're posting this one in a "Getting data in" section with "HEC" and "scripted input" labels. Are you sure it's really about getting data into your Splunk? 2. What kind of script are you talking... See more...
1. You're posting this one in a "Getting data in" section with "HEC" and "scripted input" labels. Are you sure it's really about getting data into your Splunk? 2. What kind of script are you talking about? A JS code in your browser? A script on the Search Head? Something else? 3. Are you aware of the security implications?
As a side note - pretty much every solution involving Windows and third party syslog breaks stuff somewhere. Either breaks Splunk parsing or breaks the third party parsing. At some point something i... See more...
As a side note - pretty much every solution involving Windows and third party syslog breaks stuff somewhere. Either breaks Splunk parsing or breaks the third party parsing. At some point something is almost sure to break.
Hello, I have a Windows machine with UF that sends its logs to a HF, which has the SC4S derived config loaded (see the opening entry's link). That allows to reformat the logs that passed through t... See more...
Hello, I have a Windows machine with UF that sends its logs to a HF, which has the SC4S derived config loaded (see the opening entry's link). That allows to reformat the logs that passed through the HF to IETF 5424 syslog (with framing enabled) and forward them to a syslog instance. That reformatting pretty much alters most parts of the original message. In the output you will generally see the first half of the message (not counting the SDATA part) will contain the metadata fields in the key::value format. I would like to change that in the syslog output generated by the config on the HF node.
OK. So this is not about Splunk's metadata format as much as rendering it for export. I suppose you can tweak it a little. The key part here is this transform [metadata_meta] SOURCE_KEY = _meta RE... See more...
OK. So this is not about Splunk's metadata format as much as rendering it for export. I suppose you can tweak it a little. The key part here is this transform [metadata_meta] SOURCE_KEY = _meta REGEX = (?ims)(.*) FORMAT = ~~~SM~~~$1~~~EM~~~$0 DEST_KEY = _raw It's being called as the first one (except for the one manipulating routing) and it exports whole _meta as-is. So you need to change it to: [sanitize_metadata] INGEST_EVAL = escaped_meta=replace(_meta,"::","=") [metadata_meta] SOURCE_KEY = escaped_meta REGEX = (?ims)(.*) FORMAT = ~~~SM~~~$1~~~EM~~~$0 DEST_KEY = _raw And of course adjust props to call the sanitize_metadata first TRANSFORMS-zza-syslog = syslog_canforward, sanitize_metadata, metadata_meta, metadata_source, metadata_sourcetype, metadata_index, metadata_host, metadata_subsecond, metadata_time, syslog_prefix, syslog_drop_zero
What you are referring to is the syslog serialized data or SDATA (see RFC 5424) portion of the message. That consists of only 5 values (same as the Splunk JSON envelope's 5 top-level fields). And yes... See more...
What you are referring to is the syslog serialized data or SDATA (see RFC 5424) portion of the message. That consists of only 5 values (same as the Splunk JSON envelope's 5 top-level fields). And yes, those use the equals sign as a separator. On the other hand the main part of the message will look like this: ~~~SM~~~env::env01~~~EM~~~11/29/2024 02:01:55 PM\nLogName=Security\nEventCode=4624\nEventType=0\nComputerName=DESKTOP-OOU0O6E\nSourceName=Microsoft Windows security auditing.\nType=Information\nRecordNumber=49513\nKeywords=Audit Success\nTaskCategory=Logon\nOpCode=Info\nMessage=An account was successfully logged on.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tNT AUTHORITY\\SYSTEM\r\n\tAccount Name:\t\tDESKTOP-OOU0O6E$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3E7\r\n\r\nLogon Information:\r\n\tLogon Type:\t\t5\r\n\tRestricted Admin Mode:\t-\r\n\tVirtual Account:\t\tNo\r\n\tElevated Token:\t\tYes\r\n\r\nImpersonation Level:\t\tImpersonation\r\n\r\nNew Logon:\r\n\tSecurity ID:\t\tNT AUTHORITY\\SYSTEM\r\n\tAccount Name:\t\tSYSTEM\r\n\tAccount Domain:\t\tNT AUTHORITY\r\n\tLogon ID:\t\t0x3E7\r\n\tLinked Logon ID:\t\t0x0\r\n\tNetwork Account Name:\t-\r\n\tNetwork Account Domain:\t-\r\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x2d4\r\n\tProcess Name:\t\tC:\\Windows\\System32\\services.exe\r\n\r\nNetwork Information:\r\n\tWorkstation Name:\t-\r\n\tSource Network Address:\t-\r\n\tSource Port:\t\t-\r\n\r\nDetailed Authentication Information:\r\n\tLogon Process:\t\tAdvapi \r\n\tAuthentication Package:\tNegotiate\r\n\tTransited Services:\t-\r\n\tPackage Name (NTLM only):\t-\r\n\tKey Length:\t\t0\r\n\r\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\r\n\r\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\r\n\r\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\r\n\r\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\r\n\r\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\r\n\r\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\r\n\r\nThe authentication information fields provide detailed information about this specific logon request.\r\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\r\n\t- Transited services indicate which intermediate services have participated in this logon request.\r\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\r\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested. I would like to have the first part of the syslog message to have the metadata as env=env01 or env:env01. As I understand the SC4S derived config allows you to modify most parts of the message. But is it possible for the metadata part too? If yes, how do I match to the metadata key-value pairs?