All Posts

inventsekar · ‎10-03-2023

Hi @nina ... to learn regex/rex, i have made lot of videos.. pls check it.. thanks. Splunk newbie learning videos, for absolute beginners: https://www.youtube.com/@SiemNewbies101/playlists

nina · ‎10-03-2023

hello, thankyou so much for responding. even though i am entirely new to splunk and trying to find my way with youre recommendation, they are very useful.

bcrypt · ‎10-03-2023

Thanks! Managed to get it parsing JSON correctly using those two functions. However, do you know if it's possible to have a field present in the URI requirement for curl? The parent search has a f...

Thanks! Managed to get it parsing JSON correctly using those two functions. However, do you know if it's possible to have a field present in the URI requirement for curl? The parent search has a field named clientIP which stores IP addresses depending on the client that made a request. Doesn't seem like it's possible to submit it as a concatenated string since it always needs to begin with https://. e.g. neither of the below work: index=api_test uri_path=exampleRequest | curl method=get uri=https://api.shodan.io/shodan/host/clientIP?key=APIKEY OR index=api_test uri_path=exampleRequest | curl method=get uri="https://api.shodan.io/shodan/host/" + clientIP +"?key=APIKEY"

jason_tqb · ‎10-03-2023

Hello guys. This is my first post here to ask for help with extracting fields from a JSON object. Below is an example of the record: {"pod":"fmd9p","time":"2023-10-03T21:49:39.31255352Z", "source":...

Hello guys. This is my first post here to ask for help with extracting fields from a JSON object. Below is an example of the record: {"pod":"fmd9p","time":"2023-10-03T21:49:39.31255352Z", "source":"/var/log/containers/fmd9p_default.log","container_id":"1ae53e1be","log": "I1003 14:49:39.312453 test_main.cc:149] trace_id=\"8aeb0\" event=\"Worker.Finish\" program_run_sec=25.1377 status=\"OK\""} How can I extract trace_id, event, program_run_sec, and status from the log section automatically by setting up a sourcetype? Is it doable? Thanks for any help and advise

PickleRick · ‎10-03-2023

No. Natively eventlog inputs, because that's what we're talking about, generate either "plain text" or xml events depending on the renderXml parameter. There is no built-in functionality to ingest ev...

No. Natively eventlog inputs, because that's what we're talking about, generate either "plain text" or xml events depending on the renderXml parameter. There is no built-in functionality to ingest eventlog data as json. At leas not natively with UF's eventlog input. You could of course try to use a third party solution like nxlog, kiwi or something like that to generate json events from eventlog (I'm not sure if those particular examples can do that though) but that's a different story and it's a bit pointless really since you have a perfectly well (ok, almost perfectly) working inputs and accompanying TA for windows eventlogs.

jwhughes58 · ‎10-03-2023

@PickleRickThanks. I've upvoted the idea.

vishwa · ‎10-03-2023

I have bunch of alerts, I received email alert, but I did not receive auto cut incident to service now How to troubleshoot this issue?????

PickleRick · ‎10-03-2023

https://ideas.splunk.com/ideas/EID-I-208 It turns out it was not my idea, I just upvoted and commented it from my old account because it had already been there when I wanted to create it

dhernandez · ‎10-03-2023

Hello, I'm working with a Splunk cluster which has two slave peers and I need to disable an index on the Cluster Master using the REST API. I've tried the usual endpoint (/servicesNS/nobody/{app}/co...

Hello, I'm working with a Splunk cluster which has two slave peers and I need to disable an index on the Cluster Master using the REST API. I've tried the usual endpoint (/servicesNS/nobody/{app}/configs/conf-indexes/{index}) as this doc says (https://docs.splunk.com/Documentation/Splunk/8.0.0/RESTREF/RESTconf#configs.2Fconf-.7Bfile.7D.2F.7Bs... ), but it doesn't seem to work on the Cluster Master. Can someone please provide me with the specific REST API endpoint I should use to disable an index on the Cluster Master? I have read the documentation https://docs.splunk.com/Documentation/Splunk/8.0.0/RESTREF/RESTcluster but there is no reference to what I need. Thank you in advance for your assistance

fatsug · ‎10-03-2023

We'll it's all a bit of magic isn't it In this case it was the seach head deployer pushing the CSV files to the seach head cluster. Though I've seen similar issues from the deployment server tryin...

We'll it's all a bit of magic isn't it In this case it was the seach head deployer pushing the CSV files to the seach head cluster. Though I've seen similar issues from the deployment server trying to push changes to the heavy forwarder layer. Sure, I guess even if the cause of the issue remains clouded in mystery, the actual problem is solved and I should accept this as the solution.

fatsug · ‎10-03-2023

Hmm, so there is no option for the forwarder to send the log in TA_windows/CIM compliant JSON format? I know XML is compatible because this is what we normally index, and there is no JSON compliance...

Hmm, so there is no option for the forwarder to send the log in TA_windows/CIM compliant JSON format? I know XML is compatible because this is what we normally index, and there is no JSON compliance? In that case, well then the "easy solution" has an even smaller chance of making it to the next family therapy session than a pickled Rick... I'll hold of on marking this a solution until the last bit of hope is gone But if I understand you correctly, even if the eventlog can be forwarded in JSON format (big if), this is not compliant with the TA for windows in the SH/IX cluster. Best regards

RG2 · ‎10-03-2023

I am trying to host Prometheus metrics on a Splunk app such that the metrics are available at `.../my_app/v1/metrics` endpoint. I am able to create a handler of type PersistentServerConnectionAppli...

I am trying to host Prometheus metrics on a Splunk app such that the metrics are available at `.../my_app/v1/metrics` endpoint. I am able to create a handler of type PersistentServerConnectionApplication and have it return Prometheus metrics. The response status, however, code = `500` and content = `Unexpected character while looking for value: '#'` Prometheus metrics do not confirm to any of the supported `output_modes` (atom | csv | json | json_cols | json_rows | raw | xml) so I get the same error irrespective of the output mode chosen. Is there a way to bypass the output check? Is there any other alternative to host a non-confirming-format output via a Splunk REST API?

fatsug · ‎10-03-2023

There's a fine line I'm trying not to cross. Yes, we are collecting logs being indexed in JSON format. No, they are not "standard" with regards to field names, order and content as we are not colle...

There's a fine line I'm trying not to cross. Yes, we are collecting logs being indexed in JSON format. No, they are not "standard" with regards to field names, order and content as we are not collecting/indexing the eventlog in conventional or forwarder based manner. The log were indexing is not CIM compliant or compatible with the Splunk TA for windows. We'd like to "adjust" incoming events in the HF layer to become "compliant". To evaluate if this is reasonable I'd need a couple of reference events of Splunk/CIM/TA compliant eventlog in JSON format. I seem to remember there being a setting for the Windows TA for UF to send eventlog in JSON format. Hence, the ask for sample of windows eventlog in JSON format. If I'm mistaken and XML is the only forwarded format, or if there is but no one willing to share a sample, we'll just have to deploy a test environment and try to generate the JSON data we need. I just thought this could be a faster solution given that someone could share some (masked is fine) events. If I'm mistaken and the UF cannot forward eventlog in JSON format, then case closed and we're done here Best regards

PickleRick · ‎10-03-2023

Firstly, your question is a bit inconsistent since those "methods" are not mutually exclusive. For example, a syslog event can be ingested on a network input on an UF or via SC4S pushed to HEC endpoi...

Firstly, your question is a bit inconsistent since those "methods" are not mutually exclusive. For example, a syslog event can be ingested on a network input on an UF or via SC4S pushed to HEC endpoint. Secondly, unless explicitly configured, splunk on its own doesn't retain metadata about the transport it got the data from (it can however be reflected to some extent in the source field value). Thirdly, apart from the metrics which splunk gathers anyway you'd have to scan through all of your events to calculate sum of their lengths which would be highly ineffective (that's why splunk accumulates license usage count as it ingests every single event so it doesn't have to do it retroactively if needed). So it's not that easy. What you already have in license usage metrics you already have, what you don't have will be hard to compute.

jwhughes58 · ‎10-03-2023

@PickleRickThanks. I was afraid of that when I couldn't find anything in the documentation. What is your idea so I can upvote it?

kimsey4701 · ‎10-03-2023

Good afternoon, Background: I found a configuration issue in one of our firewalls which I'm trying to remediate where an admin created a very broad access rule that has permitted traffic over a wid...

Good afternoon, Background: I found a configuration issue in one of our firewalls which I'm trying to remediate where an admin created a very broad access rule that has permitted traffic over a wide array of TCP/UDP ports. I started working to identify valid traffic which has used the rule, but a co-worker mentioned an easy win would be creating an ACL to block any ports which had not already been allowed through this very promiscuous rule. My problem is I know how to use the data model to identify TCP/UDP traffic which has been logged egressing through the rule, but how could I modify the search provided below so that I can get a result that displays which ports have NOT been logged? (Also bonus points if you can help me view numbers returned as ranges rather than individual numbers aka "5000-42000") Here is my current search: | tstats ,values(All_Traffic.dest_port) AS dest_port values(All_Traffic.dest_ip) AS dest_ip dc(All_Traffic.dest_ip) AS num_dest_ip dc(All_Traffic.dest_port) AS num_dest_port FROM datamodel=Network_Traffic WHERE index="firewall" AND sourcetype="traffic" AND fw_rule="horrible_rule" BY All_Traffic.dest_port | rename All_Traffic.* AS * Thank you in advance for any help that you may be able to provide!

PickleRick · ‎10-03-2023

TA_windows expects data in either "traditional" rendered text format (key=value multiline event) or an xml structure. If you want to send them another way you'll have to write your own extractions an...

TA_windows expects data in either "traditional" rendered text format (key=value multiline event) or an xml structure. If you want to send them another way you'll have to write your own extractions and make it CIM-conformant.

PickleRick · ‎10-03-2023

No, you can't use search commands in fields definition. You can create calculated fields but they are only limited to what you could normally put in an eval statement. With key-value extraction done...

No, you can't use search commands in fields definition. You can create calculated fields but they are only limited to what you could normally put in an eval statement. With key-value extraction done using regex (as you tried with _KEY_1 and _VAL_1 groups it's tricky to properly capture the data, you use the structure of the json object and you might hit they limit on key-value pairs extracted (100 by default if I remember correctly). Unfortunately Splunk has no way of telling it to start kv extraction from a given point within an event - it always tries to "consume" whole event. So it works well if the _raw field as a whole is just one big json object but can't handle cases like "json sent with a syslog header". It's a shame really and I think I even posted an idea about that on ideas.splunk.com. Worth upvoting.

Madmax · ‎10-03-2023

Hello! I'm trying to figure out a way to display a single value that calculates users who have disconnected divided by the time range based on the time picker. The original number comes from the...

Hello! I'm trying to figure out a way to display a single value that calculates users who have disconnected divided by the time range based on the time picker. The original number comes from the avg of total disconnects divided by the distinct user count. I need to divide that number by the number of days which is based on the time picker. The goal is to get the avg user disconnects per day based on time frame selected in time picker. For example if there are 100 disconnects and 10 distinct users =10, then divided by the number of days selected in picker(7) should equal 1.42 disconnects per day. I hope that makes sense. Here is my search: Index=... Host=HostName earliest=$time_tok.earliest$ latest=$time_tok.latest$ | stats count by "User ID" |search "User ID"=* |stats avg(count) That will only give me the Total disconnects divided by Distinct users, but I need that number divided by the time picker number of days and I can't get it to work. Thank you!!!

richgalloway · ‎10-03-2023

That's the nature of wildcards - they're *wild* and sometimes match more than is desired. The workaround is to tell Splunk what not to match, using the NOT operator and some other pattern, or use th...

That's the nature of wildcards - they're *wild* and sometimes match more than is desired. The workaround is to tell Splunk what not to match, using the NOT operator and some other pattern, or use the regex command to filter using a more precise regular expression. index=test control_id=AC-2* | regex control_id="AC-2[a-z]?" This query first reads all events where the control_id field starts with "AC-2". This is similar to the existing behavior. The regex command keeps only the events where the control_id field contains "AC-2" followed by an optional single letter.

All Posts

All Posts

Re: getting datasets

Re: getting datasets

Re: Extracting nested JSON from TA-WebTools curl response

How to set up an automatic way - sourcetype to format a json record

Re: Windows Eventlog in JSON format

Re: Can fromjson be used in props.conf or transforms.conf

Not receiving auto cut

Re: Can fromjson be used in props.conf or transforms.conf

How to disable an index on a Cluster Master using REST API

Re: Single csv lookup file not updating from deployer

Re: Windows Eventlog in JSON format

Splunk REST API: Bypass response format check

Re: Windows Eventlog in JSON format

Re: Splunk search showing ingest by source method

Re: Can fromjson be used in props.conf or transforms.conf

How can I use the Network_Traffic datamodel to find what has NOT egressed/logged for a firewall ACL/rule?

Re: Windows Eventlog in JSON format

Re: Can fromjson be used in props.conf or transforms.conf

How to display Avg of Users per Time Picker?

Re: splunk search, token issue

All Posts

Are you a member of the Splunk Community?

All Posts