Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Join the Conversation
Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
- About GaetanVP
GaetanVP
Contributor
Member since:
01-29-2021
03-01-2024
Community Statistics
| Posts | 106 |
| Solutions | 3 |
| Karma Given | 254 |
| Karma Received | 23 |
| Member Since | 01-29-2021 |
Activity Feed
- Got Karma for Re: splunk show-decrypted command usage. 02-20-2025 10:31 AM
- Got Karma for Re: Permission error after upgrade during restart. 01-07-2025 07:30 AM
- Got Karma for splunk show-decrypted command usage. 09-12-2024 03:34 AM
- Got Karma for Re: UF | Too much permissions given to Splunk when configured as a systemd service. 07-09-2024 11:12 AM
- Got Karma for Re: splunk show-decrypted command usage. 06-28-2024 12:56 PM
- Got Karma for Is this possible to simply extract the content of a journal.zst file?. 10-25-2023 11:28 AM
- Got Karma for Re: Index time extraction multiple regex match. 09-21-2023 06:05 AM
- Posted Re: Index time extraction multiple regex match on Splunk Enterprise. 09-21-2023 05:41 AM
- Karma Re: Index time extraction multiple regex match for richgalloway. 09-21-2023 05:40 AM
- Posted How to extract Index time in multiple regex match? on Splunk Enterprise. 09-19-2023 04:30 AM
- Tagged How to extract Index time in multiple regex match? on Splunk Enterprise. 09-19-2023 04:30 AM
- Posted How to Monitor Linux ssh connection with "super user"? on Getting Data In. 09-11-2023 07:12 AM
- Karma Re: Extract a threshold from lookup and create a default value for gcusello. 09-05-2023 07:58 AM
- Posted Re: How to fetch the start and the End Time from row logs on Dashboards & Visualizations. 08-31-2023 08:25 AM
- Posted Re: splunk show-decrypted command usage on Security. 08-31-2023 06:41 AM
- Karma Re: Universal forwarder 9.1.0 (linux) changed owner for PickleRick. 08-31-2023 06:13 AM
- Karma Re: Universal forwarder 9.1.0 (linux) changed owner for auradk. 08-31-2023 06:13 AM
- Posted splunk show-decrypted command usage on Security. 08-28-2023 02:26 AM
- Karma Re: Extract the data after second special character for ITWhisperer. 08-17-2023 05:45 AM
- Posted Re: Why is Json Data extraction props.conf not working? on Dashboards & Visualizations. 08-17-2023 02:12 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 1 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
07-10-2023
12:54 AM
Hello @Tao_Zeng, You can try the following : | makeresults
| eval TEST="\n User-Agent: iOS/16.4.1 iPhone\n P-Access-Network-Info: 3GPP-NR-TDD;utran-cell-id-3gpp=4600101200e020432103\n Security-Verify: ipsec-3gpp;alg=hmac-md5-96;ealg=null;mod=trans;port-c=9950;port-s=9900;prot=esp;spi-c=2155781586;spi-s=4286488018\n"
| rex field=TEST "P-Access-Network-Info: (?<KeyValue>.+?)\\\n"
| table KeyValue Thanks @yeahnah for the triple backslash, I didn't know that... I would have put only two // in order to escape the / itself but then the n is a literal character so I do not really understand... Regards, GaetanVP
... View more
- Tags:
- rex
07-10-2023
12:53 AM
Hello @Tao_Zeng, You can try the following : | makeresults
| eval TEST="\n User-Agent: iOS/16.4.1 iPhone\n P-Access-Network-Info: 3GPP-NR-TDD;utran-cell-id-3gpp=4600101200e020432103\n Security-Verify: ipsec-3gpp;alg=hmac-md5-96;ealg=null;mod=trans;port-c=9950;port-s=9900;prot=esp;spi-c=2155781586;spi-s=4286488018\n"
| rex field=TEST "P-Access-Network-Info: (?<KeyValue>.+?)\\\n"
| table KeyValue Thanks @yeahnah for the triple backslash, I didn't know that... I would have put only two // in order to escape the / itself but then the n is a literal character so I do not really understand... Regards, GaetanVP
... View more
07-10-2023
12:27 AM
Hello @kitkit321, You could try something like this (if you are connected on your Search Head, otherwise you'll need to replace https://localhost with the url of the Search Head you want to target). curl -k -u admin:"<your_password>" https://localhost:8089/services/search/jobs -d search="| metadata type=sourcetypes index=<your_index> | table sourcetype"
curl -k -u admin:"<your_password>" https://localhost:8089/services/services/search/jobs/<sid_returned_on_previous_command>/results Hope it helps ! GaetanVP
... View more
07-07-2023
10:56 AM
1 Karma
Hello @adikrhd, I suppose that we don't exactly have the same structure of data, could you give a screenshot of your data (basically when you just search your logs without any transformations) ? Mine looks like this based on your first message I suspect that your data is not correclty ingested or maybe you have the "Event 1: " before each JSON, which we didn't take into account in all the answers on this topic I think... Keep us updated so we can find a fix! GaetanVP
... View more
07-07-2023
08:37 AM
1 Karma
Hello @adikrhd, You can try something like this <<your_search_that_give_the_big_json>>
| table details.*
| rename details.* AS "*" ```details.keysN will become keyN```
| untable _name, value ```reverse the table```
| dedup value
| sort value
| table _name, value Hope it helps ! GaetanVP
... View more
07-07-2023
08:07 AM
Hello @av_ You can try to open your classic dashboard and edit the source (you will see a big xml file) Inside the <panel></panel> tag, you can add something like this (it's basically CSS lines) <html>
<!-- main background of the dashboard -->
<style>
.dashboard-body {
background-color: #b4c519
}
</style>
<!-- background of the panel -->
<style>
.dashboard-panel {
background-color: #217727
}
</style>
<!-- background of the titel panel section -->
<style>
.panel-title {
background-color: #c51951
}
</style>
</html> Keep in mind that you can simply inspect the results of the dashboard with your browser and find the HTML classes of section you want to modify and add other CSS lines to the xml. Hope it will help ! GaetanVP
... View more
07-07-2023
02:16 AM
Hello @dural_yyz, here are some updates ! So the above approach didn't work as expected... The thing is after I changed the sourcetype of my events, they will directly go to the indexing phase, I do not think it's possible to "tell them to go back to parsing queue" in order to apply a CLONE_SOURCETYPE afterwards... So I tried something else to clone and forward (to another Splunk HF) only a subset of my events... (nasty but for now I have no other options...). The main idea is to change the _TCP_ROUTING value for the event I want to clone, those events will be selected with the REGEX and directly forward to the wanted destination. Afterwards I will clean the events that were not changed by my REGEX (with an "inverse" REGEX) by sending them to null queue. Here is an example with a JSON incoming raw data (I assigned them the sourcetype "mysourcetype") : {
"Foo": "Bar",
"Hello": "World"
},
{
"Foo": "Bar",
"Hello": "Again"
} props.conf [mysourcetype]
TRANSFORMS-foo-clone = trans-clone
[mysourcetype:cloned]
TRANSFORMS-bar-drop = trans-drop transforms.conf # clone all events and change tcp output for specific events mathcing the regex
[trans-clone]
CLONE_SOURCETYPE = mysourcetype:cloned
REGEX = "Hello":\s*"World"
DEST_KEY = _TCP_ROUTING
FORMAT = my_specific_output
# drop duplicated and not forwarded logs
[trans-drop]
REGEX = (?s)^(?!.*"Workload"\s*:\s*"Aip").*$
DEST_KEY = queue
FORMAT = nullQueue I tested it and do the job for me ! I do not like the fact to use two "opposite REGEX" because of resources usage and if your REGEX is not okay, you will end up with duplicated unwanted data. Thanks you for your time and @isoutamo for the good hint about dropping some data 😋 GaetanVP
... View more
07-05-2023
01:33 AM
Hello @PickleRick ! I am a bit afraid to contradict an Ultra Champion but still, let's continue the discussion ! 😋 1) I agree with this first point 2) To mee, the _TCP_ROUTING will be changed on the cloned events, not on the original one Here is a test I made (on a standalone Splunk Enterprise test VM) : props.conf [mysourcetype]
TRANSFORMS-foo-clone = bar-clone transforms.conf [bar-clone]
CLONE_SOURCETYPE = mysourcetype:new
REGEX = "Hello":\s*"World"
DEST_KEY = MetaData:Host
FORMAT = host1 Here is the sample of data that I upload to Splunk (via GUI) and assign the "mysourcetype" sourcetype {
"Tom":"Holland",
"Hello":"World",
} At the end here is the result : As you can see, it is the host of the duplicated event (with the "mysourcetype:new") that has been changed, and not the original one. Let me know if I missed something here ! Thanks for your time, GaetanVP
... View more
07-05-2023
01:08 AM
Hello @dural_yyz thanks for you answer ! My main purpose was to clone only a subset of events (based on regex). From what I read, the CLONE_SOURCETYPE spec will automatically clone ALL EVENTS from the the sourcetype given (correct me if I'm wong). So my strategy will be to use your answer to first override the sourcetype of a subset of original event (the events that will match the REGEX field) and then use the CLONE_SOURCETYPE only on the new sourcetype. That should do the job... Thanks for your time, GaetanVP
... View more
07-04-2023
08:04 AM
Hello @PickleRick thanks for your answer, here some stuff I wanted to discuss about ! To be precise, you're not sending the cloned events to external destination. Are you sure about that point ? I am pretty sure I am sending the cloned events since I assigned them a new sourcetype and apply them some anonymization by using the new props/transforms mechanism. Also when I checked the logs in my external destination, I can confirm that the sourcetype I see is "mynewsrctype". Also, as @isoutamo already pointed out, you should be using s2s for sending events from one splunk instance to another (especially that you're using HF so you'll be sending parsed events and you'll save some CPU time at destination machine at cost of increased transfer bandwidth). Of course you can send the data raw but it doesn't make much sense because you have to parse it again (but I can think of a use case where that's actually a desirable thing). Makes sense, I didn't think about the increase of transfer bandwidth, this is very interesting. As you correctly guessed, I prefer to send cooked data ! Thanks for your time, GaetanVP
... View more
07-04-2023
07:35 AM
Hello @isoutamo, I still do not understand something... In my Heavy Forwarder's outputs.conf it clearly specified sendCookedData = false Based on this doc : https://docs.splunk.com/Documentation/Splunk/9.1.0/Admin/Outputsconf I should not received any HeartBeat on my receiver But I was not even sure that this HB logs was really for HeartBeat... When I try to use splunktcp inputs, I am back to the following issue that I am currently trying to solve. https://community.splunk.com/t5/Getting-Data-In/Why-am-I-receiving-quot-splunk-cooked-mode-v3-quot-data-from-a/m-p/173548 Thanks for your time, GaetanVP
... View more
07-04-2023
07:26 AM
1 Karma
Hello @gcusello, sorry for the late reply, Just for information the problem was linked to a bad outputs.conf I put on my Indexers. As you know, having issues with outgoing traffic would impact the data flow in a way that tcpout queue would fill up, that was the case. Thanks, GaetanVP
... View more
07-04-2023
07:14 AM
Hello Splunkers, I am using the official "Palo Alto Networks Add-on for Splunk" in order to ingest Palo logs inside my Splunk infra. My path is basically Panorama --> HF --> Indexers. I am wondering what will happen if my HF goes down during a certain amount of time ? Does the Panorama instance have a temporary outputs queue that will prevent data loss ? What could I do to make this flow of log more "resilient" ? Thanks a lot, GaetanVP
... View more
Labels
07-03-2023
08:03 AM
Hello Splunkers, I would like to change the value of frozenTimePeriodInSecs for one of my existing indexes. What should I be careful of ? Juste change the value on my Master Node and push the new bundle to my Indexers ? Also since my frozenTimePeriodInSecs will be lower than some of my bucket indexes, those events will be automatically rolled to frozen ? Thanks a lot ! GaetanVP
... View more
Labels
- Labels:
-
indexer
06-29-2023
05:55 AM
Hello Splunkers,
Do you know if I can forward cooked data from my HF1 to my HF2 ? (I tried from one HF to a Standalone Splunk instance, but never from HF to HF).
I am wondering if there is something to set on the HF2 to tell "do not try to parse any incoming data for this input"
Thanks a lot,
GaetanVP
... View more
Labels
06-29-2023
05:43 AM
Hello @isoutamo, Yes but with my props.conf file, I need to specify a sourcetype (or a host, or source, or rule...) in the [<spec>] stanza, based on this page : https://docs.splunk.com/Documentation/Splunk/9.0.5/Admin/Propsconf The thing is, I want to clone only a part of the logs which sourcetype is "sourcetypeA" and I do not know how I can say "clone only a part of those logs base on key/value foo=bar (or a regex)". Is it possible to clone only a part of the logs incoming based on certain rules ? That would be very helpful... Thanks, GaetanVP
... View more
- Tags:
- Hello @is
06-29-2023
02:04 AM
Hello Splunkers,
I have a question, would it be possible to assign a specific sourcetype to some logs inside a input stanza, depending on the content of the log itself (based on the key / fields extracted or some regex...).
For instance :
[monitor:///whatever]
if foo = bar
sourcetype = scr_type_1
else
sourcetype = scr_type_1
I have few hope about this one... Thanks a lot, GaetanVP
... View more
Labels
06-15-2023
05:53 AM
Hello @isoutamo and thanks a lot for your answer. Would you have some documentation on how to implement a s2s connection ? I mean is it only the fact to use "splunktcp" in inputs.conf instead of "tcp" stanza ? Or there is other configuration to consider ? Regards, GaetanVP
... View more
06-02-2023
05:22 AM
Hello ! As @ITWhisperer suggested, you can use his complementary cron expression, but I think you will need to duplicate your first report. Should do the work, good luck !
... View more
06-02-2023
03:13 AM
Hello Splunkers,
Here is my use-case : I am cloning some events that arrive to my Heavy Forwarder and then forward those cloned event to another Splunk (standalone - free trial) machine.
I am able to receive the logs on my targeted machine with the following configuration :
On my HF (sender / forwarder)
transforms.conf
[srctype-clone] CLONE_SOURCETYPE = mynewsrctype REGEX = .* DEST_KEY = _TCP_ROUTING FORMAT = tcp_output_conf
outputs.conf
[tcpout:tcp_output_conf] server = <ip>:<port> sendCookedData = false
On my Splunk standalone machine (receiver)
inputs.conf
[tcp://15601] disabled = false index = whatever_index sourcetype = mynewsrctype
Based on that I have some questions...
- First, I am receiving the logs but also some unwanted logs containing only "HB", does it correspond to Heart Beat ? Why do I received that ? - Should I use splunktcp instead of tcp on my receiver ? - Should I use enableS2SHeartbeat = true on my receiver ? - Should I use sendCookedData = true on my sender ?
Thanks a lot for your help !
Regards, GaetanVP
... View more
Labels
05-17-2023
06:32 AM
Hello Splunkers, I am facing a problem with my indexers that are not able to index anymore. Neither the data forwarder to those indexers, neither the internal Splunk logs... I even tried to index data (simple txt file) directly from the indexer GUI, I do not get any error but my selected indexe will not be filled/updated.
Any clue what I can do to troubleshoot ? There is nothing in splunkd.log file, what other logs should I check?
Regards, GaetanVP
... View more
Labels
- Labels:
-
indexer
01-27-2023
05:25 AM
Hello Splunkers,
Here is my use-case : I am monitoring apache logs on 3 different VMs, one VM for each env : dev, uat, prod
I do not see the point to create a specific index for each env (no security / restrictions needed). But I still want to be able to distinguish the logs by environment.
What would be the best practice to do that ? Create a tag / event type ? for each host from where the logs are coming ?
Regards, GaetanVP
... View more
Labels
- Labels:
-
indexer
01-27-2023
05:02 AM
Please do not excuse ! It's working like a charm now 👍 Thanks, GaetanVP
... View more
01-27-2023
02:50 AM
Hello @PickleRick, Haha you're right... My LB is now the SPOF ! I have never thought about that... 🤔 I will definitely take a look at Splunk Connect for Syslog, I have never tried it for now Thanks again for your help, GaetanVP
... View more
01-27-2023
02:46 AM
Hello @PickleRick, That's a nice trick indeed but I would not be able to use my DS anymore to deploy inputs.conf on both HF - since forwarder1 and forwarder2 are "hardcoded". If preferer to keep a generic inputs.conf (being deployed by DS) in my specific app location and have a custom file in /etc/system/local for each HF. Or maybe I am missing something ! But your proposition is great and can fit a lot of other situations thanks 👍
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
03-01-2024
11:05 AM
|
Karma given to
