Getting Data In

How to override and change incoming source and host names with JSON source and host fields?

Path Finder

I have JSON fields for source and host which I would like to use to override the incoming source and host. What is the easiest mechanism to accomplish this?

Thanks

Tags (3)
0 Karma
1 Solution

Path Finder

I actually created a calculated field for source and set it to the corresponding value from JSON. This seems to work correctly.

View solution in original post

0 Karma

Path Finder

I actually created a calculated field for source and set it to the corresponding value from JSON. This seems to work correctly.

View solution in original post

0 Karma

Revered Legend

Could you post some sample value of raw data (or from "your base search | head 5 | table _raw" search query)?

0 Karma

Path Finder

,"SvctagSegmentGrp":{
"ModelNumber":"DZ","ServiceLevel":"CP","TechNameIssuedCall":"STC: SOME TECH","DspReplyCode":"0000","TechBadgeNumIssuedCall":659662,"ReasonCode":"SW3F","TechIdIssuedCall":"000251779","Buid":"000000707","ItemSubClass":"002","Svctag":"B375ST1","DpsNum":"000175479487","ProductDesc":"7737,NOTEBOOK,HADLEY 17FBTX","LineOfBusiness":55}
}
}
}
}
}

(Completed message)

0 Karma

Path Finder

,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"FTF USED. ARTICLE NUMBER: non compliance","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Audibly obtained PN","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"No APN","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"VDI Told cust about Refurb replacement parts","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"VPKRS: No need for plastic replacement","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Media Check. Customer has: None","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Provided TAT: 1-2 BD","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"CRU/FRU part/s verified thru DTT/EducateDell","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Provided Moxie as a lifeline","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Sent summary email during call. Customer","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"confirmed e-mail is received.","CommentDate":20141031}
(again TBC)

0 Karma

Path Finder

,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"SYSTEM: Inspiron 17","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"BTTR: Cx declined follow-up","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Consulted: CM POC Ice Bordeos","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Heat Check: SAT","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Captured, verified and updated Customer Name and","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Contact info in SR header.","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Captured, keyspelled and updated email address","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"in SR header.","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"VA TOADE","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Cx agreed to 'One-Strike policy'","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Provided Ownership Spiel/s","CommentDate":20141031}
(again TBC)

0 Karma

Path Finder

,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"D:Media Request","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"cx called in bec he would like to reinstall the","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"OS however he don't have the Media Recovery disc,","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"cx requesting for it, he would like to reinstall","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"the OS bec there's a lot of stuff or apps on","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"system and cx would like to refresh or to clean","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"it up, he wants to remove all the files and data.","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"inform cx the this request is a one time deal","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"only, cx understand, inform also cx the warranty","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"details. done CDO - set to prio 4","CommentDate":20141031}
(again TBC)

0 Karma

Path Finder

Got it. I am working a chicken and egg problem, so I need to get back to DNA.
Here is an example message:
{
"Timestamp":"2014-11-03 14:34:55",
"Type":"INFO ",
"Class":"TopicListenner:?",
"LogEntry":
{
"source":"AMDPS",
"Content": {
"Amdps1204ipmCpy":{
"Dps1204Ipm":{
"CustomerSegmentGrp":{
"CustomerNumBuid":"000000707","CustomerNum":"7777777","DpsType":"EXG","CompanyNum":"02","RequestingService":"Safder Memon"}
,"OrderSegmentGrp":{
"ExgOrderNum":"000000000490644069","DomsStatusDate":20141103,"DspStatusCode":"0000","DomsStatusCode":"SC","PoNum":38045618}
,"PartsSegmentGrp":{
"PartsSegment":[{
"SkuMfgNbr":"89HT1","QuantityAtFsb":"0000000000","PartDescription":"KIT,MEDIA,DVD,RDVD,7737","Quantity":"000000001"}
,{
"SkuMfgNbr":"CY2KJ","QuantityAtFsb":"0000000000","PartDescription":"KIT,SW,W8H/P64,MUL24","Quantity":"000000001"}
]}
,"FsbLocationGrp":{
"FsbLocationSegment":[{
"VendorId":"0016","VendorInfo":[{
"MilesToFsb":"0007","FsbLocation":"WYOW1"}
,{
"MilesToFsb":"0092","FsbLocation":"WYUL1"}
,{
"MilesToFsb":"0092","FsbLocation":"WYGK1"}
]}
,"","",""]}
,"JobSegmentGrp":{
"JobIndx":0}
,"ContactSegmentGrp":{
"TelephoneNbr":5555555555,"ContactName":"NOONE SPECIAL","PhoneExtension":"00000"}
,"HeaderSegmentGrp":{
"CreationDate":"Mon Nov 03 14:34:54 CST 2014","OperationType":"TRN"}
,"CommentSegmentGrp":{
"CommentSegment":[{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"P:Media Request","CommentDate":20141031}
(to be continued)

0 Karma

Path Finder

,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Gave service request number to customer.","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"THIS IS AN ALABANG DISPATCH","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"S:dps media","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"STC: SOME TECH","DetailProblemDesc":"Created from DellServ Case #999999999","CommentDate":20141031}
,{
"CommentType":"EXG","TechName":"ETSB SOME OTHERTECH","DetailProblemDesc":"PART OK","CommentDate":20141101}
,{
"CommentType":"EXG","TechName":"ETSB SOME OTHERTECH","DetailProblemDesc":"RS APPROVED DPS","CommentDate":20141101}
]}
,"RaSegmentGrp":{
"RaFlag":"N"}
,"AddressSegmentGrp":{
"TimeZone":"EST","EdiRegionCode":"CA","StreetTwo":"Suite 3333","State":"ON","ZipCode":"999 999","City":"Gloucester","CountryCode":"CA","StreetOne":"9999 Duck St"}
,"StatusSegmentGrp":{
"StatusDate":"20141103:14:34:54.495440","RecordType":"C","StatusCode":"TTC"}
,"TranhistSegmentGrp":{
"EventDate":"20141103:14:34:54.448741","VendorNum":"0016","VendorDate":"20141104:16:00","EventCode":"VX","EventComments":"RTN WAYBILL#: KAM001231827"}
(again TBC)

0 Karma

Path Finder

Thanks. The issue is that I do not have a regex to populate the fields. They are automatically extracted in the data input because I am using JSON. I really just want to set the value of host and source using the extracted JSON field name if possible.

0 Karma

Motivator

Except that by the time they are automatically extracted from your JSON you can't overwrite those fields anymore. At the point in the process where host and source get assigned, none of the fields have yet been extracted - it's just a big string. By the time the JSON parser gets hold of your input, it is too late, those fields have already been assigned. That's why you need to do it ahead of time, via a regex in props.conf/transforms.conf, as @somesoni2 has pointed out.

0 Karma

Revered Legend

You can find the example here on how to use props.conf and transforms.conf to override Host and source (same way). Just replace the regex with you regex to find the host/source field from JSON.

http://answers.splunk.com/answers/1673/hostname-rename-using-transforms.html

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!