Decode json into linked list?

Just starting out? Need help? Post your questions and find answers here.
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4749
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

Decode json into linked list?

Post by Fangbeast »

I was helped with code to grab a single record from a json url but I also need to decode an entire set of entries into a linked list.

Given the code below, how do I grab the json below it and parse into a linked list?

Code: Select all

Protected jSon, *Mem, Result

Structure BatchData
  Episode.s
  imdbRating.s
  Title.s
  imdbID.s
  Released.s
EndStructure

Global NewList Batch.BatchData()

HTTPRequestString.s = URLEncoder("http://www.omdbapi.com/?t=stargate&Plot=full&r=json&Season=1&apikey=[YourKey]")

If InitNetwork()
  *Mem = ReceiveHTTPMemory(URLEncoder(HTTPRequestString.s), #PB_HTTP_NoRedirect)
  If *Mem
    jSon = CatchJSON(#PB_Any, *Mem, MemorySize(*Mem), #PB_JSON_NoCase)
    If jSon
      ExtractJSONStructure(JSONValue(jSon), *Result, BatchData)
      ForEach Batch()
        Debug Batch()\Episode
      Next Batch()
      FreeJSON(jSon)
      Result = #True
    EndIf
    FreeMemory(*Mem)
  EndIf
EndIf

{
"Season" : "1",
"Episodes" : [
{
"Episode" : "1",
"imdbRating": "8.3",
"Title" : "Children of the Gods",
"imdbID" : "tt0234794",
"Released" : "1997-07-27"
},
{
"Episode" : "2",
"imdbRating": "7.8",
"Title" : "The Enemy Within",
"imdbID" : "tt0709185",
"Released" : "1997-08-01"
},
{
"Episode" : "3",
"imdbRating": "6.2",
"Title" : "Emancipation",
"imdbID" : "tt0709075",
"Released" : "1997-08-08"
},
{
"Episode" : "4",
"imdbRating": "7.2",
"Title" : "The Broca Divide",
"imdbID" : "tt0709181",
"Released" : "1997-08-15"
},
{
"Episode" : "5",
"imdbRating": "6.8",
"Title" : "The First Commandment",
"imdbID" : "tt0709188",
"Released" : "1997-08-22"
},
{
"Episode" : "6",
"imdbRating": "7.6",
"Title" : "Cold Lazarus",
"imdbID" : "tt0709059",
"Released" : "1997-08-29"
},
{
"Episode" : "7",
"imdbRating": "8.2",
"Title" : "The Nox",
"imdbID" : "tt0709194",
"Released" : "1997-09-12"
},
{
"Episode" : "8",
"imdbRating": "7.5",
"Title" : "Brief Candle",
"imdbID" : "tt0709052",
"Released" : "1997-09-19"
},
{
"Episode" : "9",
"imdbRating": "8.2",
"Title" : "Thor's Hammer",
"imdbID" : "tt0709209",
"Released" : "1997-09-26"
},
{
"Episode" : "10",
"imdbRating": "8.4",
"Title" : "The Torment of Tantalus",
"imdbID" : "tt0709205",
"Released" : "1997-10-03"
},
{
"Episode" : "11",
"imdbRating": "7.4",
"Title" : "Bloodlines",
"imdbID" : "tt0709051",
"Released" : "1997-10-10"
},
{
"Episode" : "12",
"imdbRating": "7.2",
"Title" : "Fire and Water",
"imdbID" : "tt0709091",
"Released" : "1997-10-17"
},
{
"Episode" : "13",
"imdbRating": "7.6",
"Title" : "Hathor",
"imdbID" : "tt0709101",
"Released" : "1997-10-24"
},
{
"Episode" : "14",
"imdbRating": "7.8",
"Title" : "Singularity",
"imdbID" : "tt0709172",
"Released" : "1997-10-31"
},
{
"Episode" : "15",
"imdbRating": "7.3",
"Title" : "Cor-ai",
"imdbID" : "tt0709061",
"Released" : "1998-01-23"
},
{
"Episode" : "16",
"imdbRating": "8.2",
"Title" : "Enigma",
"imdbID" : "tt0709079",
"Released" : "1998-01-30"
},
{
"Episode" : "17",
"imdbRating": "8.3",
"Title" : "Solitudes",
"imdbID" : "tt0709175",
"Released" : "1998-02-06"
},
{
"Episode" : "18",
"imdbRating": "7.8",
"Title" : "Tin Man",
"imdbID" : "tt0709212",
"Released" : "1998-02-13"
},
{
"Episode" : "19",
"imdbRating": "8.8",
"Title" : "There But for the Grace of God",
"imdbID" : "tt0709207",
"Released" : "1998-02-20"
},
{
"Episode" : "20",
"imdbRating": "6.3",
"Title" : "Politics",
"imdbID" : "tt0709142",
"Released" : "1998-02-27"
},
{
"Episode" : "21",
"imdbRating": "8.7",
"Title" : "Within the Serpent's Grasp",
"imdbID" : "tt0709219",
"Released" : "1998-03-06"
}
],
"Response" : "True",
"totalSeasons": "10",
"Title" : "Stargate SG-1"
}
Amateur Radio, D-STAR/VK3HAF
User avatar
Bisonte
Addict
Addict
Posts: 1232
Joined: Tue Oct 09, 2007 2:15 am

Re: Decode json into linked list?

Post by Bisonte »

It's a little more entangled than you think.
The list is IN the structure with this result.

So the thing could look like this:

Code: Select all

Structure s_episodeData
  Episode.s
  imdbRating.s
  Title.s
  imdbID.s
  Released.s
EndStructure

Structure s_jsonEpisodesResult
  Season.s
  List Episodes.s_episodeData()
  Response.s
  totalSeasons.s
  Title.s
EndStructure

Procedure.s GetJSONResultString(Url.s)
  
  Protected *Mem, Result.s = ""
  
  If InitNetwork()
    *Mem = ReceiveHTTPMemory(Url, #PB_HTTP_NoRedirect)
    If *Mem
      Result = PeekS(*Mem, -1, #PB_UTF8|#PB_ByteLength)
      FreeMemory(*Mem)
    EndIf
  EndIf
  
  ProcedureReturn Result
  
EndProcedure

Procedure.i GetEpisodeResult(Url.s, *Var.s_jsonEpisodesResult)
  
  Protected jSon = ParseJSON(#PB_Any, GetJSONResultString(Url), #PB_JSON_NoCase)
  Protected Result = #False
  
  If jSon
    ExtractJSONStructure(JSONValue(jSon), *Var, s_jsonEpisodesResult)
    Result = #True
    FreeJSON(jSon)
  EndIf
  
  ProcedureReturn Result
  
EndProcedure

Define SG1.s_jsonEpisodesResult
Define Url.s = URLEncoder("http://www.omdbapi.com/?t=stargate&Plot=full&r=json&Season=1&apikey=[YourKey]")

If GetEpisodeResult(Url, @SG1)
  
  ForEach SG1\Episodes()
    Debug "S01E"+ RSet(SG1\Episodes()\Episode, 2, "0") + " - " + SG1\Episodes()\Title  
  Next
  
EndIf
Since I don't have an omdb key, I couldn't try it out directly.
PureBasic 6.10 LTS (Windows x86/x64) | Windows10 Pro x64 | Asus TUF X570 Gaming Plus | R9 5900X | 64GB RAM | GeForce RTX 3080 TI iChill X4 | HAF XF Evo | build by vannicom​​
English is not my native language... (I often use DeepL to translate my texts.)
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4749
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

Re: Decode json into linked list?

Post by Fangbeast »

Bisonte wrote:It's a little more entangled than you think.
The list is IN the structure with this result.

So the thing could look like this:

Code: Select all

Structure s_episodeData
  Episode.s
  imdbRating.s
  Title.s
  imdbID.s
  Released.s
EndStructure

Structure s_jsonEpisodesResult
  Season.s
  List Episodes.s_episodeData()
  Response.s
  totalSeasons.s
  Title.s
EndStructure

Procedure.s GetJSONResultString(Url.s)
  
  Protected *Mem, Result.s = ""
  
  If InitNetwork()
    *Mem = ReceiveHTTPMemory(Url, #PB_HTTP_NoRedirect)
    If *Mem
      Result = PeekS(*Mem, -1, #PB_UTF8|#PB_ByteLength)
      FreeMemory(*Mem)
    EndIf
  EndIf
  
  ProcedureReturn Result
  
EndProcedure

Procedure.i GetEpisodeResult(Url.s, *Var.s_jsonEpisodesResult)
  
  Protected jSon = ParseJSON(#PB_Any, GetJSONResultString(Url), #PB_JSON_NoCase)
  Protected Result = #False
  
  If jSon
    ExtractJSONStructure(JSONValue(jSon), *Var, s_jsonEpisodesResult)
    Result = #True
    FreeJSON(jSon)
  EndIf
  
  ProcedureReturn Result
  
EndProcedure

Define SG1.s_jsonEpisodesResult
Define Url.s = URLEncoder("http://www.omdbapi.com/?t=stargate&Plot=full&r=json&Season=1&apikey=[YourKey]")

If GetEpisodeResult(Url, @SG1)
  
  ForEach SG1\Episodes()
    Debug "S01E"+ RSet(SG1\Episodes()\Episode, 2, "0") + " - " + SG1\Episodes()\Title  
  Next
  
EndIf
Since I don't have an omdb key, I couldn't try it out directly.
That's fair enough. Since I have a key and am dumber than a donley's hind leg when it comes to json, cml, pointers et al, I am elected to do the assembling and trying out!!

Thanks for the help Bisonte.
Amateur Radio, D-STAR/VK3HAF
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4749
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

Re: Decode json into linked list?

Post by Fangbeast »

Bisonte, your code works but I can't get it working in my larger program Bugger!

My code looks similar to this in my OMDBAPI program. Is the problem that I call things in procedures and pointers are involved? (I have a lot more stuff in my procedure of course)

Your code is in an include called : _BS_GetJsonSeries.pbi and works as-is when I uncomment the lines in it.

Code: Select all

Structure EpisodeData
  Episode.s
  imdbRating.s
  Title.s
  imdbID.s
  Released.s
EndStructure

Structure EpisodesResult
  Season.s
  List Episodes.EpisodeData()
  Response.s
  totalSeasons.s
  Title.s
EndStructure

Procedure.s GetJSONResultString(Url.s)
  Protected *Mem, Result.s = ""
  If InitNetwork()
    *Mem = ReceiveHTTPMemory(Url, #PB_HTTP_NoRedirect)
    If *Mem
      Result = PeekS(*Mem, -1, #PB_UTF8|#PB_ByteLength)
      FreeMemory(*Mem)
    EndIf
  EndIf
  ProcedureReturn Result
EndProcedure

Procedure.i GetEpisodeResult(Url.s, *Var.EpisodesResult)
  Protected jSon = ParseJSON(#PB_Any, GetJSONResultString(Url), #PB_JSON_NoCase)
  Protected Result = #False
  If jSon
    ExtractJSONStructure(JSONValue(jSon), *Var, EpisodesResult)
    Result = #True
    FreeJSON(jSon)
  EndIf
  Debug Str(Result) + " ::: "  + #PB_Compiler_Procedure
  ProcedureReturn Result
EndProcedure

Global SeriesInfo.EpisodesResult

;Define Url.s = URLEncoder("http://www.omdbapi.com/?t=stargate&Plot=full&r=json&Season=1&apikey=[yourkey]")

;If GetEpisodeResult(Url, @SeriesInfo)
;  ForEach SeriesInfo\Episodes()
;    Debug "S01E"+ RSet(SeriesInfo\Episodes()\Episode, 2, "0") + " - " + SeriesInfo\Episodes()\Title 
;  Next
;EndIf

My code is in this file: SearchForSeries.pbi and works when I use the previous code but only to fetch te string. WIth your code replacing it at line 153, nothing gets returned.

; Search the IMDB for all episodes in a series or one episode only

Code: Select all

Procedure SearchForSeries()
  
  ClearGadgetItems(#Gadget_Batchform_Batchlist)                                                                                   ; 
    
    SearchText.s    = GetGadgetText(#Gadget_Batchform_Search)                                                                     ; 
    
    If Left(SearchText.s, 2) = "tt"                                                                                               ; Probably an IMDB id   
      
      SearchOption.s = SearchText.s                                                                                               ; 
      
      SearchText.s  = #Empty$                                                                                                     ; 
      
    EndIf                                                                                                                         ; 
  
    SeasonFlag.i    = GetGadgetState(#Gadget_Batchform_cbSeason)                                                                  ; 
    
    SeasonNumber.s  = GetGadgetText(#Gadget_Batchform_sSeason)                                                                    ; 
    
    EpisodeFlag.i   = GetGadgetState(#Gadget_Batchform_cbEpisode)                                                                 ; 
    
    EpisodeNumber.s = GetGadgetText(#Gadget_Batchform_sEpisode)                                                                   ; 
    
    If SearchText.s <> #Empty$  Or SearchOption.s <> #Empty$                                                                      ; One or other strings have to be populated
      
      If FindString(SearchText.s, "/", 1, #PB_String_NoCase) <> #NoStringFound                                                    ; See if there is a title AND a year to search for
        
        TitleField.s = StringField(SearchText.s, 1, "/")                                                                          ; Extract the title from the search string
        
        YearField.s  = StringField(SearchText.s, 2, "/")                                                                          ; Extract the year field from the search string
        
      EndIf                                                                                                                       ; 
      
      If FileSize(Program\BatchFilename) <> #FileNotFound                                                                         ; Delete the previously fetchted json file if any
        
        DeleteFile(Program\BatchFilename)                                                                                         ; Probably won't need this any more
        
      EndIf                                                                                                                       ; 
      
      HTTPRequestString.s     =       Program\HTTPHeader                                                                          ; Base address          "http://www.omdbapi.com/?"  : debug ""  + HTTPRequestString.s
      
      If SearchText.s <> #Empty$                                                                                                  ; 
        
        If TitleField.s <> #Empty$                                                                                                ; 
          
          HTTPRequestString.s   +       Omdb\Title  + TitleField.s                                                                ; Search for a title    "&t=star trek"
          
          HTTPRequestString.s   + "&" + Omdb\Year   + YearField.s                                                                 ; 
          
        ElseIf TitleField.s = #Empty$                                                                                             ; 
          
          HTTPRequestString.s   +       Omdb\Title  + SearchText.s                                                                ; Search for a title    "&t=star trek"
          
        EndIf                                                                                                                     ;
        
      Else                                                                                                                        ; 
        
        If SearchOption.s <> #Empty$                                                                                              ; 
          
          HTTPRequestString.s   +       Omdb\Imdb   + SearchOption.s                                                              ; Search for an IMDB id "&t=tt0929629"
          
        EndIf                                                                                                                     ; 
        
      EndIf                                                                                                                       ; 
      
      HTTPRequestString.s     + "&" + Omdb\Plot     + "full"                                                                      ; Return the full plot  "&Plot=full"
      
      HTTPRequestString.s     + "&" + Omdb\Type     + "json"                                                                      ; Return data as a JSON file
      
      If SeasonFlag.i  = #True                                                                                                    ; 
        
        HTTPRequestString.s     + "&" + Omdb\Season.s + SeasonNumber.s                                                            ; 

      EndIf                                                                                                                       ; 
      
      If EpisodeFlag.i  = #True                                                                                                   ; 
        
        HTTPRequestString.s     + "&" + Omdb\Episode.s + EpisodeNumber.s                                                           ; 

      EndIf                                                                                                                       ; 
      
      HTTPRequestString.s     + "&" + Omdb\Apikey   + Program\OMDBAPIkey.s                                                        ; Insert your OMDB API key. Must not give this out!
      
      HTTPRequestString.s     = URLEncoder(HTTPRequestString.s)                                                                   ; Encode the URL to avoid problems with some browsers
      
      ;Debug HTTPRequestString.s
      
      If GetEpisodeResult(HTTPRequestString.s, @SeriesInfo)
        
        ForEach SeriesInfo\Episodes()
          
          Debug "S01E"+ RSet(SeriesInfo\Episodes()\Episode, 2, "0") + " - " + SeriesInfo\Episodes()\Title 
          
        Next
        
      EndIf
      
    ;If GetEpisodeResult(Url, @SeriesInfo)
    ;  ForEach SeriesInfo\Episodes()
    ;    Debug "S01E"+ RSet(SeriesInfo\Episodes()\Episode, 2, "0") + " - " + SeriesInfo\Episodes()\Title 
    ;  Next
    ;EndIf
      
    Else                                                                                                                          ; 
      
      SetGadgetText(#Gadget_Batchform_Statusbar, "Search string or search option was empty so we can do nothing")                 ; 
      
      SetInfoBarArea("Headings", "Error", "Search string or search option was empty so we can do nothing", #PB_Compiler_Procedure); 
  
    EndIf                                                                                                                         ; 
    
EndProcedure

Last edited by Fangbeast on Sun May 26, 2019 1:49 am, edited 1 time in total.
Amateur Radio, D-STAR/VK3HAF
infratec
Always Here
Always Here
Posts: 6866
Joined: Sun Sep 07, 2008 12:45 pm
Location: Germany

Re: Decode json into linked list?

Post by infratec »

Is
in your HTTPRequestString :?:
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4749
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

Re: Decode json into linked list?

Post by Fangbeast »

infratec wrote:Is
in your HTTPRequestString :?:
Yes, there is. I use a structure to return all the components the query needs from gadget selections.

if I remove this section and debug the returned Debug HTTPRequestString.s, the full string is there and is returned by my usual code.

I think my problem is that this bit of code is referencing pointers in the wrong order (That's where I always have trouble) and therefore get no results. If I use Bisonte's code as he provided it, in the order he presented it, it works. Even with his original code words changed to be more 'Englishy'.

This part in my own procedure returns nothing as if there is nothing in the returned json structure.

Code: Select all

      If GetEpisodeResult(HTTPRequestString.s, @SeriesInfo)
        ForEach SeriesInfo\Episodes()
          Debug "S01E"+ RSet(SeriesInfo\Episodes()\Episode, 2, "0") + " - " + SeriesInfo\Episodes()\Title 
        Next
      EndIf
Amateur Radio, D-STAR/VK3HAF
User avatar
RSBasic
Moderator
Moderator
Posts: 1218
Joined: Thu Dec 31, 2009 11:05 pm
Location: Gernsbach (Germany)
Contact:

Re: Decode json into linked list?

Post by RSBasic »

[ot]
Fangbeast wrote:Posts: 4444
Congratulations to this number. :D
[/ot]
Image
Image
User avatar
Mijikai
Addict
Addict
Posts: 1360
Joined: Sun Sep 11, 2016 2:17 pm

Re: Decode json into linked list?

Post by Mijikai »

I did something with List() & JSON recently maybe it helps:
viewtopic.php?p=536072#p536072
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4749
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

Re: Decode json into linked list?

Post by Fangbeast »

RSBasic wrote:[ot]
Fangbeast wrote:Posts: 4444
Congratulations to this number. :D
[/ot]
it means I am dumber than everyone else so have to post more questions:):)
Amateur Radio, D-STAR/VK3HAF
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4749
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

Re: Decode json into linked list?

Post by Fangbeast »

Mijikai wrote:I did something with List() & JSON recently maybe it helps:
viewtopic.php?p=536072#p536072
Thanks, but I don't see how it helps me.

Code: Select all

If GetEpisodeResult(InternetAddress, @SeriesInfo)
 ForEach SeriesInfo\Episodes()
   Debug "S01E"+ RSet(SeriesInfo\Episodes()\Episode, 2, "0") + " - " + SeriesInfo\Episodes()\Title 
 Next
EndIf
This part works when it is OUTSIDE a procedure but not when it is in it. I've been playing with passing the address *Var to it in the procedure heading but getting nowhere because I don't know what I am doing.
Amateur Radio, D-STAR/VK3HAF
User avatar
Bisonte
Addict
Addict
Posts: 1232
Joined: Tue Oct 09, 2007 2:15 am

Re: Decode json into linked list?

Post by Bisonte »

I think your "HTTPRequestString.s" contains wrong data.

If my example works in "single" mode...

Debug your HTTPRequestString to see if its right... maybe this is the fault.
PureBasic 6.10 LTS (Windows x86/x64) | Windows10 Pro x64 | Asus TUF X570 Gaming Plus | R9 5900X | 64GB RAM | GeForce RTX 3080 TI iChill X4 | HAF XF Evo | build by vannicom​​
English is not my native language... (I often use DeepL to translate my texts.)
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4749
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

Re: Decode json into linked list?

Post by Fangbeast »

Bisonte wrote:I think your "HTTPRequestString.s" contains wrong data.

If my example works in "single" mode...

Debug your HTTPRequestString to see if its right... maybe this is the fault.
With api keys removed:

Your string
My string
I can't see any difference. Must be something I have done in the project somewhere. The moment I put the lines below inside the procedure, it returns nothing.

Code: Select all

     If GetEpisodeResult(URLEncoder(HTTPRequestString.s), @OMDBAPI)
        ForEach OMDBAPI\Episodes()
          Debug "S01E"+ RSet(OMDBAPI\Episodes()\Episode, 2, "0") + " - " + OMDBAPI\Episodes()\Title 
        Next
      EndIf
You had this line

Code: Select all

Define OMDBAPI.s_jsonEpisodesResult
Which I had to turn into

Code: Select all

Global OMDBAPI.s_jsonEpisodesResult
else it would complain about the structure not being defined but that was the only change I made. Probably the problem is that it isn't reaching the structure members? Something I have to pass to my procedure perhaps?

Would you have time to look at the project some day?
Amateur Radio, D-STAR/VK3HAF
User avatar
Bisonte
Addict
Addict
Posts: 1232
Joined: Tue Oct 09, 2007 2:15 am

Re: Decode json into linked list?

Post by Bisonte »

Fangbeast wrote:Would you have time to look at the project some day?
Ok. Then I would need your current code, because in your last code you post, these functions are not yet included.
PureBasic 6.10 LTS (Windows x86/x64) | Windows10 Pro x64 | Asus TUF X570 Gaming Plus | R9 5900X | 64GB RAM | GeForce RTX 3080 TI iChill X4 | HAF XF Evo | build by vannicom​​
English is not my native language... (I often use DeepL to translate my texts.)
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4749
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

Re: Decode json into linked list?

Post by Fangbeast »

Bisonte wrote:
Fangbeast wrote:Would you have time to look at the project some day?
Ok. Then I would need your current code, because in your last code you post, these functions are not yet included.
Hopefully think link works. DropBox can be inconsistent. If you run this code for the first time, it will ask to store your OMDBAPI key in the database permanently.

https://www.dropbox.com/s/detdb8maqg7hy ... te.7z?dl=0
Amateur Radio, D-STAR/VK3HAF
User avatar
Bisonte
Addict
Addict
Posts: 1232
Joined: Tue Oct 09, 2007 2:15 am

Re: Decode json into linked list?

Post by Bisonte »

I got it... and after nearly an hour of searching ....
Where you are defining the variable omdbapi ? I can't find it....
PureBasic 6.10 LTS (Windows x86/x64) | Windows10 Pro x64 | Asus TUF X570 Gaming Plus | R9 5900X | 64GB RAM | GeForce RTX 3080 TI iChill X4 | HAF XF Evo | build by vannicom​​
English is not my native language... (I often use DeepL to translate my texts.)
Post Reply