It is currently Sun Mar 07, 2021 11:10 pm

All times are UTC + 1 hour




Post new topic Reply to topic  [ 27 posts ]  Go to page Previous  1, 2
Author Message
 Post subject:
PostPosted: Thu Dec 09, 2004 12:36 pm 
Offline
User
User
User avatar

Joined: Sun Oct 19, 2003 11:21 am
Posts: 13
Location: www
First, thanks a lot for this great lib! It works perfectly.

But...

Could someone be so kind to explain how to use it to download a file to memory?

urlfile\savepath=*mem or @string

Is it possible?

_________________
pIII@1Ghz, wXP, 512MB, pb3.93full


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Dec 11, 2004 3:59 pm 
Offline
PureBasic Expert
PureBasic Expert

Joined: Fri Apr 25, 2003 6:41 pm
Posts: 1364
ok, i change my code (note: This code is *NOT* the source of the getURL-Lib. At the moment i must say: DON'T USE THE LIB IT!)

Follow this do my code better than the GetURL-Source:

* abort Download is possible

* Can handle error-messages
GetURL don't return anything, when a 404 (file not found) happend. And getURL-detect can detect a 404, where no one is. And when a error happend, that geturl don't know, the startet thred of the lib run eternaly.

* Download more than one file at the same time
(ok, it is possible with my code, but it can be a little bit complicate to do this...)

* Read and interpretate the date of a Internet file

* HTTP-Get command header is analysed
GetURL need a HTTP-Head-Command first and then he cut the Get-Header (and in bad cases, the header aren't equal; theoretical). My code only send only a get-command and analyse the header.

* support "splitted" header
When the buffer length is small and the header is big, geturl-lib can't handle this. My code can.

* Proxy without password and Web-User/Proxy-User Name+Password supported.
The Source above of the GetURL-Lib can't do this (with PB3.92), because of a bad call of Base64Encoder().

I have tested proxy (without password) and side with authorization. But i can't test it with a proxy with password.
It would be nice, when somebody can test this.

Code:
;-Create and Read file with API
Structure API_FileHandle
  FHandle.l
  Buffer.l
  BufferLen.l
  ReadPos.l
  DataInBuffer.l
  readed.l
EndStructure

Procedure API_CloseFile(*File.API_FileHandle)
  If *File\FHandle<>#INVALID_HANDLE_VALUE
    CloseHandle_(*File\FHandle)
  EndIf
  If *File\Buffer
    FreeMemory(*File\Buffer):*File\Buffer=0
  EndIf
EndProcedure
Procedure API_FileCreate(*File.API_FileHandle,File$)
  *File\FHandle=CreateFile_(File$,#GENERIC_WRITE   ,0,0,#CREATE_ALWAYS   ,#FILE_ATTRIBUTE_NORMAL,0)
  If *File\FHandle=#INVALID_HANDLE_VALUE
    ProcedureReturn #False
  Else
    ProcedureReturn #True
  EndIf
EndProcedure
Procedure API_WriteData(*File.API_FileHandle,*Buffer,len)
  If WriteFile_(*File\FHandle,*Buffer,len,@written,0)
    ProcedureReturn written
  Else
    ProcedureReturn 0
  EndIf
EndProcedure

;-Help

Procedure.s MyStringField(a$,Index,Seperator$)
  AkPos=0
  LenSeperator=Len(Seperator$)
  LenA=Len(a$)
  For i=1 To Index
    a=FindString(a$,Seperator$,AkPos+LenSeperator)
    If a>0
      If AkPos>0
        StartPos=AkPos+LenSeperator
      Else
        StartPos=1
      EndIf
      AkPos=a
    ElseIf AkPos<LenA
      If AkPos>0
        StartPos=AkPos+LenSeperator
      Else
        StartPos=1
      EndIf
      AkPos=LenA+1
    Else
      i=Index
      AkPos=0
      StartPos=0
    EndIf
  Next
 
  If StartPos
    ProcedureReturn Mid(a$,StartPos,AkPos-StartPos)
  Else
    ProcedureReturn ""
  EndIf
EndProcedure

;-Structures and Constants

Structure HTTPGetId
  url$
  server$
  File$
  FileName$
  Port.l
  wenc$
  ProxyServer$
  ProxyPort.l
  penc$
  cookie$
  location$
  ConnectionID.l
  ErrorMessage$
  FileSize.l
  Date.l
  URLError.l
  Buffer.l
  BufferLength.l
  Realm$
 
  InHeader.l
  FirstLine.l
  currentline$
  lastchar.l
  eol.l
  Received.l
  Typ$
 
  FileHandle.API_FileHandle
EndStructure
#URLError_OK=200
#URLError_CantCreateFile=1
#URLERROR_OutOfMemory=2


;-URL-Commands
;{Day,Month
Dim wkday$(6)
Dim weekday$(6)
Dim Month$(12)
wkday$(1)="MON"
wkday$(2)="TUE"
wkday$(3)="WED"
wkday$(4)="THU"
wkday$(5)="FRI"
wkday$(6)="SAT"
wkday$(0)="SUN"
weekday$(1)="MONDAY"
weekday$(2)="TUESDAY"
weekday$(3)="WEDNESDAY"
weekday$(4)="THURSDAY"
weekday$(5)="FRIDAY"
weekday$(6)="SATURDAY"
weekday$(0)="SUNDAY"
Month$(1)="JAN"
Month$(2)="FEB"
Month$(3)="MAR"
Month$(4)="APR"
Month$(5)="MAY"
Month$(6)="JUN"
Month$(7)="JUL"
Month$(8)="AUG"
Month$(9)="SEP"
Month$(10)="OCT"
Month$(11)="NOV"
Month$(12)="DEC"
;}
Procedure.s HTTP_CreateDate(Date)
  ProcedureReturn FormatDate(wkday$(DayOfWeek(Date))+" %dd "+Month$(Month(Date))+" %yyyy %hh:%ii:%ss GMT",Date)
EndProcedure
Procedure HTTP_AnalyseDate(TestTime$)
  TestTime$=ReplaceString(Trim(UCase(TestTime$)),"  "," ")
  Day=0:Month$="":Year=0:Time$=""
  For i=0 To 6
    If Left(TestTime$,4)=wkday$(i)+","                        ;{"rfc1123-Date"
      Day=Val(StringField(TestTime$,2," "))
      Month$=StringField(TestTime$,3," ")
      Year=Val(StringField(TestTime$,4," "))
      Time$=StringField(TestTime$,5," ")
      Break
      ;}
    ElseIf Left(TestTime$,Len(weekday$(i))+1)=weekday$(i)+"," ;{"rfc850-Date"
      SubTime$=StringField(TestTime$,2," ")
      Day=Val(StringField(SubTime$,1,"-"))
      Month$=StringField(SubTime$,2,"-")
      Year=Val(StringField(SubTime$,3,"-"))
      If Year>80:Year+1900:Else:Year+2000:EndIf
      Time$=StringField(TestTime$,3," ")
      Break
      ;}
    ElseIf Left(TestTime$,4)=wkday$(i)+" "                    ;{"asctime-Date"
      Day=Val(StringField(TestTime$,3," "))
      Month$=StringField(TestTime$,2," ")
      Year=Val(StringField(TestTime$,5," "))
      Time$=StringField(TestTime$,4," ")
      Break
      ;}
    EndIf
  Next
  For i=1 To 12
    If Month$(i)=Month$ : Month=i:Break : EndIf
  Next
  Date=ParseDate("%hh:%ii:%ss",Time$)
  Hour=Hour(Date)
  Min=Minute(Date)
  Sec=Second(Date)
  ProcedureReturn Date(Year,Month,Day,Hour,Min,Sec)
EndProcedure 
Procedure.s HTTP_CryptedUserPass(user$,pass$)
  If user$
    conc$=user$+":"+pass$
    OutputBuffer = AllocateMemory(Len(conc$)*2)
    Base64Encoder(conc$,Len(conc$),OutputBuffer,Len(conc$)*2)
    penc$=PeekS(OutputBuffer)
    FreeMemory(OutputBuffer)
  Else
    penc$=""
  EndIf
  ProcedureReturn penc$
EndProcedure

Declare HTTP_ReConnect(*Handle.HTTPGetId)
Declare HTTP_ChangeURL(*Handle.HTTPGetId,url$)
Declare HTTP_CloseURL(*Handle.HTTPGetId)

;internal
Procedure.s HTTP_CreateRequestString(*Handle.HTTPGetId)
  If *Handle\ProxyServer$
    com$=UCase(*Handle\Typ$)+" "+*Handle\url$+" HTTP/1.0"+Chr(13)+Chr(10)
  Else
    com$=UCase(*Handle\Typ$)+" "+*Handle\File$+" HTTP/1.0"+Chr(13)+Chr(10)
  EndIf
  com$+"Host: "+*Handle\server$+Chr(13)+Chr(10)
  com$+"Accept: */*"+Chr(13)+Chr(10)
 
  com$+"User-Agent: PBGPIHTTPs"+Chr(13)+Chr(10)
  If *Handle\ProxyServer$<>"" And *Handle\penc$
    com$+"Proxy-Authorization: Basic "+*Handle\penc$+Chr(13)+Chr(10)
  EndIf
  If *Handle\wenc$<>""
    com$+"Authorization: Basic "+*Handle\wenc$+Chr(13)+Chr(10)
  EndIf
  If *Handle\cookie$<>""
    com$+"Cookie: "+*Handle\cookie$+Chr(13)+Chr(10)
  EndIf
  If *Handle\location$<>""
    com$+"Location: "+*Handle\location$+Chr(13)+Chr(10)
  EndIf
  com$+Chr(13)+Chr(10)
  Debug com$
  ProcedureReturn com$
EndProcedure
;Get Information
Procedure HTTP_IsHeaderReceived(*Handle.HTTPGetId)
  ProcedureReturn #True-*Handle\InHeader
EndProcedure
Procedure HTTP_GetProgress(*Handle.HTTPGetId); Header must be received!
  If *Handle\FileSize
    ProcedureReturn *Handle\Received*100/*Handle\FileSize
  EndIf
EndProcedure
Procedure HTTP_GetError(*Handle.HTTPGetId); Header must be received!
  ProcedureReturn *Handle\URLError
EndProcedure
Procedure.s HTTP_GetErrorMessage(*Handle.HTTPGetId); Header must be received!
  ProcedureReturn *Handle\ErrorMessage$
EndProcedure
Procedure.s HTTP_GetNewLocation(*Handle.HTTPGetId); Header must be received!
  ProcedureReturn *Handle\location$
EndProcedure
Procedure.s HTTP_GetAuthenticateRealm(*Handle.HTTPGetId); Header must be received!
  ProcedureReturn *Handle\Realm$
EndProcedure
Procedure HTTP_GetFileSize(*Handle.HTTPGetId); Header must be received!
  ProcedureReturn *Handle\FileSize
EndProcedure
Procedure HTTP_GetFileDate(*Handle.HTTPGetId); Header must be received!
  ProcedureReturn *Handle\Date
EndProcedure
;Change Settings
Procedure HTTP_ChangeURL(*Handle.HTTPGetId,url$)
  ;Port finden
  s_start=FindString(url$,":",7)
  s_end=FindString(url$,"/",s_start)
  If s_start And s_end
    port$=Mid(url$,s_start+1,s_end-(s_start+1))
  EndIf
  ;Servername finden
  For a=1 To Len(url$)
    s_start=FindString(url$,"//",1)
    s_end=FindString(url$,"/",s_start+2)
    server$=Mid(url$,s_start+2,s_end-(s_start+2))
    If port$<>""
      server$=ReplaceString(server$,":"+port$,"",1)
    EndIf
  Next
  ;standard-Name
  If Port=0: Port=80 :EndIf
  ;Dateiname
  File$=ReplaceString(url$,"http://"+server$,"",1)
 
  *Handle\url$=url$
  *Handle\Port=Port
  *Handle\server$=server$
  *Handle\File$=File$
EndProcedure
Procedure HTTP_ChangeWWWAuthenticate(*Handle.HTTPGetId,wenc$)
  *Handle\wenc$=wenc$
EndProcedure
Procedure HTTP_ChangeProxyAuthenticate(*Handle.HTTPGetId,penc$)
  *Handle\penc$=penc$
EndProcedure
;Commands
Procedure HTTP_Connect_(*Handle.HTTPGetId,LocalFile$,url$,wenc$,ProxyServer$,ProxyPort,penc$,BufferLength,Typ$);internal
  If BufferLength<=0: BufferLength=10240 :EndIf
 
  HTTP_ChangeURL(*Handle.HTTPGetId,url$)
 
  If ProxyPort=0: ProxyPort=3196 :EndIf
 
  ;{ Infos speichern
  *Handle\FileName$=LocalFile$
  *Handle\wenc$=wenc$
  *Handle\ProxyServer$=ProxyServer$
  *Handle\ProxyPort=ProxyPort
  *Handle\penc$=penc$
  *Handle\cookie$=""
  *Handle\Buffer=0
  *Handle\BufferLength=BufferLength
  *Handle\Typ$=Typ$
  ;}
 
  ProcedureReturn HTTP_ReConnect(*Handle)
EndProcedure
Procedure HTTP_OpenUrl(*Handle.HTTPGetId,LocalFile$,url$,wenc$,ProxyServer$,ProxyPort,penc$,BufferLength)
  ProcedureReturn HTTP_Connect_(*Handle.HTTPGetId,LocalFile$,url$,wenc$,ProxyServer$,ProxyPort,penc$,BufferLength,"GET")
EndProcedure
Procedure HTTP_UrlInfo(*Handle.HTTPGetId,url$,wenc$,ProxyServer$,ProxyPort,penc$,BufferLength)
  ProcedureReturn HTTP_Connect_(*Handle.HTTPGetId,"",url$,wenc$,ProxyServer$,ProxyPort,penc$,BufferLength,"HEAD")
EndProcedure
Procedure HTTP_ReConnect(*Handle.HTTPGetId); Usefull, when url is moved or authenticate is needed
  HTTP_CloseURL(*Handle)
 
  *Handle\location$=""
  *Handle\FileSize=0
  *Handle\Date=0
  *Handle\ErrorMessage$=""
  *Handle\InHeader=#True
  *Handle\FirstLine=#True
  *Handle\currentline$=""
  *Handle\lastchar=0
  *Handle\eol=0
  *Handle\URLError=0
  *Handle\Realm$=""
  *Handle\Received=0
  Debug *Handle\server$
  Debug *Handle\Port
  If *Handle\ProxyServer$<>""
    *Handle\ConnectionID = OpenNetworkConnection(*Handle\ProxyServer$, *Handle\ProxyPort)
  Else
    *Handle\ConnectionID = OpenNetworkConnection(*Handle\server$, *Handle\Port)
  EndIf
 
  If *Handle\ConnectionID
    com$=HTTP_CreateRequestString(*Handle)
    SendNetworkData(*Handle\ConnectionID,@com$,Len(com$))
    ProcedureReturn #True
  Else
    ProcedureReturn #False
  EndIf
EndProcedure
Procedure HTTP_CloseURL(*Handle.HTTPGetId); little note: don't earse file-info (HTTP_GET*)
  If *Handle\ConnectionID
    CloseNetworkConnection(*Handle\ConnectionID)
    *Handle\ConnectionID=#False
  EndIf
  If *Handle\Buffer
    FreeMemory(*Handle\Buffer)
    *Handle\Buffer=0
  EndIf
  API_CloseFile(*Handle\FileHandle)
EndProcedure
Procedure HTTP_ReceiveData(*Handle.HTTPGetId) ; #True=Not ready  #False=AllDatasRecived
  Result=#True
  NCEvent = NetworkClientEvent(*Handle\ConnectionID)
  If NCEvent=2 ; Raw-data
    If *Handle\Buffer=0
      *Handle\Buffer=AllocateMemory(*Handle\BufferLength)
      If *Handle\Buffer=0
        *Handle\URLError=#URLERROR_OutOfMemory
        ProcedureReturn #False
      EndIf
    EndIf
   
    InBuffer=ReceiveNetworkData(*Handle\ConnectionID,*Handle\Buffer,*Handle\BufferLength)
    *start.BYTE=*Handle\Buffer
    If *Handle\InHeader ;{ Header analysieren
      While InBuffer
        char=(*start\b & $FF):InBuffer-1:*start+1
        Select char
          Case #CR;: #crlf$
          Case #LF
            If *Handle\lastchar=#CR; Ende der Zeile entdeckt!
              If *Handle\eol; zum zweiten mal -> nun daten
                *Handle\InHeader=#False
                ;{-Header eingelesen ->Auswerten
                If *Handle\URLError<>200
                  Result=#False
                Else;OK, datei öffnen
                  If *Handle\FileSize And *Handle\Typ$="GET" And *Handle\FileName$
                    If API_FileCreate(*Handle\FileHandle,*Handle\FileName$)=#False
                      *Handle\URLError=#URLError_CantCreateFile
                      result=#false
                    EndIf
                  Else
                    Result=#False
                  EndIf
                EndIf
                ;}
                Break
              Else
                *Handle\eol=#True
                Debug *Handle\currentline$
                If *Handle\FirstLine;{-Status-code auslesen
                  *Handle\FirstLine=#False
                  a1=FindString(*Handle\currentline$," ",0)
                  a2=FindString(*Handle\currentline$," ",a1+1):If a2=0:a2=Len(*Handle\currentline$)+1:EndIf
                  ;HTTPVer:Left(*Handle\currentline$,a1-1)
                  *Handle\URLError=Val(Mid(*Handle\currentline$,a1+1,a2-a1-1))
                  *Handle\ErrorMessage$=Right(*Handle\currentline$,Len(*Handle\currentline$)-a2)
                  ;}
                Else;{-Header-Zeile analysieren
                  a=FindString(*Handle\currentline$,":",1)
                  Variable$=Trim(Left(*Handle\currentline$,a-1))
                  Set$=Trim(Right(*Handle\currentline$,Len(*Handle\currentline$)-a))
                  Select UCase(Variable$)
                    Case "PROXY-AUTHENTICATE";{  Realm auslesen
                      a=FindString(Set$,"realm=",0)
                      b=FindString(Set$,#DQUOTE$,a+7)
                      If a And b
                        *Handle\Realm$=Mid(Set$,a+7,b-a-7)
                      EndIf
                      ;}
                    Case "WWW-AUTHENTICATE";{    Realm auslesen
                      a=FindString(Set$,"realm=",0)
                      b=FindString(Set$,#DQUOTE$,a+7)
                      If a And b
                        *Handle\Realm$=Mid(Set$,a+7,b-a-7)
                      EndIf
                      ;}
                    Case "LOCATION":            *Handle\location$=Set$
                    Case "SET-COOKIE";{          Cookie suchen
                      a=FindString(Set$,";",0):If a=0: a=Len(Set$)+1 :EndIf
                      *Handle\cookie$=Left(Set$,a-1)
                      ;}
                    Case "CONTENT-LENGTH":      *Handle\FileSize=Val(Set$)
                    Case "LAST-MODIFIED":       *Handle\Date=HTTP_AnalyseDate(Set$)
                  EndSelect   
                  ;}
                EndIf
                *Handle\currentline$=""
              EndIf
            EndIf
          Default
            *Handle\eol=#False
            *Handle\currentline$+Chr(char)
        EndSelect
        *Handle\lastchar=char
      Wend
      ;}
    EndIf
    If *Handle\InHeader=#False ;{
      *Handle\Received+InBuffer
      API_WriteData(*Handle\FileHandle,*start,InBuffer)
      If *Handle\Received=*Handle\FileSize
        Result=#False;fertig
        API_CloseFile(*Handle\FileHandle)
      EndIf
      ;}
    EndIf
  EndIf
 
  ProcedureReturn Result
EndProcedure


;- example

;url.s="http://192.168.1.1/index.html"
url.s="http://www.sedtech.com/isedquickpdf/downloads/4.35/iSQP0435DLL.zip"
;url.s="http://gpihome.de/Crillion/English_asdfdafs_.txt"

;File$="C:\test.txt"
File$="c:\test.zip"

; ;Proxy required?
; ProxyServer$="192.168.1.1"
; ProxyPort=8080
; ProxyServer$="195.140.251.142"
; ProxyPort=8080
;ProxyPass$=CreateCryptedPass("user","pass")

; ;user and password required?
; WebPass$=CreateCryptedPass(user$,pass$)



If InitNetwork()
 
  OpenConsole()
  PrintN("***Getting file information***")
  If HTTP_UrlInfo(Info.HTTPGetId,url,WebPass$,ProxyServer$,ProxyPort,ProxyPass$,0)
    PrintN("connect...")
    While HTTP_ReceiveData(Info); wait for header
      Delay(100)
    Wend
    HTTP_CloseURL(Info)
    Select HTTP_GetError(Info)
      Case #URLError_OK
        PrintN("FileSize:"+Str(HTTP_GetFileSize(Info)))
        PrintN("FileDate:"+FormatDate("%dd.%mm.%yyyy %hh:%ii:%ss",HTTP_GetFileDate(Info)))
      Case #URLERROR_OutOfMemory:PrintN("Out of memory")
      Case #URLError_CantCreateFile:PrintN("Can't create file")
        ;for all other errors: See download
      Default
        PrintN(Str(HTTP_GetError(Info))+" "+HTTP_GetErrorMessage(Info))
    EndSelect
  EndIf
 
 
  PrintN("")
  PrintN("***Getting file***")
  If HTTP_OpenUrl(Get.HTTPGetId,File$,url,WebPass$,ProxyServer$,ProxyPort,ProxyPass$,0)
    Repeat
      PrintN("Connect...")
      ReTry=#False:Progress=0
      While HTTP_ReceiveData(Get)
        a=HTTP_GetProgress(Get)
        If a<>Progress
          PrintN("Progress: "+Str(a)+"%")
          Progress=a
        EndIf
        ;if want abort
        ;  http_closeurl(get$)
        ;  printn("Abort")
        ;  break 2
        ;endif
        Delay(100)
      Wend
      HTTP_CloseURL(Get)
     
      Select HTTP_GetError(Get)
        Case #URLError_OK:PrintN("Download complete")
        Case #URLERROR_OutOfMemory:PrintN("Out of memory")
        Case #URLError_CantCreateFile:PrintN("Can't create file")
        Case 301;Moved permanently
          PrintN("Moved permanently:"+HTTP_GetNewLocation(Get))
          HTTP_ChangeURL(Get,HTTP_GetNewLocation(Get))
          If HTTP_ReConnect(Get); neu anfordern
            ReTry=#True
          Else
            PrintN("Can't connect")
          EndIf
        Case 302;Moved temporarily
          PrintN("Moved temporarily:"+HTTP_GetNewLocation(Get))
          HTTP_ChangeURL(Get,HTTP_GetNewLocation(Get))
          If HTTP_ReConnect(Get); neu anfordern
            ReTry=#True
          Else
            PrintN("Can't connect")
          EndIf
        Case 305;Use Proxy
          PrintN("Use Proxy:"+HTTP_GetNewLocation(Get))
        Case 401;unauthorized
          PrintN("Unauthorized:"+HTTP_GetAuthenticateRealm(Get))
          Print("  UserName:"):user$=Input():PrintN("")
          Print("  Password:"):pass$=Input():PrintN("")
          If user$
            WebPass$=HTTP_CryptedUserPass(user$,pass$)
            HTTP_ChangeWWWAuthenticate(Get,WebPass$)
            If HTTP_ReConnect(Get); neu anfordern
              ReTry=#True
            Else
              PrintN("Can't connect")
            EndIf
          Else
            PrintN("  abort")
          EndIf
        Case 407;Proxy Authentication Required
          PrintN("Proxy Authentication Required:"+HTTP_GetAuthenticateRealm(Get))
          Print("  UserName:"):user$=Input():PrintN("")
          Print("  Password:"):pass$=Input():PrintN("")
          If user$
            ProxyPass$=HTTP_CryptedUserPass(user$,pass$)
            HTTP_ChangeProxyAuthenticate(Get,ProxyPass$)
            If HTTP_ReConnect(Get); neu anfordern
              ReTry=#True
            Else
              PrintN("Can't connect")
            EndIf
          Else
            PrintN("  abort")
          EndIf
        Default
          PrintN("ServerError:")
          PrintN(Str(HTTP_GetError(Get))+" "+HTTP_GetErrorMessage(Get))
      EndSelect
    Until ReTry=#False
   
    PrintN("")
    PrintN("FileSize:"+Str(HTTP_GetFileSize(Get)))
    PrintN("FileDate:"+FormatDate("%dd.%mm.%yyyy %hh:%ii:%ss",HTTP_GetFileDate(Get)))
   
  Else
    PrintN("Cann't connect")
  EndIf
 
  PrintN("")
  PrintN("Press any key")
  Input()
  CloseConsole()
EndIf

End


Last edited by GPI on Sat Dec 11, 2004 4:56 pm, edited 1 time in total.

Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Dec 11, 2004 4:38 pm 
Offline
User
User
User avatar

Joined: Sun Oct 19, 2003 11:21 am
Posts: 13
Location: www
@GPI: VERY impressive! 8O

...and breathtaking quick.

Is there a possibility downloading to *memory?

_________________
pIII@1Ghz, wXP, 512MB, pb3.93full


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Dec 11, 2004 4:55 pm 
Offline
PureBasic Expert
PureBasic Expert

Joined: Fri Apr 25, 2003 6:41 pm
Posts: 1364
first of all: i change a routine in the code above (HTTP_ReceiveData())

Code:
;please update

Structure HTTPGetId
  url$
  server$
  File$
  FileName$
  Port.l
  wenc$
  ProxyServer$
  ProxyPort.l
  penc$
  cookie$
  location$
  ConnectionID.l
  ErrorMessage$
  FileSize.l
  Date.l
  URLError.l
  Buffer.l
  BufferLength.l
  Realm$
 
  InHeader.l
  FirstLine.l
  currentline$
  lastchar.l
  eol.l
  Received.l
  Typ$
 
  OutBuffer.l
  FileHandle.API_FileHandle
EndStructure

;new procedure

Procedure HTTP_ReceiveDataMemory(*Handle.HTTPGetId) ; #True=Not ready  #False=AllDatasRecived
  Result=#True
  NCEvent = NetworkClientEvent(*Handle\ConnectionID)
  If NCEvent=2 ; Raw-data
    If *Handle\Buffer=0
      *Handle\Buffer=AllocateMemory(*Handle\BufferLength)
      If *Handle\Buffer=0
        *Handle\URLError=#URLERROR_OutOfMemory
        ProcedureReturn #False
      EndIf
    EndIf
   
    InBuffer=ReceiveNetworkData(*Handle\ConnectionID,*Handle\Buffer,*Handle\BufferLength)
    *start.BYTE=*Handle\Buffer
    If *Handle\InHeader ;{ Header analysieren
      While InBuffer
        char=(*start\b & $FF):InBuffer-1:*start+1
        Select char
          Case #CR;: #crlf$
          Case #LF
            If *Handle\lastchar=#CR; Ende der Zeile entdeckt!
              If *Handle\eol; zum zweiten mal -> nun daten
                *Handle\InHeader=#False
                ;{-Header eingelesen ->Auswerten
                If *Handle\URLError<>200
                  Result=#False
                Else;OK, datei öffnen
                  If *Handle\FileSize And *Handle\Typ$="GET"
                    *Handle\OutBuffer=AllocateMemory(*Handle\FileSize)
                    If *Handle\OutBuffer=0
                      *Handle\URLError=#URLERROR_OutOfMemory
                      Result=#False
                    EndIf
                  Else
                    Result=#False
                  EndIf
                EndIf
                ;}
                Break
              Else
                *Handle\eol=#True
                Debug *Handle\currentline$
                If *Handle\FirstLine;{-Status-code auslesen
                  *Handle\FirstLine=#False
                  a1=FindString(*Handle\currentline$," ",0)
                  a2=FindString(*Handle\currentline$," ",a1+1):If a2=0:a2=Len(*Handle\currentline$)+1:EndIf
                  ;HTTPVer:Left(*Handle\currentline$,a1-1)
                  *Handle\URLError=Val(Mid(*Handle\currentline$,a1+1,a2-a1-1))
                  *Handle\ErrorMessage$=Right(*Handle\currentline$,Len(*Handle\currentline$)-a2)
                  ;}
                Else;{-Header-Zeile analysieren
                  a=FindString(*Handle\currentline$,":",1)
                  Variable$=Trim(Left(*Handle\currentline$,a-1))
                  Set$=Trim(Right(*Handle\currentline$,Len(*Handle\currentline$)-a))
                  Select UCase(Variable$)
                    Case "PROXY-AUTHENTICATE";{  Realm auslesen
                      a=FindString(Set$,"realm=",0)
                      b=FindString(Set$,#DQUOTE$,a+7)
                      If a And b
                        *Handle\Realm$=Mid(Set$,a+7,b-a-7)
                      EndIf
                      ;}
                    Case "WWW-AUTHENTICATE";{    Realm auslesen
                      a=FindString(Set$,"realm=",0)
                      b=FindString(Set$,#DQUOTE$,a+7)
                      If a And b
                        *Handle\Realm$=Mid(Set$,a+7,b-a-7)
                      EndIf
                      ;}
                    Case "LOCATION":            *Handle\location$=Set$
                    Case "SET-COOKIE";{          Cookie suchen
                      a=FindString(Set$,";",0):If a=0: a=Len(Set$)+1 :EndIf
                      *Handle\cookie$=Left(Set$,a-1)
                      ;}
                    Case "CONTENT-LENGTH":      *Handle\FileSize=Val(Set$)
                    Case "LAST-MODIFIED":       *Handle\Date=HTTP_AnalyseDate(Set$)
                  EndSelect   
                  ;}
                EndIf
                *Handle\currentline$=""
              EndIf
            EndIf
          Default
            *Handle\eol=#False
            *Handle\currentline$+Chr(char)
        EndSelect
        *Handle\lastchar=char
      Wend
      ;}
    EndIf
    If *Handle\InHeader=#False ;{
      If *Handle\Received+InBuffer<=*Handle\FileSize
        CopyMemory(*start,*Handle\OutBuffer+*Handle\Received,InBuffer)
      EndIf
      *Handle\Received+InBuffer
      If *Handle\Received=*Handle\FileSize
        Result=#False;fertig
        API_CloseFile(*Handle\FileHandle)
      EndIf
      ;}
    EndIf
  EndIf
 
  ProcedureReturn Result
EndProcedure

;-new example
If InitNetwork()
 
  OpenConsole()
  PrintN("***Getting file information***")
  If HTTP_UrlInfo(Info.HTTPGetId,url,WebPass$,ProxyServer$,ProxyPort,ProxyPass$,0)
    PrintN("connect...")
    While HTTP_ReceiveData(Info); wait for header
      Delay(100)
    Wend
    HTTP_CloseURL(Info)
    Select HTTP_GetError(Info)
      Case #URLError_OK
        PrintN("FileSize:"+Str(HTTP_GetFileSize(Info)))
        PrintN("FileDate:"+FormatDate("%dd.%mm.%yyyy %hh:%ii:%ss",HTTP_GetFileDate(Info)))
      Case #URLERROR_OutOfMemory:PrintN("Out of memory")
      Case #URLError_CantCreateFile:PrintN("Can't create file")
        ;for all other errors: See download
      Default
        PrintN(Str(HTTP_GetError(Info))+" "+HTTP_GetErrorMessage(Info))
    EndSelect
  EndIf
 
 
  PrintN("")
  PrintN("***Getting file***")
  If HTTP_OpenUrl(Get.HTTPGetId,File$,url,WebPass$,ProxyServer$,ProxyPort,ProxyPass$,0)
    Repeat
      PrintN("Connect...")
      ReTry=#False:Progress=0
      While HTTP_ReceiveDataMemory(Get)
        a=HTTP_GetProgress(Get)
        If a<>Progress
          PrintN("Progress: "+Str(a)+"%")
          Progress=a
        EndIf
        ;if want abort
        ;  http_closeurl(get$)
        ;  printn("Abort")
        ;  break 2
        ;endif
        Delay(100)
      Wend
      HTTP_CloseURL(Get)
     
      Select HTTP_GetError(Get)
        Case #URLError_OK:PrintN("Download complete")
          OutBuffer=HTTP_GetOutBuffer(Get)
          Size=HTTP_GetFileSize(Get)
          Print("save to "+File$+".mem."+GetExtensionPart(File$))
          API_FileCreate(out.API_FileHandle,File$+".mem."+GetExtensionPart(File$))
          API_WriteData(out,OutBuffer,Size)
          API_CloseFile(out)
          freememory(outbuffer)
          PrintN(" done")
        Case #URLERROR_OutOfMemory:PrintN("Out of memory")
        Case #URLError_CantCreateFile:PrintN("Can't create file")
        Case 301;Moved permanently
          PrintN("Moved permanently:"+HTTP_GetNewLocation(Get))
          HTTP_ChangeURL(Get,HTTP_GetNewLocation(Get))
          If HTTP_ReConnect(Get); neu anfordern
            ReTry=#True
          Else
            PrintN("Can't connect")
          EndIf
        Case 302;Moved temporarily
          PrintN("Moved temporarily:"+HTTP_GetNewLocation(Get))
          HTTP_ChangeURL(Get,HTTP_GetNewLocation(Get))
          If HTTP_ReConnect(Get); neu anfordern
            ReTry=#True
          Else
            PrintN("Can't connect")
          EndIf
        Case 305;Use Proxy
          PrintN("Use Proxy:"+HTTP_GetNewLocation(Get))
        Case 401;unauthorized
          PrintN("Unauthorized:"+HTTP_GetAuthenticateRealm(Get))
          Print("  UserName:"):user$=Input():PrintN("")
          Print("  Password:"):pass$=Input():PrintN("")
          If user$
            WebPass$=HTTP_CryptedUserPass(user$,pass$)
            HTTP_ChangeWWWAuthenticate(Get,WebPass$)
            If HTTP_ReConnect(Get); neu anfordern
              ReTry=#True
            Else
              PrintN("Can't connect")
            EndIf
          Else
            PrintN("  abort")
          EndIf
        Case 407;Proxy Authentication Required
          PrintN("Proxy Authentication Required:"+HTTP_GetAuthenticateRealm(Get))
          Print("  UserName:"):user$=Input():PrintN("")
          Print("  Password:"):pass$=Input():PrintN("")
          If user$
            ProxyPass$=HTTP_CryptedUserPass(user$,pass$)
            HTTP_ChangeProxyAuthenticate(Get,ProxyPass$)
            If HTTP_ReConnect(Get); neu anfordern
              ReTry=#True
            Else
              PrintN("Can't connect")
            EndIf
          Else
            PrintN("  abort")
          EndIf
        Default
          PrintN("ServerError:")
          PrintN(Str(HTTP_GetError(Get))+" "+HTTP_GetErrorMessage(Get))
      EndSelect
    Until ReTry=#False
   
    PrintN("")
    PrintN("FileSize:"+Str(HTTP_GetFileSize(Get)))
    PrintN("FileDate:"+FormatDate("%dd.%mm.%yyyy %hh:%ii:%ss",HTTP_GetFileDate(Get)))
   
  Else
    PrintN("Cann't connect")
  EndIf
 
  PrintN("")
  PrintN("Press any key")
  Input()
  CloseConsole()
EndIf

End


And don't forget to free the output-buffer!


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Dec 11, 2004 5:39 pm 
Offline
Addict
Addict
User avatar

Joined: Thu Oct 16, 2003 8:30 pm
Posts: 1998
Location: North Florida
@GPI..

So, this second code-post, can be added to the first code-post above it.
And there will be no conflicts?

I have your old lib.. so what you're saying is to use this code, and NOT
the Lib right?

- np


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Dec 11, 2004 5:54 pm 
Offline
PureBasic Expert
PureBasic Expert

Joined: Fri Apr 25, 2003 6:41 pm
Posts: 1364
NoahPhense wrote:
@GPI..

So, this second code-post, can be added to the first code-post above it.
And there will be no conflicts?

Replace the structure and the example. paste the two procedures

>I have your old lib..

the GetURL-Lib is from Num3, not from me.

> so what you're saying is to use this code, and NOT
the Lib right?

When the lib is based on the source in the thread-start-posting: Yes, you should not use the lib.

btw.: Thanks to num3, because with his lib, i can't write my code.


- np[/quote]


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Dec 11, 2004 5:56 pm 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Wed May 14, 2003 3:38 pm
Posts: 229
Location: Lüneburg - Germany
here it's working with HTTP-Proxy!

EDIT: with password ;)


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Dec 11, 2004 7:13 pm 
Offline
User
User
User avatar

Joined: Sun Oct 19, 2003 11:21 am
Posts: 13
Location: www
Maybe I'm blind, but... where's the HTTP_GetOutBuffer() Procedure?

Error in line 637 (not a function)...

_________________
pIII@1Ghz, wXP, 512MB, pb3.93full


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Dec 11, 2004 8:33 pm 
Offline
PureBasic Expert
PureBasic Expert

Joined: Fri Apr 25, 2003 6:41 pm
Posts: 1364
ups.
Code:
Procedure HTTP_GetOutBuffer(*Handle.HTTPGetId); File must be downloaded with HTTP_ReceiveDataMemory()
  ProcedureReturn *Handle\OutBuffer
EndProcedure


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sun Dec 12, 2004 4:19 pm 
Offline
PureBasic Expert
PureBasic Expert
User avatar

Joined: Fri Apr 25, 2003 4:51 pm
Posts: 2811
Location: Portugal, Lisbon
NEWS!!!

This library has been rebuilt -> HTTP library

_________________
Elevator Maintenance: http://central-elevadores.pt
Good web hosting![/size]


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sun Dec 12, 2004 4:51 pm 
Offline
PureBasic Expert
PureBasic Expert

Joined: Fri Apr 25, 2003 6:41 pm
Posts: 1364
Num3 wrote:
NEWS!!!

This library has been rebuilt -> HTTP library


Source?


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Mon Dec 13, 2004 9:21 am 
Offline
Moderator
Moderator

Joined: Sat Dec 27, 2003 3:55 am
Posts: 3321
Location: Great Southern Land
@GPI

Thanks for the code there. Appreciated. :)

_________________
@}--`--,-- A rose by any other name ..


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 27 posts ]  Go to page Previous  1, 2

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 30 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

 


Powered by phpBB © 2008 phpBB Group
subSilver+ theme by Canver Software, sponsor Sanal Modifiye