Sequential file problem

Just starting out? Need help? Post your questions and find answers here.
User avatar
C87
Enthusiast
Enthusiast
Posts: 176
Joined: Mon Jul 17, 2017 7:22 am
Location: Cotswolds England

Sequential file problem

Post by C87 »

Without wanting to bore you with the reasons, I need to create a text file of possibly thousands of records. Each record will be a specific length. Once in the file I then need to locate specific records, amend the contents and write the changes back. I have no problem at all in creating the file but I am having some difficulty in locating specific records, never mind amending them. The following code creates a text file of 200 records and then attempts to retrieve two specific records but only shows the first two in the file, not the ones I need.

Help appreciated but I am of the mind that PureBasic will simply not allow me to do this.

C87

Code: Select all

; *****************************************************************************
;***  Attempt to locate specific records in what I  think is a sequential file                                                
;***   but doesn't appear to be                                                                                                                                                
;******************************************************************************
    mRecord.s = ""
    mLastRandom.s = ""
    mName.s =Chr(255)
    Rec.i = 0     : mData.i = 0
    
If CreateFile(0, "C:\Temp\Test.txt")         ; we create a new text file...
   For Rec=1 To 200  ; Add 200 records/rows into a file each 80 characters long
       mRecord = ""
       For mData = 1 To 40 
            ;
            mLastRandom = Str( Random(99,10))
            mRecord = mRecord + mLastRandom    ; created a string 80 characters lomg
            ;
       Next   ; mData
       ;
       WriteStringN(0, mRecord)  ;   write to file
   Next   ; Rec 200
    ;
    CloseFile(0)                       ; close the previously opened file and store the written data this way
Else
     MessageRequester("ERROR ON ADDING DATA","File not created!", #PB_MessageRequester_Warning)
EndIf
 ;  
 ;*************************************************************************************************
 ;*▼*▼*▼*▼*▼*▼*▼*▼*▼** this code works and displays the 200 records created above all showing as expected
 ;   
 ;If ReadFile(0,"C:\Temp\Test.txt")           
 ;     ;
 ;     While Eof(0) = 0           ; loop as long the 'end of file' isn't reached, so it must move record ot record sequentially
 ;           Debug ReadString(0)      ; display line by line in the debug window
 ;     Wend
 ;   CloseFile(0)               ; 
 ;    ;
 ;EndIf
 ; *************************************************************************************************
 ; THE ABOVE WORKS AS EXPECTED LISTING EACH OF THE 200 RECORDS 
 ; HOWEVER: IF I AMEND THE CODE TO LOCATE AN INDIVIDUAL RECORD IT DRIFTS OFF INTO A WORLD OF ITS OWN UNLESS I SEND IT TO EOF()
 ; As the above works I presumed that it reads each record in turn. 
 ; There doesn't appear to be an implicit record number I therefore added an incrementing variable, presuming it advances one each time on moving on through the file
 ; !!!!!!!! apparently not it seems !!
 ;
If ReadFile(0,"C:\Temp\Test.txt")
     mRec.i=0
    While Eof(0) = 0           ; loop as long the 'end of file' isn't reached
        mRec = mREc +1
        If mRec = 50                        
             Debug ReadString(0)      ; This will not display record Nº 50 but record Nº 1
        EndIf
        If mRec = 120 
            Debug ReadString(0)     ; this displays record Nº 2 not record Nº 120
        EndIf
        ;                                        If I add another If/Debug/Endif it simply displays the 3rd record regardless of the mRec value in the If/Endif
        If mRec = 200                   ;}  without this If/Endif it just drifts off into its own world
            mEOF = Lof(0)                ;}
            FileSeek(0,mEOF)            ;}
         EndIf                                ;} 
    Wend
    CloseFile(0)               ; close 
     ;
EndIf
;
If it's falling over......just remember the computer is never wrong!
Christophe_fr
User
User
Posts: 13
Joined: Thu Dec 13, 2018 9:58 pm
Location: france

Re: Sequential file problem

Post by Christophe_fr »

It's normal that you only read the first 2 lines, since you are only 2 readstring. Try this

Code: Select all

; *****************************************************************************
;***  Attempt to locate specific records in what I  think is a sequential file                                               
;***   but doesn't appear to be                                                                                                                                               
;******************************************************************************
    mRecord.s = ""
    mLastRandom.s = ""
    mName.s =Chr(255)
    Rec.i = 0     : mData.i = 0
   
If CreateFile(0, "C:\Temp\Test.txt")         ; we create a new text file...
   For Rec=1 To 200  ; Add 200 records/rows into a file each 80 characters long
       mRecord = ""
       For mData = 1 To 40
            ;
            mLastRandom = Str( Random(99,10))
            mRecord = mRecord + mLastRandom    ; created a string 80 characters lomg
            ;
       Next   ; mData
       ;
       WriteStringN(0, mRecord)  ;   write to file
   Next   ; Rec 200
    ;
    CloseFile(0)                       ; close the previously opened file and store the written data this way
Else
     MessageRequester("ERROR ON ADDING DATA","File not created!", #PB_MessageRequester_Warning)
EndIf
 ; 
 ;*************************************************************************************************
 ;*▼*▼*▼*▼*▼*▼*▼*▼*▼** this code works and displays the 200 records created above all showing as expected
 ;   
 ;If ReadFile(0,"C:\Temp\Test.txt")           
 ;     ;
 ;     While Eof(0) = 0           ; loop as long the 'end of file' isn't reached, so it must move record ot record sequentially
 ;           Debug ReadString(0)      ; display line by line in the debug window
 ;     Wend
 ;   CloseFile(0)               ;
 ;    ;
 ;EndIf
 ; *************************************************************************************************
 ; THE ABOVE WORKS AS EXPECTED LISTING EACH OF THE 200 RECORDS
 ; HOWEVER: IF I AMEND THE CODE TO LOCATE AN INDIVIDUAL RECORD IT DRIFTS OFF INTO A WORLD OF ITS OWN UNLESS I SEND IT TO EOF()
 ; As the above works I presumed that it reads each record in turn.
 ; There doesn't appear to be an implicit record number I therefore added an incrementing variable, presuming it advances one each time on moving on through the file
 ; !!!!!!!! apparently not it seems !!
 ;
If ReadFile(0,"C:\Temp\Test.txt")
     mRec.i=0
     While Eof(0) = 0           ; loop as long the 'end of file' isn't reached
       f$ = ReadString(0)
        mRec = mREc +1
        If mRec = 50                       
             Debug f$      ; This will not display record Nº 50 but record Nº 1
        EndIf
        If mRec = 120
            Debug f$     ; this displays record Nº 2 not record Nº 120
        EndIf
        ;                                        If I add another If/Debug/Endif it simply displays the 3rd record regardless of the mRec value in the If/Endif
        If mRec = 200                   ;}  without this If/Endif it just drifts off into its own world
            mEOF = Lof(0)                ;}
            FileSeek(0,mEOF)            ;}
         EndIf                                ;}
    Wend
    CloseFile(0)               ; close
     ;
EndIf
;
Excuse for my english. Translated from French into English by Google Translation
Sorry for my English. My language is French
TassyJim
Enthusiast
Enthusiast
Posts: 149
Joined: Sun Jun 16, 2013 6:27 am
Location: Tasmania (Australia)

Re: Sequential file problem

Post by TassyJim »

You don't have to step through each line if you are certain about the record size (including the end-of-line)

Code: Select all

If ReadFile(0,"C:\Temp\Test.txt")
     mRec.i=0
     mRec = 50
     FileSeek(0,(80+2)*(Mrec-1)) ; record size plus two for the CRLF in Windows files
     Debug ReadString(0)
     mRec = 120
     FileSeek(0,(80+2)*(Mrec-1))
     Debug ReadString(0)
    CloseFile(0)               ; close
     ;
EndIf
Jim
User avatar
C87
Enthusiast
Enthusiast
Posts: 176
Joined: Mon Jul 17, 2017 7:22 am
Location: Cotswolds England

Re: Sequential file problem

Post by C87 »

Hi Chrisophe_fr, that didn't work. It just shows the first line in the file twice.
TassyJim wrote:You don't have to step through each line if you are certain about the record size (including the end-of-line)

Jim
Jim, your suggestion works. Many thanks C87


P.S.
G'day mate, Strewth that's dinky-di, I'm stoked.

Tassy is my daughter's favourite place over there. Mind you that may be something to do with being stopped for speeding and then not fined. Her fiancée is from Perth but they are living over here in Yorkshire at present.
If it's falling over......just remember the computer is never wrong!
Christophe_fr
User
User
Posts: 13
Joined: Thu Dec 13, 2018 9:58 pm
Location: france

Re: Sequential file problem

Post by Christophe_fr »

Hi C87, I'm sorry that it does not work for you. I redid the test, for me it works :?

img 1: https://ibb.co/fdgcNhY
img 2: https://ibb.co/RHRPzpL

Excuse for my english. Translated from French into English by Google Translation
Sorry for my English. My language is French
User avatar
Mijikai
Addict
Addict
Posts: 1360
Joined: Sun Sep 11, 2016 2:17 pm

Re: Sequential file problem

Post by Mijikai »

Solution without a textfile but easy & fast string support (Unicode only!):

Code: Select all

EnableExplicit

Structure RECORD_STRUCT
  String.s{80}
EndStructure

Global File.s
Global *Record

Procedure.i CreatePseudoRecord(File.s)
  Protected FileHandle.i
  Protected Record.s
  Protected Entry.s
  Protected Entries.i
  Protected Index.i
  For Entries = 1 To 200
    Entry = #Null$
    For Index = 1 To 40
      Entry + Str(Random(99,10))
    Next
    Record + Entry
  Next
  FileHandle = CreateFile(#PB_Any,File)
  If FileHandle
    WriteData(FileHandle,@Record,StringByteLength(Record))
    CloseFile(FileHandle)
    ProcedureReturn #True
  EndIf
EndProcedure

Procedure.i ReadRecord(File.s)
  Protected FileHandle.i
  Protected *Buffer
  Protected BufferSize.i
  FileHandle = ReadFile(#PB_Any,File)
  If FileHandle
    BufferSize = Lof(FileHandle)
    If BufferSize > #Null
      *Buffer = AllocateMemory(BufferSize)
      If *Buffer
        ReadData(FileHandle,*Buffer,BufferSize)
      EndIf
    EndIf
    CloseFile(FileHandle)  
  EndIf
  ProcedureReturn *Buffer
EndProcedure

Procedure.s RecordEntry(*Record,Index.i)
  Protected *Entry.RECORD_STRUCT
  Index * 160;160 = unicode!
  If Index < MemorySize(*Record);simple sanity check!
    *Entry = *Record + Index
    ProcedureReturn *Entry\String
  EndIf
EndProcedure

File = "record.bin"
If CreatePseudoRecord(File)
  *Record = ReadRecord(File)
  If *Record
    ;----------
    Debug RecordEntry(*Record,0)
    Debug RecordEntry(*Record,50)
    Debug RecordEntry(*Record,5)
    ;----------
    FreeMemory(*Record)
  EndIf
EndIf  
  
End
Why no Database!?
Last edited by Mijikai on Fri Dec 14, 2018 7:52 pm, edited 1 time in total.
User avatar
C87
Enthusiast
Enthusiast
Posts: 176
Joined: Mon Jul 17, 2017 7:22 am
Location: Cotswolds England

Re: Sequential file problem

Post by C87 »

Christophe_fr wrote:Hi C87, I'm sorry that it does not work for you. I redid the test, for me it works :?
img 1: https://ibb.co/fdgcNhY
img 2: https://ibb.co/RHRPzpL
Excuse for my english. Translated from French into English by Google Translation
Apologies Christophe_fr I put the F$ = Readstring(0) outside the While/Wend. You are quite correct, your suggestion does work. :oops:
In fact it works very well. As well as reading a row located, I can add a couple of lines of code to amend the selected row and write it back with the changes. Doing exactly what I needed to do.

I do suspect that your English is better than my Spanish with or without Google!
Last edited by C87 on Mon Dec 17, 2018 4:09 pm, edited 2 times in total.
If it's falling over......just remember the computer is never wrong!
User avatar
C87
Enthusiast
Enthusiast
Posts: 176
Joined: Mon Jul 17, 2017 7:22 am
Location: Cotswolds England

Re: Sequential file problem

Post by C87 »

Thanks for your proposal Mijikia, I'm going to go through it.
You mentioned "why no database". I am actually hiding various security details within a text file containing random ASCII 33 to 254, not simply numbers. Around 3,000 lines at 254 chars long each that appear as unreadable in any editor.
If it's falling over......just remember the computer is never wrong!
User avatar
skywalk
Addict
Addict
Posts: 3960
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Sequential file problem

Post by skywalk »

You can do that in a SQLite db also and with more flexibility and code reuse. :wink:
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
C87
Enthusiast
Enthusiast
Posts: 176
Joined: Mon Jul 17, 2017 7:22 am
Location: Cotswolds England

Re: Sequential file problem

Post by C87 »

skywalk wrote:You can do that in a SQLite db also and with more flexibility and code reuse. :wink:
Hello Skywalk, You are quite right, particularly as I have started using SQLite with PureBasic, probably sensible. It’s a case of old habits I suppose. I’ve used text files renamed as various other files for remote installation for years. Usually copied into some obscure folder. They work on the principle that if a client decides to tinker and take a look they are faced with 20 screens of gibberish. They usually only contained two or three hidden four digit numbers but they’d never be able to relate them to anything, even if they could find them. From that I’d generate their license file and place that somewhere else. Well that was the theory and it has served me well as far as I know. Only one problem I had was when I used the .dbf extension and one Antivirus opened it and I think tried write the header back, corrupting it! But I take your point about SQLite.
C87
If it's falling over......just remember the computer is never wrong!
User avatar
skywalk
Addict
Addict
Posts: 3960
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Sequential file problem

Post by skywalk »

I would not think antivirus apps edit scanned files? :shock:
If they find something viral, they move them or make them hidden.

Since you obfuscate the contents of your security data, it could reside in your application data folder as a database file, or better, in your main app's database file in a separate table. This is easier to track than isolated text files.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
Mijikai
Addict
Addict
Posts: 1360
Joined: Sun Sep 11, 2016 2:17 pm

Re: Sequential file problem

Post by Mijikai »

I made a more complete example :)
read/replace/add

RECORD Module:

Code: Select all

DeclareModule RECORD
  Declare.i Access(File.s)
  Declare.i Entries()
  Declare.s Entry(Index.i,NewEntry.s = #Null$);if index is < #Null and NewEntry THE ENTRY WILL BE ADDED TO THE FILEND!
  Declare.i Close()
EndDeclareModule

Module RECORD
  
  EnableExplicit

  Structure RECORD_STRUCT
    String.s{80}
  EndStructure
  
  Global Dim Record.w(87)
  Global *Entry.RECORD_STRUCT = @Record()
  Global FileHandle.i
  Global Entries.i
  
  Procedure.i Access(File.s)
    If FileHandle
      CloseFile(FileHandle)
      Entries = #Null
    EndIf 
    FileHandle = OpenFile(#PB_Any,File)
    If FileHandle
      Entries = Lof(FileHandle) / 160
      ProcedureReturn #True    
    EndIf 
  EndProcedure
  
  Procedure.i Entries()
    ProcedureReturn Entries
  EndProcedure
  
  Procedure.s Entry(Index.i,NewEntry.s = #Null$)
    Protected BufferSize.i
    If FileHandle
      Index * 160
      If NewEntry
        BufferSize = StringByteLength(NewEntry)
        If Index < #Null
          FileSeek(FileHandle,Lof(FileHandle),#PB_Absolute)
          If WriteData(FileHandle,@NewEntry,BufferSize) = BufferSize
            Entries + 1
            ProcedureReturn NewEntry  
          EndIf 
        Else
          If Index < Lof(FileHandle)
            FileSeek(FileHandle,Index,#PB_Absolute)   
          Else
            FileSeek(FileHandle,Lof(FileHandle),#PB_Absolute) 
          EndIf
        EndIf  
        If WriteData(FileHandle,@NewEntry,BufferSize) = BufferSize
          ProcedureReturn NewEntry  
        EndIf 
      Else
        If Index < Lof(FileHandle)
          FileSeek(FileHandle,Index,#PB_Absolute)
          If ReadData(FileHandle,@Record(),160) = 160
            ProcedureReturn *Entry\String
          EndIf
        EndIf
      EndIf 
    EndIf 
  EndProcedure
  
  Procedure.i Close()
    If FileHandle
      CloseFile(FileHandle)
    EndIf 
  EndProcedure
  
EndModule

EnableExplicit

Global File.s
Global Record.i

Procedure.i CreatePseudoRecord(File.s)
  Protected FileHandle.i
  Protected Record.s
  Protected Entry.s
  Protected Entries.i
  Protected Index.i
  For Entries = 1 To 200
    Entry = #Null$
    For Index = 1 To 40
      Entry + Str(Random(99,10))
    Next
    Record + Entry
  Next
  FileHandle = CreateFile(#PB_Any,File)
  If FileHandle
    WriteData(FileHandle,@Record,StringByteLength(Record))
    CloseFile(FileHandle)
    ProcedureReturn #True
  EndIf
EndProcedure

File = "record.bin"
If CreatePseudoRecord(File)
  If RECORD::Access(File)
    Debug RECORD::Entries()
    Debug "---------------------"
    For Record = 0 To RECORD::Entries() - 1
      Debug RECORD::Entry(Record)  
    Next
    Debug "---------------------"
    Debug RECORD::Entry(66,"00000034704479755457482989933479958480184854798023139688561099699823547249762632")
    Debug RECORD::Entry(66)
    Debug RECORD::Entry(-1,"00000034704479755457482989933479958480184854798023139688561099699823547249700000")
    Debug RECORD::Entries()
    Debug RECORD::Entry(200)
    RECORD::Close() 
  EndIf 
EndIf 

End
infratec
Always Here
Always Here
Posts: 6810
Joined: Sun Sep 07, 2008 12:45 pm
Location: Germany

Re: Sequential file problem

Post by infratec »

Hi, hi,

years ago I wrote a tool to read and and write FoxPro dbf files.
In prinzipal that's the same stuff. The only difference:
There is a header in front where the fieldnames, types and length are stored.

Simply: it's a file based database :mrgreen:
User avatar
C87
Enthusiast
Enthusiast
Posts: 176
Joined: Mon Jul 17, 2017 7:22 am
Location: Cotswolds England

Re: Sequential file problem

Post by C87 »

skywalk wrote:I would not think antivirus apps edit scanned files? :shock:
If they find something viral, they move them or make them hidden.
It certainly appeared that the antivirus corrupted the text file with an extension of .DBF. At the time I thought it was something to do with it being inside a Winzip executable. Whether the antivirus should have, or not done this the fact remains it did alter the file by trying to add a header. I wasn't interested to find out why or how, so I did a work-around.
skywalk wrote:Since you obfuscate the contents of your security data, it could reside in your application data folder as a database file, or better, in your main app's database file in a separate table. This is easier to track than isolated text files.
I have been writing software since the early 1980s, so I am aware of where I can place my data. I had no need to track files as my software placed them into the folders where my other software would find them. I have successfully used a text file to control ownership and the number of users since I stopped using dongles to control licensing. As I wrote a lot of MS-Access applications I preferred to keep some critical data out of the backend databases. I am now retired but messing with PureBasic from time to time as it reminds me of Clipper. I'm not up to speed with PureBasic yet but getting there.....hopefully. It's always little things that cause big problems!
Mijikai wrote:I made a more complete example :)
read/replace/add
Cheers for that Mijiai, quite brilliant.
Infratec wrote:Hi, hi,
years ago I wrote a tool to read and and write FoxPro dbf files.
In prinzipal that's the same stuff. The only difference:
There is a header in front where the fieldnames, types and length are stored. Simply: it's a file based database :mrgreen:
That's what I thought at the time when I saw the file, although the header wasn't complete
If it's falling over......just remember the computer is never wrong!
User avatar
skywalk
Addict
Addict
Posts: 3960
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Sequential file problem

Post by skywalk »

Certainly, no pressure from me to use a database. I use text files also. But, reading/writing/inserting/deleting binary records while moving file position pointers is built-in to a database. I have too much code to write to reinvent that wheel. :wink:
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
Post Reply