It is currently Wed Nov 13, 2019 1:54 pm

All times are UTC + 1 hour




Post new topic Reply to topic  [ 16 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Sequential file problem
PostPosted: Thu Dec 13, 2018 9:43 pm 
Offline
User
User

Joined: Mon Jul 17, 2017 7:22 am
Posts: 66
Location: Cotswolds England
Without wanting to bore you with the reasons, I need to create a text file of possibly thousands of records. Each record will be a specific length. Once in the file I then need to locate specific records, amend the contents and write the changes back. I have no problem at all in creating the file but I am having some difficulty in locating specific records, never mind amending them. The following code creates a text file of 200 records and then attempts to retrieve two specific records but only shows the first two in the file, not the ones I need.

Help appreciated but I am of the mind that PureBasic will simply not allow me to do this.

C87

Code:
; *****************************************************************************
;***  Attempt to locate specific records in what I  think is a sequential file                                               
;***   but doesn't appear to be                                                                                                                                               
;******************************************************************************
    mRecord.s = ""
    mLastRandom.s = ""
    mName.s =Chr(255)
    Rec.i = 0     : mData.i = 0
   
If CreateFile(0, "C:\Temp\Test.txt")         ; we create a new text file...
   For Rec=1 To 200  ; Add 200 records/rows into a file each 80 characters long
       mRecord = ""
       For mData = 1 To 40
            ;
            mLastRandom = Str( Random(99,10))
            mRecord = mRecord + mLastRandom    ; created a string 80 characters lomg
            ;
       Next   ; mData
       ;
       WriteStringN(0, mRecord)  ;   write to file
   Next   ; Rec 200
    ;
    CloseFile(0)                       ; close the previously opened file and store the written data this way
Else
     MessageRequester("ERROR ON ADDING DATA","File not created!", #PB_MessageRequester_Warning)
EndIf
 ; 
 ;*************************************************************************************************
 ;*▼*▼*▼*▼*▼*▼*▼*▼*▼** this code works and displays the 200 records created above all showing as expected
 ;   
 ;If ReadFile(0,"C:\Temp\Test.txt")           
 ;     ;
 ;     While Eof(0) = 0           ; loop as long the 'end of file' isn't reached, so it must move record ot record sequentially
 ;           Debug ReadString(0)      ; display line by line in the debug window
 ;     Wend
 ;   CloseFile(0)               ;
 ;    ;
 ;EndIf
 ; *************************************************************************************************
 ; THE ABOVE WORKS AS EXPECTED LISTING EACH OF THE 200 RECORDS
 ; HOWEVER: IF I AMEND THE CODE TO LOCATE AN INDIVIDUAL RECORD IT DRIFTS OFF INTO A WORLD OF ITS OWN UNLESS I SEND IT TO EOF()
 ; As the above works I presumed that it reads each record in turn.
 ; There doesn't appear to be an implicit record number I therefore added an incrementing variable, presuming it advances one each time on moving on through the file
 ; !!!!!!!! apparently not it seems !!
 ;
If ReadFile(0,"C:\Temp\Test.txt")
     mRec.i=0
    While Eof(0) = 0           ; loop as long the 'end of file' isn't reached
        mRec = mREc +1
        If mRec = 50                       
             Debug ReadString(0)      ; This will not display record Nº 50 but record Nº 1
        EndIf
        If mRec = 120
            Debug ReadString(0)     ; this displays record Nº 2 not record Nº 120
        EndIf
        ;                                        If I add another If/Debug/Endif it simply displays the 3rd record regardless of the mRec value in the If/Endif
        If mRec = 200                   ;}  without this If/Endif it just drifts off into its own world
            mEOF = Lof(0)                ;}
            FileSeek(0,mEOF)            ;}
         EndIf                                ;}
    Wend
    CloseFile(0)               ; close
     ;
EndIf
;

_________________
If it's falling over......just remember the computer is never wrong!


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Thu Dec 13, 2018 10:10 pm 
Offline
User
User

Joined: Thu Dec 13, 2018 9:58 pm
Posts: 13
Location: france
It's normal that you only read the first 2 lines, since you are only 2 readstring. Try this

Code:
; *****************************************************************************
;***  Attempt to locate specific records in what I  think is a sequential file                                               
;***   but doesn't appear to be                                                                                                                                               
;******************************************************************************
    mRecord.s = ""
    mLastRandom.s = ""
    mName.s =Chr(255)
    Rec.i = 0     : mData.i = 0
   
If CreateFile(0, "C:\Temp\Test.txt")         ; we create a new text file...
   For Rec=1 To 200  ; Add 200 records/rows into a file each 80 characters long
       mRecord = ""
       For mData = 1 To 40
            ;
            mLastRandom = Str( Random(99,10))
            mRecord = mRecord + mLastRandom    ; created a string 80 characters lomg
            ;
       Next   ; mData
       ;
       WriteStringN(0, mRecord)  ;   write to file
   Next   ; Rec 200
    ;
    CloseFile(0)                       ; close the previously opened file and store the written data this way
Else
     MessageRequester("ERROR ON ADDING DATA","File not created!", #PB_MessageRequester_Warning)
EndIf
 ;
 ;*************************************************************************************************
 ;*▼*▼*▼*▼*▼*▼*▼*▼*▼** this code works and displays the 200 records created above all showing as expected
 ;   
 ;If ReadFile(0,"C:\Temp\Test.txt")           
 ;     ;
 ;     While Eof(0) = 0           ; loop as long the 'end of file' isn't reached, so it must move record ot record sequentially
 ;           Debug ReadString(0)      ; display line by line in the debug window
 ;     Wend
 ;   CloseFile(0)               ;
 ;    ;
 ;EndIf
 ; *************************************************************************************************
 ; THE ABOVE WORKS AS EXPECTED LISTING EACH OF THE 200 RECORDS
 ; HOWEVER: IF I AMEND THE CODE TO LOCATE AN INDIVIDUAL RECORD IT DRIFTS OFF INTO A WORLD OF ITS OWN UNLESS I SEND IT TO EOF()
 ; As the above works I presumed that it reads each record in turn.
 ; There doesn't appear to be an implicit record number I therefore added an incrementing variable, presuming it advances one each time on moving on through the file
 ; !!!!!!!! apparently not it seems !!
 ;
If ReadFile(0,"C:\Temp\Test.txt")
     mRec.i=0
     While Eof(0) = 0           ; loop as long the 'end of file' isn't reached
       f$ = ReadString(0)
        mRec = mREc +1
        If mRec = 50                       
             Debug f$      ; This will not display record Nº 50 but record Nº 1
        EndIf
        If mRec = 120
            Debug f$     ; this displays record Nº 2 not record Nº 120
        EndIf
        ;                                        If I add another If/Debug/Endif it simply displays the 3rd record regardless of the mRec value in the If/Endif
        If mRec = 200                   ;}  without this If/Endif it just drifts off into its own world
            mEOF = Lof(0)                ;}
            FileSeek(0,mEOF)            ;}
         EndIf                                ;}
    Wend
    CloseFile(0)               ; close
     ;
EndIf
;


Excuse for my english. Translated from French into English by Google Translation

_________________
Sorry for my English. My language is French


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Thu Dec 13, 2018 10:26 pm 
Offline
User
User

Joined: Sun Jun 16, 2013 6:27 am
Posts: 89
Location: Tasmania (Australia)
You don't have to step through each line if you are certain about the record size (including the end-of-line)
Code:
If ReadFile(0,"C:\Temp\Test.txt")
     mRec.i=0
     mRec = 50
     FileSeek(0,(80+2)*(Mrec-1)) ; record size plus two for the CRLF in Windows files
     Debug ReadString(0)
     mRec = 120
     FileSeek(0,(80+2)*(Mrec-1))
     Debug ReadString(0)
    CloseFile(0)               ; close
     ;
EndIf


Jim


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Fri Dec 14, 2018 9:27 am 
Offline
User
User

Joined: Mon Jul 17, 2017 7:22 am
Posts: 66
Location: Cotswolds England
Hi Chrisophe_fr, that didn't work. It just shows the first line in the file twice.

TassyJim wrote:
You don't have to step through each line if you are certain about the record size (including the end-of-line)

Jim

Jim, your suggestion works. Many thanks C87


P.S.
G'day mate, Strewth that's dinky-di, I'm stoked.

Tassy is my daughter's favourite place over there. Mind you that may be something to do with being stopped for speeding and then not fined. Her fiancée is from Perth but they are living over here in Yorkshire at present.

_________________
If it's falling over......just remember the computer is never wrong!


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Fri Dec 14, 2018 11:43 am 
Offline
User
User

Joined: Thu Dec 13, 2018 9:58 pm
Posts: 13
Location: france
Hi C87, I'm sorry that it does not work for you. I redid the test, for me it works :?

img 1: https://ibb.co/fdgcNhY
img 2: https://ibb.co/RHRPzpL

Excuse for my english. Translated from French into English by Google Translation

_________________
Sorry for my English. My language is French


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Fri Dec 14, 2018 1:18 pm 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Sun Sep 11, 2016 2:17 pm
Posts: 561
Solution without a textfile but easy & fast string support (Unicode only!):
Code:
EnableExplicit

Structure RECORD_STRUCT
  String.s{80}
EndStructure

Global File.s
Global *Record

Procedure.i CreatePseudoRecord(File.s)
  Protected FileHandle.i
  Protected Record.s
  Protected Entry.s
  Protected Entries.i
  Protected Index.i
  For Entries = 1 To 200
    Entry = #Null$
    For Index = 1 To 40
      Entry + Str(Random(99,10))
    Next
    Record + Entry
  Next
  FileHandle = CreateFile(#PB_Any,File)
  If FileHandle
    WriteData(FileHandle,@Record,StringByteLength(Record))
    CloseFile(FileHandle)
    ProcedureReturn #True
  EndIf
EndProcedure

Procedure.i ReadRecord(File.s)
  Protected FileHandle.i
  Protected *Buffer
  Protected BufferSize.i
  FileHandle = ReadFile(#PB_Any,File)
  If FileHandle
    BufferSize = Lof(FileHandle)
    If BufferSize > #Null
      *Buffer = AllocateMemory(BufferSize)
      If *Buffer
        ReadData(FileHandle,*Buffer,BufferSize)
      EndIf
    EndIf
    CloseFile(FileHandle) 
  EndIf
  ProcedureReturn *Buffer
EndProcedure

Procedure.s RecordEntry(*Record,Index.i)
  Protected *Entry.RECORD_STRUCT
  Index * 160;160 = unicode!
  If Index < MemorySize(*Record);simple sanity check!
    *Entry = *Record + Index
    ProcedureReturn *Entry\String
  EndIf
EndProcedure

File = "record.bin"
If CreatePseudoRecord(File)
  *Record = ReadRecord(File)
  If *Record
    ;----------
    Debug RecordEntry(*Record,0)
    Debug RecordEntry(*Record,50)
    Debug RecordEntry(*Record,5)
    ;----------
    FreeMemory(*Record)
  EndIf
EndIf 
 
End

Why no Database!?


Last edited by Mijikai on Fri Dec 14, 2018 7:52 pm, edited 1 time in total.

Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Fri Dec 14, 2018 5:56 pm 
Offline
User
User

Joined: Mon Jul 17, 2017 7:22 am
Posts: 66
Location: Cotswolds England
Christophe_fr wrote:
Hi C87, I'm sorry that it does not work for you. I redid the test, for me it works :?
img 1: https://ibb.co/fdgcNhY
img 2: https://ibb.co/RHRPzpL
Excuse for my english. Translated from French into English by Google Translation


Apologies Christophe_fr I put the F$ = Readstring(0) outside the While/Wend. You are quite correct, your suggestion does work. :oops:
In fact it works very well. As well as reading a row located, I can add a couple of lines of code to amend the selected row and write it back with the changes. Doing exactly what I needed to do.

I do suspect that your English is better than my Spanish with or without Google!

_________________
If it's falling over......just remember the computer is never wrong!


Last edited by C87 on Mon Dec 17, 2018 4:09 pm, edited 2 times in total.

Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Fri Dec 14, 2018 6:08 pm 
Offline
User
User

Joined: Mon Jul 17, 2017 7:22 am
Posts: 66
Location: Cotswolds England
Thanks for your proposal Mijikia, I'm going to go through it.
You mentioned "why no database". I am actually hiding various security details within a text file containing random ASCII 33 to 254, not simply numbers. Around 3,000 lines at 254 chars long each that appear as unreadable in any editor.

_________________
If it's falling over......just remember the computer is never wrong!


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Fri Dec 14, 2018 6:53 pm 
Offline
Addict
Addict
User avatar

Joined: Wed Dec 23, 2009 10:14 pm
Posts: 3110
Location: Boston, MA
You can do that in a SQLite db also and with more flexibility and code reuse. :wink:

_________________
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Fri Dec 14, 2018 7:21 pm 
Offline
User
User

Joined: Mon Jul 17, 2017 7:22 am
Posts: 66
Location: Cotswolds England
skywalk wrote:
You can do that in a SQLite db also and with more flexibility and code reuse. :wink:

Hello Skywalk, You are quite right, particularly as I have started using SQLite with PureBasic, probably sensible. It’s a case of old habits I suppose. I’ve used text files renamed as various other files for remote installation for years. Usually copied into some obscure folder. They work on the principle that if a client decides to tinker and take a look they are faced with 20 screens of gibberish. They usually only contained two or three hidden four digit numbers but they’d never be able to relate them to anything, even if they could find them. From that I’d generate their license file and place that somewhere else. Well that was the theory and it has served me well as far as I know. Only one problem I had was when I used the .dbf extension and one Antivirus opened it and I think tried write the header back, corrupting it! But I take your point about SQLite.
C87

_________________
If it's falling over......just remember the computer is never wrong!


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Fri Dec 14, 2018 8:22 pm 
Offline
Addict
Addict
User avatar

Joined: Wed Dec 23, 2009 10:14 pm
Posts: 3110
Location: Boston, MA
I would not think antivirus apps edit scanned files? :shock:
If they find something viral, they move them or make them hidden.

Since you obfuscate the contents of your security data, it could reside in your application data folder as a database file, or better, in your main app's database file in a separate table. This is easier to track than isolated text files.

_________________
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Fri Dec 14, 2018 8:23 pm 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Sun Sep 11, 2016 2:17 pm
Posts: 561
I made a more complete example :)
read/replace/add

RECORD Module:
Code:
DeclareModule RECORD
  Declare.i Access(File.s)
  Declare.i Entries()
  Declare.s Entry(Index.i,NewEntry.s = #Null$);if index is < #Null and NewEntry THE ENTRY WILL BE ADDED TO THE FILEND!
  Declare.i Close()
EndDeclareModule

Module RECORD
 
  EnableExplicit

  Structure RECORD_STRUCT
    String.s{80}
  EndStructure
 
  Global Dim Record.w(87)
  Global *Entry.RECORD_STRUCT = @Record()
  Global FileHandle.i
  Global Entries.i
 
  Procedure.i Access(File.s)
    If FileHandle
      CloseFile(FileHandle)
      Entries = #Null
    EndIf
    FileHandle = OpenFile(#PB_Any,File)
    If FileHandle
      Entries = Lof(FileHandle) / 160
      ProcedureReturn #True   
    EndIf
  EndProcedure
 
  Procedure.i Entries()
    ProcedureReturn Entries
  EndProcedure
 
  Procedure.s Entry(Index.i,NewEntry.s = #Null$)
    Protected BufferSize.i
    If FileHandle
      Index * 160
      If NewEntry
        BufferSize = StringByteLength(NewEntry)
        If Index < #Null
          FileSeek(FileHandle,Lof(FileHandle),#PB_Absolute)
          If WriteData(FileHandle,@NewEntry,BufferSize) = BufferSize
            Entries + 1
            ProcedureReturn NewEntry 
          EndIf
        Else
          If Index < Lof(FileHandle)
            FileSeek(FileHandle,Index,#PB_Absolute)   
          Else
            FileSeek(FileHandle,Lof(FileHandle),#PB_Absolute)
          EndIf
        EndIf 
        If WriteData(FileHandle,@NewEntry,BufferSize) = BufferSize
          ProcedureReturn NewEntry 
        EndIf
      Else
        If Index < Lof(FileHandle)
          FileSeek(FileHandle,Index,#PB_Absolute)
          If ReadData(FileHandle,@Record(),160) = 160
            ProcedureReturn *Entry\String
          EndIf
        EndIf
      EndIf
    EndIf
  EndProcedure
 
  Procedure.i Close()
    If FileHandle
      CloseFile(FileHandle)
    EndIf
  EndProcedure
 
EndModule

EnableExplicit

Global File.s
Global Record.i

Procedure.i CreatePseudoRecord(File.s)
  Protected FileHandle.i
  Protected Record.s
  Protected Entry.s
  Protected Entries.i
  Protected Index.i
  For Entries = 1 To 200
    Entry = #Null$
    For Index = 1 To 40
      Entry + Str(Random(99,10))
    Next
    Record + Entry
  Next
  FileHandle = CreateFile(#PB_Any,File)
  If FileHandle
    WriteData(FileHandle,@Record,StringByteLength(Record))
    CloseFile(FileHandle)
    ProcedureReturn #True
  EndIf
EndProcedure

File = "record.bin"
If CreatePseudoRecord(File)
  If RECORD::Access(File)
    Debug RECORD::Entries()
    Debug "---------------------"
    For Record = 0 To RECORD::Entries() - 1
      Debug RECORD::Entry(Record) 
    Next
    Debug "---------------------"
    Debug RECORD::Entry(66,"00000034704479755457482989933479958480184854798023139688561099699823547249762632")
    Debug RECORD::Entry(66)
    Debug RECORD::Entry(-1,"00000034704479755457482989933479958480184854798023139688561099699823547249700000")
    Debug RECORD::Entries()
    Debug RECORD::Entry(200)
    RECORD::Close()
  EndIf
EndIf

End


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Sat Dec 15, 2018 10:23 am 
Offline
Addict
Addict

Joined: Sun Sep 07, 2008 12:45 pm
Posts: 4407
Location: Germany
Hi, hi,

years ago I wrote a tool to read and and write FoxPro dbf files.
In prinzipal that's the same stuff. The only difference:
There is a header in front where the fieldnames, types and length are stored.

Simply: it's a file based database :mrgreen:


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Sat Dec 15, 2018 5:27 pm 
Offline
User
User

Joined: Mon Jul 17, 2017 7:22 am
Posts: 66
Location: Cotswolds England
skywalk wrote:
I would not think antivirus apps edit scanned files? :shock:
If they find something viral, they move them or make them hidden.

It certainly appeared that the antivirus corrupted the text file with an extension of .DBF. At the time I thought it was something to do with it being inside a Winzip executable. Whether the antivirus should have, or not done this the fact remains it did alter the file by trying to add a header. I wasn't interested to find out why or how, so I did a work-around.

skywalk wrote:
Since you obfuscate the contents of your security data, it could reside in your application data folder as a database file, or better, in your main app's database file in a separate table. This is easier to track than isolated text files.

I have been writing software since the early 1980s, so I am aware of where I can place my data. I had no need to track files as my software placed them into the folders where my other software would find them. I have successfully used a text file to control ownership and the number of users since I stopped using dongles to control licensing. As I wrote a lot of MS-Access applications I preferred to keep some critical data out of the backend databases. I am now retired but messing with PureBasic from time to time as it reminds me of Clipper. I'm not up to speed with PureBasic yet but getting there.....hopefully. It's always little things that cause big problems!

Mijikai wrote:
I made a more complete example :)
read/replace/add

Cheers for that Mijiai, quite brilliant.

Infratec wrote:
Hi, hi,
years ago I wrote a tool to read and and write FoxPro dbf files.
In prinzipal that's the same stuff. The only difference:
There is a header in front where the fieldnames, types and length are stored. Simply: it's a file based database :mrgreen:

That's what I thought at the time when I saw the file, although the header wasn't complete

_________________
If it's falling over......just remember the computer is never wrong!


Top
 Profile  
Reply with quote  
 Post subject: Re: Sequential file problem
PostPosted: Sat Dec 15, 2018 9:56 pm 
Offline
Addict
Addict
User avatar

Joined: Wed Dec 23, 2009 10:14 pm
Posts: 3110
Location: Boston, MA
Certainly, no pressure from me to use a database. I use text files also. But, reading/writing/inserting/deleting binary records while moving file position pointers is built-in to a database. I have too much code to write to reinvent that wheel. :wink:

_________________
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 16 posts ]  Go to page 1, 2  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 19 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

 


Powered by phpBB © 2008 phpBB Group
subSilver+ theme by Canver Software, sponsor Sanal Modifiye