Monitoring a directory for new files being added

[infratec]

Adjusted the code for newer PB versions

Code: Select all

EnableExplicit

;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; App Name: ReadDirectoryChanges
; Author  : Sparkie
; Date    : April 23, 2005
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; Window Enumerations
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Enumeration
  #Win_Main
EndEnumeration

;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; Gadget Enumerations
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Enumeration
  #Button_Dir
  #Text_Dir
  #ListIcon_RDCW
EndEnumeration

;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; Constants for FILE_NOTIFY_INFORMATION (*fni) actions
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
#FILE_ACTION_ADDED= 1
#FILE_ACTION_REMOVED = 2
#FILE_ACTION_MODIFIED = 3
#FILE_ACTION_RENAMED_OLD_NAME = 4
#FILE_ACTION_RENAMED_NEW_NAME = 5

;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; Constants for FILE_NOTIFY_INFORMATION offsets
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; This is how we access information when more than 1 action
; occurs for a file in our watched directory
#ActionOffset = 4
#CharLenOffset = 8
#FileNameOffset = 12

;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; FILE_NOTIFY_INFORMATION structure
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Structure FILE_NOTIFY_INFORMATION
  NextEntryOffset.l
  action.l
  FileNameLength.l
  FileName.u[1]
EndStructure


;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; Globals
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Prototype Prototype_ReadDirectoryChanges(hDirectory.i, *lpBuffer, nBufferLength.l, bWatchSubtree.l, dwNotifyFilter.l, *lpBytesReturned, *lpOverlapped, *lpCompletionRoutine)
Global ReadDirectoryChanges.Prototype_ReadDirectoryChanges


;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; Open KERNEL32.DLL for ReadDirectoryChangesW function
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
If OpenLibrary(0, "KERNEL32.DLL")
  
  ReadDirectoryChanges = GetFunction(0, "ReadDirectoryChangesW")
  
  If ReadDirectoryChanges = 0
    ; If function not found, close KERNEL32.DLL and quit
    CloseLibrary(0)
    MessageRequester("Error", "ReadDirectoryChangesW function not found.")
    End
  EndIf 
Else
  ; If KERNEL32.DLL not found, quit
  MessageRequester("Error", "KERNEL32.DLL not found.")
  End
EndIf


;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; Procedure to retrieve our directory changes
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; fniOffset is used for muliple action on 1 file
Procedure GetRDCWInfo(*fni.FILE_NOTIFY_INFORMATION)
  
  Protected action.i, timeStamp$, uniLen.i, fileName$, action$
  
  
  ; Get a time stamp for this action
  timeStamp$ = FormatDate("%mm/%dd/%yyyy %hh:%ii:%ss", Date())
  
  
  ; Get the file name
  fileName$ = PeekS(@*fni\FileName[0], *fni\FileNameLength / 2)
  ; Set the string to be displayed for the action
  Select *fni\action
    Case #FILE_ACTION_ADDED
      action$ = "was added to directory."
    Case #FILE_ACTION_REMOVED
      action$ = "was removed from directory."
    Case #FILE_ACTION_MODIFIED
      action$ = "attribute or time stamp modified."
    Case #FILE_ACTION_RENAMED_OLD_NAME
      action$ = "was renamed."
    Case #FILE_ACTION_RENAMED_NEW_NAME
      action$ = "is the new file name."
  EndSelect
  ; Add the info to the ListIconGadget
  If Not *fni\action = #FILE_ACTION_MODIFIED ; For now, I am content with knowing when an entry was created, deleted or renamed.
    AddGadgetItem(#ListIcon_RDCW, -1, fileName$ + #LF$ + action$ + #LF$ + timeStamp$)
  EndIf
EndProcedure

;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; Procedure (Thread) to set the ReadDirectoryChangesW calls
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Procedure RDCW(*FilePath)
  
  Protected volume.i, size.l, nextEntry.i, previousEntry.i, FilePath$, HDVolume$, hDir.i, *fniHelp
  Protected *fni.FILE_NOTIFY_INFORMATION
  
  
  FilePath$ = PeekS(*FilePath)
  HDVolume$ = Left(FilePath$, 1)
  
  ; Get a handle to our watched directory
  hDir = CreateFile_(FilePath$, #FILE_LIST_DIRECTORY|#GENERIC_WRITE, #FILE_SHARE_WRITE | #FILE_SHARE_READ | #FILE_SHARE_DELETE, #Null, #OPEN_EXISTING, #FILE_FLAG_BACKUP_SEMANTICS, #Null)
  
  ; Allocate memory for our FILE_NOTIFY_INFORMATION pointer
  *fni = AllocateMemory(4096) ; DWORD aligned !!!
  If *fni
    ; Loop calls to ReadDirectoryChangesW
    ; We're watching for rename, delete, create, write
    
    ;Create a file handle to the Volume being watched / not the directory but the main Volume
    volume = CreateFile_( "\\.\" + HDVolume$ + ":\0", #GENERIC_WRITE, #FILE_SHARE_WRITE, #Null, #OPEN_EXISTING, 0, #Null)
    
    ; FlushFileBuffers before call
    FlushFileBuffers_(volume)
    ; now do a call
    While ReadDirectoryChanges(hDir, *fni, 1024, #True, #FILE_NOTIFY_CHANGE_FILE_NAME | #FILE_NOTIFY_CHANGE_DIR_NAME | #FILE_NOTIFY_CHANGE_LAST_WRITE, @size, #Null, #Null)
      ; Get the info for offset 0
      *fniHelp = *fni
      GetRDCWInfo(*fniHelp)
      ; See if there are more entries for this call
      nextEntry = *fni\NextEntryOffset
      ; If so, get the info
      While nextEntry > 0
        If previousEntry = nextEntry
          Break
        EndIf
        *fniHelp + *fni\NextEntryOffset
        GetRDCWInfo(*fniHelp)
        ; See if there are more entries for this call
        nextEntry = PeekL(*fni + nextEntry)
        previousEntry = nextEntry
      Wend
      ; No more entries, go to next ReadDirectoryChangesW call after flushing buffers again
      FlushFileBuffers_(volume)
    Wend
    FreeMemory(*fni)
  EndIf
  CloseHandle_(hDir)
EndProcedure




Define event.i, myPath$, myThread.i, quit.i


;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
; Main window and gadgets
;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
If OpenWindow(0, 0, 0, 545, 350, "ReadDirectoryChangesW", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ButtonGadget(#Button_Dir, 5, 5, 150, 20, "Select Directory to watch")
  TextGadget(#Text_Dir, 5, 5, 535, 20, "")
  DisableGadget(#Text_Dir, 1)
  ListIconGadget(#ListIcon_RDCW, 5, 40, 535, 300, "File", 200, #PB_ListIcon_FullRowSelect | #PB_ListIcon_AlwaysShowSelection | #PB_ListIcon_GridLines)
  AddGadgetColumn(#ListIcon_RDCW, 1, "Action", 200)
  AddGadgetColumn(#ListIcon_RDCW, 2, "Time", 130)
  
  
  ;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  ; Main event loop
  ;>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  Repeat
    event = WaitWindowEvent()
    Select event
      Case #PB_Event_Gadget
        Select EventGadget()
          Case #Button_Dir
            ; Button pressed for getting directory to watch
            myPath$ = PathRequester("Select a Folder to watch", "c:\")
            ; If there is a valid path, start our ReadDirectoryChangesW thread
            If myPath$
              ; check for drive letter
              If Mid(myPath$, 2, 1) <> ":"
                CloseLibrary(0)
                MessageRequester("Error", "Only monitor harddrives whith letters no UNC for now.")
                End
              EndIf
              DisableGadget(#Button_Dir, 1)
              
              ; --> Our thread for change notifications with the directory handle
              ;myThread = CreateThread(@RDCW(), hDir)
              myThread = CreateThread(@RDCW(), @myPath$)
              DisableGadget(#Text_Dir, 0)
              SetGadgetText(#Text_Dir, myPath$ + " is being watched.")
            EndIf
        EndSelect
        
      Case #PB_Event_CloseWindow
        ; Kill our ReadDirectoryChangesW thread
        KillThread(myThread)
        
        ; If KERNEL32.DLL is open, close it
        If IsLibrary(0)
          CloseLibrary(0)
        EndIf
        quit = #True
        
    EndSelect
  Until quit
  
EndIf

[Kwai chang caine]

Thanks INFRATEC (and Sparkie) very usefull for me

[BarryG]

Infratec, your update of Sparkie's code is brilliant (thanks!) but is there a way to stop it logging the actions of the logging app itself? I only want to see file activity from other exes, and not my own. For example, when the user saves my exe's settings, I don't want to see an action of "MySettings.ini" being logged as a new file created by that same exe. Possible?

[infratec]

You have to inspect

Code: Select all

fileName$

inside

Code: Select all

GetRDCWInfo()

If the filename is one of yours, simply ignore it.

[Marc56us]

Hi all,

I take advantage of this old topic to put here a small tool that I have updated. It's pure Basic: no API.

It detects new or modified files based on the A(Archive) attribute which is automatically set by Windows (and DOS) as soon as a file is created or modified.

It's up to you to add an interface and even a shutdown system.

Code: Select all

; Monitor_Dir.pb - Marc56us - 2021/02/14 - PB any version (Windows only)
; https://www.purebasic.fr/english/viewtopic.php?p=565741#p565741
; Monitoring a directory for new files being added
; Monitor directory for New or Modified files
; Using systeme attibute A (Archive)
; Windows Only

EnableExplicit

Define Directory$ = GetTemporaryDirectory()
Define FileSpecs$ = "*.*"

Debug "Monitoring: " + Directory$
Debug "New file or modified file:" 
Repeat
    Delay(1000)
    ExamineDirectory(0, Directory$, FileSpecs$)
    While NextDirectoryEntry(0)
        If DirectoryEntryAttributes(0) = #PB_FileSystem_Archive 
            Define File$        = DirectoryEntryName(0)
            Define FullName$    = Directory$ + File$
            
            If Not SetFileAttributes(FullName$, #PB_FileSystem_Normal)
                Debug "Erreur changing attribut for: " + File$
                End
            Else
                Debug FormatDate("%yyyy/%dd/%mm %hh:%ii:%ss - ", Date()) + File$
            EndIf
            
        EndIf
        SetFileAttributes(FullName$, #PB_FileSystem_Normal)    
    Wend
ForEver

End

Windows Only (because of attribute archive)
No sub-directory

Feel free to use and adapt.

[infratec]

Yes, this works too, but ...
Did you look at your HDD LED?
1 second (in worst case) is not immediately.
If you remove the Delay, what's about your CPU load?
Other HDD access are slowed down, becuase the heads have always to move.
(Ok, maybe you only use SSDs, then that reason is obsolete, but still then the commands has to be executed)

Sorry, but all in all, this is not a good solution.
This were the reasons to create internal OS functions to avoid this HDD access orgy.

For linux I have an equivalent to the windows API (inotify).
viewtopic.php?p=341626#p341626

OSX is not my world, but maybe the linux way is there also possible

[Marc56us]

Loop is here only for testing.

I do like this one time in a part of some program like in old time when using command like XCOPY or BACKUP
(check and optional change archive attrib)

Yes you are right this is not a monitoring method, just a simple check for new and modified files

[BarryG]

You have to inspect fileName$ inside GetRDCWInfo().
If the filename is one of yours, simply ignore it.

Yeah, true, but then I have to keep a list of my own files. Was hoping there'd be a way to bulk-ignore any from my process.

I take advantage of this old topic to put here a small tool that I have updated. It's pure Basic: no API.
It detects new or modified files based on the A(Archive) attribute which is automatically set by Windows (and DOS) as soon as a file is created or modified.

But what about deleted/moved files? That's why I need Sparkie's/Infratec's code.

[thanos]

Hello.
Is it possible to fire an event in case of edit a file (e.g. contents of a text file)?

[BarryG]

Is it possible to fire an event in case of edit a file (e.g. contents of a text file)?

It's in the first post. Just remove the If/Then test where #FILE_ACTION_MODIFIED is purposely ignored (lines 110 and 112).

[thanos]

@BarryG
I did not read the first post, I followed the shortcut
Regards

[highend]

Does anyone have a version (that is up to date for newer PB versions) which is still able to monitor changes on UNC paths as well?