Scintilla mit Universal-lexer (MAC/LINUX/WIN)

Hier könnt Ihr gute, von Euch geschriebene Codes posten. Sie müssen auf jeden Fall funktionieren und sollten möglichst effizient, elegant und beispielhaft oder einfach nur cool sein.
GPI
Beiträge: 1511
Registriert: 29.08.2004 13:18
Kontaktdaten:

Scintilla mit Universal-lexer (MAC/LINUX/WIN)

Beitrag von GPI »

Edit:
Der erste Post ist veraltet, bitte in zweiten weiterlesen.
Ich möchte allerdings das erste Beispiel da lassen, da es vielleicht etwas leichter verständlich ist.

----------------------------------------------------------------------------------------

Nachdem ich jetzt ja LUA in PureBasic nutzen kann, möchte ich auch gerne eine einfache Codeeingabe in Programm ermöglichen. Scintilla ist da ja perfekt. Blöderweise sind bei PB die ganzen eingebauten Lexer entfernt worden und nur in Windows könnte man eine SciLexer dll bekommen (wobei man hier schwer eine 64BIT-Version bekommt).

Ich hab mir mal den "Container"-Lexer genauer angeschaut und eine Formatierung für LUA eingebaut. Der Code ist in zwei Teile unterteilt. Einmal das Scintilla-Modul, das sich allgemein um Scintilla kümmert und Schnittstellen für den Lexer schafft und einmal den LUA-Lexer. Das ganze ist so gestalltet, das man einfach weitere Lexer hinzufügen kann.

Das ganze sollte unter Windows, Mac und Linux laufen, letztere beiden nicht getestet.

WICHTIG:
Scintilla::Init() kopiert, wenn der Debugger an ist, die scintilla.dll von Compiler in das aktuelle Verzeichnis und benennt sie in Scintilla32/64.dll um. Ich bin zu faul, die immer manuell rauszusuchen :)

Code: Alles auswählen

;-* 
;-* scintilla
;-*

DeclareModule Scintilla
  EnableExplicit
  ;-** Structure
  Structure Color
    fore.i
    back.i
  EndStructure
  Structure FontStyle
    fore.i
    back.i
    FontStyle.i; combination form #pb_font_bold, #pb_font_italic, #pb_font_underline. #PB_Font_StrikeOut is used for "eol filled"
    font.s
    size.i
  EndStructure
  Structure ColorAlpha
    color.i
    alpha.i
  EndStructure
  Structure LexerStyle; used to store the Style for the lexer
    None.FontStyle
    String.FontStyle
    SingleString.FontStyle
    StringBlock.FontStyle
    CommentBlock.FontStyle
    Comment.FontStyle
    Number.FontStyle
    AlphaNum.FontStyle
    Keyword.FontStyle
    SpecialKeyword.FontStyle
    LineNumber.FontStyle
    Label.FontStyle
    FolderMark.Color
    RepeatedSelection.ColorAlpha
    Brace.ColorAlpha
    BadBrace.ColorAlpha
    Currentline.ColorAlpha
    fold.i
    AutoIndention.i
    TabSize.i
    MinLinenumberSize.i
  EndStructure
  
  Prototype prot_LexerCallback(gadget,*scinotify.SCNotification)
  Prototype prot_LexerInit(gadget,*LexerStyle.LexerStyle)
  Structure LexerInfo; for the lexer - use()-Command
    callback.prot_LexerCallback
    init.prot_LexerInit
  EndStructure
  
  ;-** Declare
  Declare init()
  Declare GetDefault(*Lexer.LexerStyle)
  Declare SetStyle(Gadget,*lexer.LexerStyle)
  Declare Gadget(id,x,y,cx,cy,*lexer=0,*style.lexerstyle=0)
  Declare SetText(gadget,text.s)
  Declare AppendText(gadget,text.s)
  Declare.s GetText(gadget)
  ;for use in Lexer
  Prototype prot_FindWordStart(gadget,endpos)
  Declare  _Autocomplete(gadget,FindStart.prot_findwordstart,Map special())  
  Declare _CallBack(Gadget, *scinotify.SCNotification)
  Declare _fastUTF8(string.s="")
  CompilerIf #PB_Compiler_Debugger
    Declare _SetProperty(gadget,Key,Value)  
  CompilerEndIf
  Declare _GetPropertyFlag(gadget,Key)
  Declare _SetPropertyFlag(gadget,Key,Value)
  Declare _Indention(gadget)
  Declare _MarkBrace(gadget,brace.s)
  
  ;-** Constants
  ;-{ AutoIndention
  Enumeration 0
    #AutoIndention_none
    #AutoIndention_simple
    #AutoIndention_complex
  EndEnumeration
  ;}
  ;-{ Property 
  Enumeration #STYLE_MAX -1
    #Property_flag
    #Property_MinLinenumberSize
    #Property_AutoIndention
  EndEnumeration
  EnumerationBinary
    #PropertyFlag_Fold
    #PropertyFlag_RepeatedSelection
    #PropertyFlag_Brace
  EndEnumeration  
  ; container-lexer doesn't support sci_set/getporperty - so i used a style to save some flag/values as style
  ;}
  ;-{ Style
  Enumeration 1
    ;#Style_Default - set by PB
    #style_String
    #style_SingleString
    #style_StringBlock
    #style_CommentBlock
    #style_Comment
    #style_Number
    #style_AlphaNum
    #style_Keyword
    #style_SpecialKeyword
    #style_Label
  EndEnumeration
  ;}
  ;-{ Indicator
  Enumeration 1
    #indicator_selection
    #indicator_brace
    #indicator_badbrace
  EndEnumeration
  ;}
  ;-{ linestate
  ;                        12345678
  #linestate_submask          =$00ff0000
  #linestate_subshift         =16
  #linestate_start            =$01000000
  #linestate_CommentBlock          =$02000000
  #linestate_QuoteBlock            =$04000000
  #linestate_IndentBeforeMask =$0000F000
  #linestate_IndentBeforeShift=4+4+4
  #linestate_IndentAfterMask  =$00000F00
  #linestate_IndentAfterShift =4+4
  #linestate_IndentBase       =$7
  #linestate_IndentLevelMask  =$000000FF
  ; the flags commentblock and quoteblock are used to mark then lines. In combination with start it indicates the first line of the block
  ; submask is used to mask a sub-counter - in lua the block quote/comments have "Levels" for example [[ ]]  [=[ ]=] the sub-counter indicate the level of the block
  ; Indent before and after is the same like in purebasic editor and can range from -7 to 7 (IndentBase is added before storing)
  ; IndentLevelMask is used to store the indention in a range from 0 to 255
  ;}
  ;-{ Margin
  Enumeration 0
    #margin_linenumber
    #margin_fold
  EndEnumeration
  
  ;-** global macros
  Macro _GetProperty(gadget,Key)
    sci(gadget,#SCI_STYLEGETFORE,key)
  EndMacro
  CompilerIf Not #PB_Compiler_Debugger
    Macro _SetProperty(gadget,key,value)
      sci(gadget,#SCI_STYLESETFORE,key,value)
    EndMacro
  CompilerEndIf  
EndDeclareModule

Module Scintilla
  ;-** Macros  
  Macro SCI(Gadget,Msg,a=0,b=0)
    ScintillaSendMessage(Gadget,Msg,a,b)
  EndMacro     
  Macro __SetStyle(name,vFore=-1,vBack=-1,vFontstyle=-1,vFont="",vSize=0)
    name\fore=vFore
    name\back=vBack
    name\FontStyle=vFontStyle
    name\font=vFont
    name\size=vSize
  EndMacro
  
  ;-** internal routines
  Procedure __SetFontStyle(id,style,*FontStyle.FontStyle)
    If *FontStyle\fore>=0
      sci(id,#SCI_STYLESETFORE,style,*FontStyle\fore)
    EndIf
    If *FontStyle\back>=0
      sci(id,#SCI_STYLESETBACK,style,*FontStyle\back)
    EndIf
    If *FontStyle\FontStyle>=0
      sci(id,#SCI_STYLESETBOLD,style,Bool(*FontStyle\FontStyle & #PB_Font_Bold))
      sci(id,#SCI_STYLESETITALIC,style,Bool(*FontStyle\FontStyle & #PB_Font_Italic))
      sci(id,#SCI_STYLESETUNDERLINE,style,Bool(*FontStyle\FontStyle & #PB_Font_Underline))
      sci(id,#SCI_STYLESETEOLFILLED,style,Bool(*FontStyle\FontStyle & #PB_Font_StrikeOut))
    EndIf
    If *FontStyle\size>0
      sci(id,#SCI_STYLESETSIZE,style,*FontStyle \size)
    EndIf
    If *FontStyle\font<>""
      sci(id,#SCI_STYLESETFONT,style,_fastUTF8(*FontStyle\font))
    EndIf  
  EndProcedure
  Procedure __setLineWidth(gadget)
    Protected maxlines,width,digit,min
    maxlines=sci(gadget,#SCI_GETLINECOUNT)
    min=_GetProperty(gadget,#Property_MinLinenumberSize)
    
    While maxlines>0
      maxlines/10
      digit+1
    Wend
    If digit<min
      digit=min
    EndIf
    
    width=sci(gadget,#SCI_TEXTWIDTH,#STYLE_LINENUMBER, @"9");because i send only one char, unicode=ascii
    width*(digit+1)
    
    If sci(Gadget,#SCI_GETMARGINWIDTHN,#margin_linenumber)<>width
      sci(gadget,#SCI_SETMARGINWIDTHN,#margin_linenumber,width)
    EndIf
  EndProcedure
  Procedure __RepeatedSelection(gadget)
    Static indicator_set,startPos,endPos
    Protected *buf.ascii,buflength,size,i
    
    size=sci(gadget,#SCI_GETLENGTH)
    sci(gadget,#SCI_SETINDICATORCURRENT,#indicator_selection)
    
    If indicator_set
      sci(gadget, #SCI_INDICATORCLEARRANGE,0,size)
      indicator_set=#False
    EndIf
    
    startPos=sci(gadget,#SCI_GETSELECTIONSTART)
    endpos=sci(gadget,#SCI_GETSELECTIONEND)
    
    If startPos=sci(gadget,#SCI_WORDSTARTPOSITION,startpos,#True) And endpos=sci(gadget,#SCI_WORDENDPOSITION,startPos,#True)
      buflength=sci(gadget,#SCI_GETSELTEXT)
      If buflength>2
        *buf=AllocateMemory(buflength)
        
        If *buf          
          sci(gadget,#SCI_GETSELTEXT,0,*buf)
          
          sci(gadget,#SCI_SETTARGETSTART,0)
          sci(gadget,#SCI_SETTARGETEND, size )
          sci(gadget,#SCI_SETSEARCHFLAGS, #SCFIND_MATCHCASE|#SCFIND_WHOLEWORD)          
          
          Repeat
            i= sci(gadget,#SCI_SEARCHINTARGET,buflength-1,*buf)
            If i<0
              Break
            EndIf
            
            If i<>startPos
              sci(gadget,#SCI_INDICATORFILLRANGE,i,buflength-1)
            EndIf            
            
            sci(gadget,#SCI_SETTARGETSTART,i+buflength-1)
            sci(gadget,#SCI_SETTARGETEND, size )
          ForEver
          
          indicator_set=#True
          
          FreeMemory(*buf)
        EndIf
      EndIf
      
    EndIf            
  EndProcedure
  
  ;-** procedures
  Procedure init()
    ;with Debugger - copy the Scintilla.dll in the main directory 
    ;Initalize the scintilla-gadget by loading the scintilla32/scintilla64 dll or as fallback scintilla.dll/scilexer.dll
    CompilerIf #PB_Compiler_OS=#PB_OS_Windows
      CompilerIf #PB_Compiler_Processor=#PB_Processor_x86
        #DLL_Scintilla="Scintilla32.dll"
      CompilerElse
        #DLL_Scintilla="Scintilla64.dll"
      CompilerEndIf
      
      CompilerIf #PB_Compiler_Debugger
        If FileSize(#DLL_Scintilla)<>FileSize(#PB_Compiler_Home+"Compilers\Scintilla.dll")
          CopyFile(#PB_Compiler_Home+"Compilers\Scintilla.dll",#DLL_Scintilla)
        EndIf
      CompilerEndIf
      If InitScintilla(#DLL_Scintilla)=0
        If InitScintilla("Scintilla.dll")=0
          If InitScintilla("SciLexer.dll")=0
            ProcedureReturn #False
          EndIf
        EndIf
      EndIf      
    CompilerEndIf
    ProcedureReturn #True
  EndProcedure
  Procedure SetText(Gadget,text.s)
    ;Quick (and dirty) method to set a long Text, because SetGadgetText doesn't work
    sci(Gadget,#SCI_SETTEXT,0,_fastUTF8(text))
    _fastUTF8()
  EndProcedure
  Procedure AppendText(Gadget,text.s)
    ;Append the text without scrolling to the position
    sci(Gadget,#SCI_APPENDTEXT,StringByteLength(text,#PB_UTF8),_fastUTF8(text))
    _fastUTF8()
  EndProcedure
  Procedure.s GetText(Gadget)
    ;GetGadgetText doesn't work (because it use unicode not ascii)
    Protected *buf,bufsize,ret.s
    bufsize=sci(gadget,#SCI_GETLENGTH)+1
    *buf=AllocateMemory(bufsize)
    If *buf
      sci(Gadget,#SCI_GETTEXT,bufsize,*buf)
      ret=PeekS(*buf,-1,#PB_UTF8)
      FreeMemory(*buf)
    EndIf
    ProcedureReturn ret
  EndProcedure
  Procedure GetDefault(*Lexer.LexerStyle)
    ;Get Default Style 
    Protected back=$DFFFFF
    Protected backMargin=$dfefef
    Protected backgrey=$DFDFDF
    __SetStyle(*Lexer\None          ,$000000,back,0,"Courier New",10)
    __SetStyle(*Lexer\String        ,$ff7f00)
    __SetStyle(*lexer\SingleString  ,$ff7f00)
    __SetStyle(*lexer\StringBlock   ,$ff7f00,backgrey,#PB_Font_StrikeOut)
    __SetStyle(*Lexer\Comment       ,$AAAA00)
    __SetStyle(*Lexer\CommentBlock  ,$AAAA00,backgrey,#PB_Font_StrikeOut)
    __SetStyle(*Lexer\Number        ,$333300)
    __SetStyle(*Lexer\AlphaNum      ,$666600)
    __SetStyle(*Lexer\Keyword       ,$666600,back,#PB_Font_Bold)
    __SetStyle(*Lexer\SpecialKeyword,$000000,back,#PB_Font_Bold)
    __SetStyle(*Lexer\LineNumber    ,$666666,backMargin)
    __SetStyle(*Lexer\Label         ,$ff00ff)
    *Lexer\FolderMark\fore=$ffffff
    *Lexer\FolderMark\back=$000000
    *Lexer\RepeatedSelection\color=$006600
    *Lexer\RepeatedSelection\alpha=50
    *Lexer\Brace\color=$006600
    *lexer\brace\alpha=50
    *Lexer\BadBrace\color=$0000ff
    *lexer\BadBrace\alpha=128
    *lexer\Fold=#True
    *lexer\AutoIndention=#AutoIndention_complex
    *Lexer\TabSize=2
    *Lexer\Currentline\color=$afFFFF
    *Lexer\Currentline\alpha=#SC_ALPHA_NOALPHA
    *lexer\MinLinenumberSize=3
  EndProcedure
  Procedure SetStyle(Gadget,*lexer.LexerStyle)
    ;Set Style for a Scintilla-Gadget
    Protected width
    With *lexer
      __SetFontStyle(Gadget,#STYLE_DEFAULT,\None)
      ScintillaSendMessage(Gadget,#SCI_STYLECLEARALL)
      
      __SetFontStyle(Gadget,#style_String,\string)
      __SetFontStyle(Gadget,#style_SingleString,\SingleString)
      __setFontStyle(Gadget,#style_StringBlock,\StringBlock)
      __SetFontStyle(Gadget,#style_CommentBlock,\CommentBlock)
      __SetFontStyle(Gadget,#style_Comment,\Comment)
      __SetFontStyle(Gadget,#style_Number,\Number)
      __SetFontStyle(Gadget,#style_AlphaNum,\AlphaNum)
      __SetFontStyle(Gadget,#style_keyword,\Keyword)
      __SetFontStyle(Gadget,#style_SpecialKeyword,\SpecialKeyword)
      __SetFontStyle(Gadget,#STYLE_LINENUMBER,\LineNumber)
      __SetFontStyle(Gadget,#style_Label,\Label)
      sci(Gadget,#SCI_SETFOLDMARGINCOLOUR,#margin_fold,\LineNumber\back)
      sci(Gadget,#SCI_SETFOLDMARGINHICOLOUR,#margin_fold,\LineNumber\back)
      
      sci(Gadget, #SCI_MARKERSETFORE, #SC_MARKNUM_FOLDER,\FolderMark\fore)
      sci(Gadget, #SCI_MARKERSETBACK, #SC_MARKNUM_FOLDER, \FolderMark\back)
      sci(Gadget, #SCI_MARKERSETFORE, #SC_MARKNUM_FOLDEROPEN,\FolderMark\fore)
      sci(Gadget, #SCI_MARKERSETBACK, #SC_MARKNUM_FOLDEROPEN, \FolderMark\back)
      sci(Gadget, #SCI_MARKERSETFORE, #SC_MARKNUM_FOLDEROPENMID, \FolderMark\fore)
      sci(Gadget, #SCI_MARKERSETBACK, #SC_MARKNUM_FOLDEROPENMID, \FolderMark\back)
      sci(Gadget, #SCI_MARKERSETFORE, #SC_MARKNUM_FOLDERSUB, \FolderMark\fore)
      sci(Gadget, #SCI_MARKERSETBACK, #SC_MARKNUM_FOLDERSUB, \FolderMark\back)
      sci(Gadget, #SCI_MARKERSETFORE, #SC_MARKNUM_FOLDERTAIL, \FolderMark\fore)
      sci(Gadget, #SCI_MARKERSETBACK, #SC_MARKNUM_FOLDERTAIL, \FolderMark\back)
      sci(Gadget, #SCI_MARKERSETFORE, #SC_MARKNUM_FOLDERMIDTAIL,\FolderMark\fore)
      sci(Gadget, #SCI_MARKERSETBACK, #SC_MARKNUM_FOLDERMIDTAIL, \FolderMark\back)
      
      sci(gadget,#SCI_INDICSETSTYLE,#indicator_selection,#INDIC_STRAIGHTBOX);fullbox
      sci(gadget,#SCI_INDICSETFORE,#indicator_selection,\RepeatedSelection\color)
      sci(gadget,#SCI_INDICSETALPHA,#indicator_selection,\RepeatedSelection\alpha)
      sci(gadget,#SCI_INDICSETUNDER,#indicator_selection,#True)
      
      sci(gadget,#SCI_INDICSETSTYLE,#indicator_brace,#INDIC_STRAIGHTBOX);fullbox
      sci(gadget,#SCI_INDICSETFORE,#indicator_brace,\Brace\color)
      sci(gadget,#SCI_INDICSETALPHA,#indicator_brace,\Brace\alpha)
      sci(gadget,#SCI_INDICSETUNDER,#indicator_brace,#True)
      
      sci(gadget,#SCI_INDICSETSTYLE,#indicator_badbrace,#INDIC_STRAIGHTBOX);fullbox
      sci(gadget,#SCI_INDICSETFORE,#indicator_badbrace,\badBrace\color)
      sci(gadget,#SCI_INDICSETALPHA,#indicator_badbrace,\badBrace\alpha)
      sci(gadget,#SCI_INDICSETUNDER,#indicator_badbrace,#True)
      
      _SetPropertyFlag(gadget,#PropertyFlag_Fold,\fold)
      _SetPropertyFlag(gadget,#PropertyFlag_RepeatedSelection,Bool(\RepeatedSelection\alpha))
      If \fold 
        width=sci(gadget,#SCI_TEXTWIDTH,#STYLE_LINENUMBER,@"9")*2
        ScintillaSendMessage(Gadget,#SCI_SETMARGINWIDTHN,#margin_fold, width)
      Else
        ScintillaSendMessage(Gadget,#SCI_SETMARGINWIDTHN,#margin_fold, 0)
      EndIf
      _SetPropertyFlag(gadget,#PropertyFlag_Brace,Bool(\Brace\alpha Or \BadBrace\alpha))
      _SetProperty(gadget,#Property_AutoIndention,\AutoIndention)    
      _SetProperty(gadget,#Property_MinLinenumberSize,\MinLinenumberSize)
      
      sci(gadget,#SCI_SETTABWIDTH,\TabSize)
      
      sci(gadget,#SCI_SETCARETLINEVISIBLE,Bool( \Currentline\alpha))
      sci(gadget,#SCI_SETCARETLINEBACK,\Currentline\color)
      sci(Gadget,#SCI_SETCARETLINEBACKALPHA,\Currentline\alpha)
        
      
    EndWith
  EndProcedure
  Procedure Gadget(id,x,y,cx,cy,*lexer.LexerInfo=0,*style.lexerstyle=0)
    ;Create a Scintilla gadget. *lexer-info should be a lexer.use()-call
    Protected Gadget , keyword.s
    Protected defaultstyle.lexerstyle,defaultlexer.lexerinfo
    Protected ret
    
    If *style=0
      GetDefault(defaultstyle)
      *style=defaultstyle
    EndIf
    If *lexer=0
      defaultlexer\callback=@_CallBack()
      *lexer\callback=defaultlexer
    EndIf
    
    ret = ScintillaGadget(id,x,y,cx,cy,*lexer\callback)
    
    If id <> #PB_Any
      Gadget = id
    Else
      Gadget = ret
    EndIf
    
    ; Use Indicators for Brace-Highlight
    sci(gadget,#SCI_BRACEHIGHLIGHTINDICATOR,#True,#indicator_brace)
    sci(gadget,#SCI_BRACEBADLIGHTINDICATOR,#True,#indicator_badbrace)
    
    ; Fix the horizontal scrolling bar
    sci(gadget,#SCI_SETSCROLLWIDTHTRACKING,#True)
    sci(gadget,#SCI_SETSCROLLWIDTH,100)
    
    ; Scroll before the Cursor reach the boarder
    sci(gadget,#SCI_SETXCARETPOLICY,#CARET_SLOP|#CARET_EVEN|#CARET_STRICT 	,100)
    sci(gadget,#SCI_SETYCARETPOLICY,#CARET_SLOP|#CARET_EVEN|#CARET_STRICT 	,3)
    
    ; Set fold-Margin
    ScintillaSendMessage(Gadget,#SCI_SETMARGINTYPEN,  #margin_fold, #SC_MARGIN_SYMBOL )
    ScintillaSendMessage(Gadget,#SCI_SETMARGINMASKN,  #margin_fold, #SC_MASK_FOLDERS)
    ScintillaSendMessage(Gadget,#SCI_SETMARGINSENSITIVEN, #margin_fold, #True)
    
    ; Fold-Symbols
    ScintillaSendMessage(Gadget,#SCI_MARKERDEFINE, #SC_MARKNUM_FOLDEROPEN     , #SC_MARK_BOXMINUS)
    ScintillaSendMessage(Gadget,#SCI_MARKERDEFINE, #SC_MARKNUM_FOLDER         , #SC_MARK_BOXPLUS)
    ScintillaSendMessage(Gadget,#SCI_MARKERDEFINE, #SC_MARKNUM_FOLDERSUB      , #SC_MARK_VLINE)
    ScintillaSendMessage(Gadget,#SCI_MARKERDEFINE, #SC_MARKNUM_FOLDERTAIL     , #SC_MARK_LCORNER)
    ScintillaSendMessage(Gadget,#SCI_MARKERDEFINE, #SC_MARKNUM_FOLDEREND      , #SC_MARK_BOXPLUSCONNECTED)
    ScintillaSendMessage(Gadget,#SCI_MARKERDEFINE, #SC_MARKNUM_FOLDEROPENMID  , #SC_MARK_BOXMINUSCONNECTED)
    ScintillaSendMessage(Gadget,#SCI_MARKERDEFINE, #SC_MARKNUM_FOLDERMIDTAIL  , #SC_MARK_TCORNER)
        
    ; Automatic fold handling
    sci(Gadget,#SCI_SETAUTOMATICFOLD,#SC_AUTOMATICFOLD_SHOW|#SC_AUTOMATICFOLD_CLICK|#SC_AUTOMATICFOLD_CHANGE)
    
    
    If ret And *lexer\init
      *lexer\init(gadget,*style)
    EndIf
    
    ProcedureReturn ret
    
  EndProcedure  
  ;-** Procedures for Lexer
  Procedure _fastUTF8(str.s="")
    ; Easy method for convert to UTF8 without allocate & free memory for each call
    Static *buf,bufsize
    Protected needed
    If str=""
      If *buf
        FreeMemory(*buf)
        *buf=0
      EndIf
      ProcedureReturn 0
    EndIf
    
    needed=StringByteLength(str,#PB_UTF8)+1;null-terminated 
    If *buf=0 Or bufsize<needed
      If *buf
        FreeMemory(*buf)
      EndIf
      
      bufsize=needed:If bufsize<1024:bufsize=1024:EndIf
      *buf=AllocateMemory(bufsize)
    EndIf
    
    If *buf
      PokeS(*buf,str,-1,#PB_UTF8)
    EndIf
        
    ProcedureReturn *buf
  EndProcedure
  ;fakeproperty, because the container-lexer doesn't support properties
  CompilerIf #PB_Compiler_Debugger
    Procedure _SetProperty(gadget,key,value)
      ;Procedurevariant for the _SetProperty-Macro to add a range-check for value
      If value<0 Or value>$ffffff
        Debug "WARNING: SetProperty - value is not in 0-$ffffff ("+Hex(value)+")"
      EndIf
      ProcedureReturn sci(gadget,#SCI_STYLESETFORE,key,value)
    EndProcedure
  CompilerEndIf
  Procedure _GetPropertyFlag(gadget,key)
    ;Get a flag in the #proerty_flag "style"
    ProcedureReturn Bool(_GetProperty(gadget,#Property_flag)&key)
  EndProcedure
  Procedure _SetPropertyFlag(gadget,key,value)
    ;Set a Flag in the #proerty_flag "style"
    Protected store=_GetProperty(gadget,#Property_flag)
    If value
      store | key
    Else
      store & ~key
    EndIf
     _SetProperty(gadget,#Property_flag,store)
  EndProcedure
  Procedure _CallBack(Gadget, *scinotify.SCNotification)
    ;default callback for scintilla-gadget. The Lexer-callback should call this to do some 
    ;basic-stuff like repeated_selection, width-change of the linenumber-collumn
    Protected lineNumber,width
    
    Select *scinotify\nmhdr\code
        
      Case #SCN_UPDATEUI
        If *scinotify\updated=#SC_UPDATE_SELECTION
          If _GetPropertyFlag(gadget,#PropertyFlag_RepeatedSelection)
            __RepeatedSelection(gadget)
          EndIf          
        EndIf
        
      Case #SCN_ZOOM
        ;correct the linenumber and fold-margin
        __setLineWidth(gadget)
        If _GetPropertyFlag(gadget,#PropertyFlag_Fold)
          width=sci(gadget,#SCI_TEXTWIDTH,#STYLE_LINENUMBER,@"9")*2
          ScintillaSendMessage(Gadget,#SCI_SETMARGINWIDTHN, #margin_fold, width)
        EndIf
        
      Case #SCN_MODIFIED
        ;check if the linenumber-margin is width engouht
        If (*scinotify\modificationType  & (#SC_MOD_INSERTTEXT|#SC_MOD_DELETETEXT)) And *scinotify\linesAdded
          __setLineWidth(gadget)
        EndIf
        
      Case #SCN_MARGINCLICK
        ;Disabled by automatic handling
;         Debug "Marginclick:"+*scinotify\margin
;         If *scinotify\margin = #margin_fold
;           lineNumber = ScintillaSendMessage(Gadget,#SCI_LINEFROMPOSITION,*scinotify\position)
;           ScintillaSendMessage(Gadget,#SCI_TOGGLEFOLD, lineNumber, 0)
;         EndIf
        
    EndSelect
  EndProcedure
  Procedure _Indention(gadget)
    ;simple method to add an automatic indention. Simple copy the indention of the previous line
    ;should be called after a cr/lf char is added to the documnet / callback
    Protected lineNumber,mode,*buf,bufsize,*buf2.ascii,pos
    
    pos=sci(gadget,#SCI_GETCURRENTPOS)
    lineNumber=sci(gadget,#SCI_LINEFROMPOSITION,pos)
    If lineNumber>0
      bufsize=sci(gadget,#SCI_GETLINE,lineNumber-1,0)+1
      
      *buf=AllocateMemory(bufsize)
      If *buf
        sci(gadget,#SCI_GETLINE,lineNumber-1,*buf)
        *buf2=*buf
        While *buf2\a=32 Or *buf2\a=9 ;space und tab
          *buf2+1
        Wend
        *buf2\a=0
        sci(gadget,#SCI_REPLACESEL,0,*buf)
        FreeMemory (*buf)
      EndIf
    EndIf
    
  EndProcedure
  Procedure _MarkBrace(gadget,brace.s)
    ;check if under the current position is a brace and hilight it
    ;brace is a combination from "(){}[]<>"
    Protected pos,char,findpos
    Static doRemove
    pos=sci(gadget,#SCI_GETCURRENTPOS)
    If pos=sci(gadget,#SCI_GETANCHOR)
      
      char=sci(gadget,#SCI_GETCHARAT,pos)
      If FindString(brace,Chr(char))
        findpos=sci(gadget,#SCI_BRACEMATCH,pos)
        If findpos>=0
          sci(gadget,#SCI_BRACEHIGHLIGHT,pos,findpos)
        Else
          sci(gadget,#SCI_BRACEBADLIGHT,pos)
        EndIf
        doremove=#True
      ElseIf doremove
        sci(gadget,#SCI_BRACEHIGHLIGHT,-1,-1)
        sci(gadget,#SCI_BRACEBADLIGHT,-1)
        doRemove=#False
      EndIf
    ElseIf doremove
      sci(gadget,#SCI_BRACEHIGHLIGHT,-1,-1)
      sci(gadget,#SCI_BRACEBADLIGHT,-1)
      doRemove=#False
    EndIf
    
  EndProcedure
  Procedure _Autocomplete(gadget,FindStart.prot_findwordstart,Map special())    
    ; used from the lexer for the autocomplete-box
    ; special is a MAP, the mapkey contain the words for the list. 
    ; A map is used, because with this map it is easier to detect, if a word is a special word for styling
    Protected startPos,endPos,i,char,range.ScTextRange,word.s
    NewList words.s()
    Protected wordlist.s
    
    If sci(gadget,#SCI_AUTOCACTIVE)=#False
      endPos=sci(gadget,#SCI_GETCURRENTPOS)
      
      startPos=FindStart(gadget,endpos)
      
      If endpos-startPos>1
        
        range\chrg\cpMin=startpos
        range\chrg\cpMax=endPos
        range\lpstrText=AllocateMemory(endpos-startpos+1)
        
        sci(gadget,#SCI_GETTEXTRANGE,0,range)
        word=PeekS(range\lpstrText,-1,#PB_UTF8)
        FreeMemory(range\lpstrText)

        ForEach special()
          If Left(MapKey(special()),Len(word))=word
            AddElement(words())
            words()=MapKey(special())
          EndIf
        Next
        
        If ListSize(words())>0 ; And ListSize(words())<30
          SortList(words(),#PB_Sort_Ascending)
          ForEach words()
            wordlist+" "+words()
          Next
          sci(gadget,#SCI_AUTOCSHOW,endpos-startpos,_fastUTF8(Mid(wordlist,2)))
          _fastUTF8()
        EndIf
      EndIf      
      
    EndIf
    
  EndProcedure
EndModule

;-
;- Lua_Lexer
;-
DeclareModule Lua_Lexer
  EnableExplicit
  Declare use()
EndDeclareModule
Module Lua_Lexer
  UseModule scintilla
  Declare Highlight(Gadget,startPos,endPos)
  
  Global NewMap keyword()
  Global NewMap foldstart()
  Global NewMap foldend()
  Global NewMap special()
  Global NewMap before()
  Global NewMap after()
  
  ;-{ Constant lex
  Enumeration
    #lex_None
    #lex_Alpha
    #lex_Space
    #lex_NewLine
    #lex_Misc
    #lex_Quote
    #lex_Slash
    #lex_Bracket
    #lex_SingleQuote
    #lex_Label
  EndEnumeration
  ;}
  ;-{ Constant force
  Enumeration
    #force_None
    #force_Comment
    #force_Quote
    #force_SingleQuote
    #force_QuoteBlock
    #force_CommentBlock
  EndEnumeration
  ;}
  ;-{ Constant Num
  Enumeration
    #num_none
    #num_ok
    #num_point
    #num_exp
  EndEnumeration
  ;}
  
  Macro SCI(Gadget,Msg,a=0,b=0)
    ScintillaSendMessage(Gadget,Msg,a,b)
  EndMacro
  
  Procedure IsLuaNumber(*word.character)
    ;check, if the string (unicode) is a valid LUA-Number
    Protected i,point,exp,lastEXP,num
    Protected hex,c
    Protected ok=#num_ok
    
    If *word\c='0'
      *word+SizeOf(character)
      If *word\c='x' Or *word\c='X'
        *word+SizeOf(character)
        hex=#True
      EndIf
    EndIf
    
    While *word\c
      c=*word\c
      
      If (c>='0' And c<='9') Or ( hex=#True And ((c>='a' And c<='f') Or (c>='A' And c<='F')) )
        LastExp=#False
        num=#True
      ElseIf c='.'
        If point
          ok=#num_none
          Break
        EndIf
        point=#True
      ElseIf (c='+' Or c='-') 
        If lastexp=#False
          ok=#num_none
          Break
        EndIf
        lastexp=#False
      ElseIf c='E' Or c='e' Or ( Hex=#True And (c='P' Or c='p')); bei hex kommt man hier eh nicht mit e hin
        If exp Or num=#False
          ok=#num_none
          Break
        EndIf
        exp=#True
        point=#True
        lastEXP=#True
      Else
        ok=#num_none
        Break
      EndIf
      *word+SizeOf(character)
    Wend
    
    If ok
      If lastEXP
        ProcedureReturn #num_exp
      ElseIf Point
        ProcedureReturn #num_point
      EndIf
    EndIf
    
    ProcedureReturn ok
    
  EndProcedure
  Procedure CheckLine(gadget,lineNumber)
    ;check it the lineindention is correct and correct the indention
    Protected mode,pos,currentpos,char
    Protected indentlevel,indentafter,indentbefore,word.s
    Protected linestate
    Protected spaces,tabsize
    
    pos=sci(gadget,#SCI_POSITIONFROMLINE,lineNumber)
    tabsize=sci(gadget,#SCI_GETTABWIDTH)
    
    currentpos=pos
    Repeat
      char=sci(gadget,#SCI_GETCHARAT,currentpos)
      currentpos+1
      If char=32
        spaces+1
      ElseIf char=9
        spaces+(tabsize-(spaces%tabsize))
      Else
        Break
      EndIf
    ForEver 
      linestate= sci(Gadget,#SCI_GETLINESTATE,lineNumber)
      If lineState & (#linestate_QuoteBlock|#linestate_CommentBlock)=0 Or  lineState & #linestate_start
        IndentLevel=(linestate&#linestate_IndentLevelMask)
        If spaces<>tabsize*indentlevel
          sci(gadget,#SCI_SETTARGETSTART,pos)            
          sci(gadget,#SCI_SETTARGETEND,currentpos-1)
          word=LSet("",indentlevel,Chr(9))
          sci(gadget,#SCI_REPLACETARGET,StringByteLength(word,#PB_UTF8),_fastUTF8(word))
        EndIf
      EndIf
     
  EndProcedure
  Procedure luaIndention(gadget)
    ; the "complex" auto-indention-routine for lua
    Protected lineNumber,pos
    Protected indentlevel,indentafter,word.s
    Protected linestate
    Protected spaces,tabsize
    
    pos=sci(gadget,#SCI_GETCURRENTPOS)
    Highlight(gadget,pos,pos)
    
    lineNumber=sci(gadget,#SCI_LINEFROMPOSITION,pos)
    CheckLine(gadget,lineNumber-1)
    If lineNumber>0
      linestate= sci(Gadget,#SCI_GETLINESTATE,lineNumber)
      ;Debug Hex(linestate)
      IndentAfter=((lineState&#linestate_IndentAfterMask)>>#linestate_IndentAfterShift) - #linestate_IndentBase
      IndentLevel=(linestate&#linestate_IndentLevelMask) 
      ;Debug Hex(linestate&(#linestate_CommentBlock|#linestate_QuoteBlock))+" "+Hex(linestate&#linestate_end)
      If linestate&(#linestate_CommentBlock|#linestate_QuoteBlock)=0 ; Or linestate&#linestate_end
                                                           ;indentlevel+indentafter
        word=LSet("",indentlevel,Chr(9))
        sci(gadget,#SCI_REPLACESEL,StringByteLength(word,#PB_UTF8),_fastUTF8(word))
      EndIf
    EndIf
    
  EndProcedure
  Procedure Highlight(Gadget,startPos,endPos)
    ;Style form startpos to endpos
    Protected lineNumber,lineNumber2,foldLevel
    Protected thisLevel,nextLevel
    Protected currentPos
    Protected State,oldState
    Protected word.s,char,charnext,stylePos,style
    Protected force
    Protected ignoreNext
    Protected i, achar.s
    Protected forceLabel
    Protected lineStateSubCount,EndMark.s
    Protected lineState
    Protected doFold
    Protected IndentLevel,IndentBefore,IndentAfter
    Protected eolchar
    
    linenumber = SCI(Gadget, #SCI_LINEFROMPOSITION, startpos)
    doFold=_GetPropertyFlag(gadget,#PropertyFlag_Fold)
   
    If sci(gadget,#SCI_GETEOLMODE)=#SC_EOL_CR 
      eolchar=13
    Else
      eolchar=10
    EndIf
    
    If linenumber = 0
      foldlevel = #SC_FOLDLEVELBASE
      lineStateSubCount=0
    Else
      linenumber - 1
      foldlevel = SCI(Gadget, #SCI_GETFOLDLEVEL, linenumber) & #SC_FOLDLEVELNUMBERMASK
      
      linestate= sci(Gadget,#SCI_GETLINESTATE,lineNumber)
      If lineState & (#linestate_start|#linestate_CommentBlock)=#linestate_CommentBlock
        lineStateSubCount=(lineState & #linestate_submask)>>#linestate_subshift
        endmark="--]"+RSet("]",lineStateSubCount,"=")
        force=#force_CommentBlock
      EndIf
      If lineState & (#linestate_start|#linestate_QuoteBlock)=#linestate_QuoteBlock
        lineStateSubCount=(lineState & #linestate_submask)>>#linestate_subshift
        endmark="]"+RSet("]",lineStateSubCount,"=")
        force=#force_QuoteBlock      
      EndIf    
      IndentBefore=((lineState&#linestate_IndentBeforeMask)>>#linestate_IndentBeforeShift) - #linestate_IndentBase
      IndentLevel=(linestate&#linestate_IndentLevelMask) - IndentBefore
      IndentBefore=0
    EndIf
    
    thisLevel=foldLevel
    nextLevel=foldLevel
    linestate=0
    
    currentPos = SCI(Gadget, #SCI_POSITIONFROMLINE, linenumber)
    stylePos=currentPos
    
    SCI(gadget, #SCI_STARTSTYLING, currentpos)
    oldState=#lex_None
    charnext= Sci(Gadget, #SCI_GETCHARAT, currentpos)
    
    While currentpos <= endpos
      char = charnext
      charnext=Sci(Gadget, #SCI_GETCHARAT, currentpos+1)
      Select char 
        Case ':'
          If oldState=#lex_alpha And ((charnext>='a' And charnext<='z') Or (charnext>='A' And charnext<='Z') Or charnext='_')
            state=#lex_Alpha
          Else            
            state=#lex_Label
          EndIf
          
        Case 'a' To 'z','A' To 'Z','0' To '9','_'
          If Left(word,2)="::"
            state=#lex_Label
          Else
            state=#lex_Alpha 
          EndIf
          
        Case '(', ')'
          state=#lex_bracket
        Case '"'
          state=#lex_Quote
        Case 39 ;'
          state=#lex_SingleQuote
        Case '/'
          state=#lex_Slash
          
        Case 0 To 32, 127 ;whitespaces and eolchar
          If char=eolchar
            state=#lex_NewLine
          Else
            state=#lex_Space
          EndIf
          
          ;erweiterung, um Zahlen zu erkennen          
        Case '+','-'
          If oldState=#lex_Alpha And IsLuaNumber(@word)=#num_exp;muss nach einen exp erfolgen!
            state=#lex_Alpha
          Else
            state=#lex_misc
          EndIf
        Case '.'
          If oldState=#lex_Alpha And IsLuaNumber(@word)=#num_ok ;darf weder point noch exp haben, nur ok!
            state=#lex_Alpha
          ElseIf oldState=#lex_Alpha And ((charnext>='a' And charnext<='z') Or (charnext>='A' And charnext<='Z') Or charnext='_')
            state=#lex_Alpha
          Else            
            state=#lex_Misc
          EndIf
          
        Default
          state=#lex_Misc
      EndSelect
      
      
      If oldstate=#lex_NewLine Or state<>oldState
        
        Select force
            
          Case #force_CommentBlock;{
            lineState|#linestate_CommentBlock
            style=#style_CommentBlock
            If oldState=#lex_misc And Right(word,Len(EndMark))=EndMark
              force=#force_None
              ;linestate|#linestate_end
            EndIf
            ;}
            
          Case #force_QuoteBlock;{
            linestate|#linestate_QuoteBlock
            style=#style_StringBlock
            If oldState=#lex_misc And Right(word,Len(EndMark))=EndMark
              force=#force_None
              ;linestate|#linestate_end
            EndIf
            ;}
            
          Case #force_Quote;{
            style=#style_String
            Select oldState
              Case #lex_Slash
                ignoreNext=#True
              Case #lex_Quote
                If ignoreNext=#False
                  force=#force_None
                Else
                  ignoreNext=#False
                EndIf
              Case #lex_NewLine
                force=#force_None
              Default
                ignoreNext=#False
            EndSelect
            ;}
            
          Case #force_SingleQuote;{
            style=#style_SingleString
            Select oldState
              Case #lex_Slash
                ignoreNext=#True
              Case #lex_SingleQuote
                If ignoreNext=#False
                  force=#force_None
                Else
                  ignoreNext=#False
                EndIf
              Case #lex_NewLine
                force=#force_None
              Default
                ignoreNext=#False
            EndSelect
            ;}
            
          Case #force_None;{
            Select oldState
              Case #lex_SingleQuote
                ignoreNext=#False
                force=#force_SingleQuote
                style=#style_SingleString
                forceLabel=#False
                
              Case #lex_Quote
                ignoreNext=#False
                force=#force_Quote
                style=#style_String
                forceLabel=#False
                
              Case #lex_bracket
                style=#STYLE_DEFAULT
                forceLabel=#False
                
              Case #lex_Space
                style=#STYLE_DEFAULT
                
              Case #lex_Slash,#lex_NewLine
                style=#STYLE_DEFAULT
                forceLabel=#False
                
              Case #lex_Label
                If Left(word,2)="::" And FindString(Mid(word,3,Len(word)-4),":")=0
                  style=#STYLE_label
                Else
                  style=#STYLE_DEFAULT
                EndIf
                forceLabel=#False
                
              Case #lex_Alpha              
                If FindMapElement(keyword(),word)
                  style=#style_Keyword
                  If word="goto"
                    forceLabel=#True
                  Else
                    forceLabel=0
                  EndIf
                ElseIf FindMapElement(special(),word)
                  style=#style_SpecialKeyword
                ElseIf IsLuaNumber(@word)
                  style=#style_Number
                  forceLabel=#False              
                ElseIf forceLabel
                  forceLabel=#False
                  style=#style_label
                Else 
                  style=#style_AlphaNum
                EndIf          
                
                ;Folding
                If FindMapElement(foldstart(),word)
                  thislevel | #SC_FOLDLEVELHEADERFLAG
                  nextlevel + 1
                EndIf
                If FindMapElement(foldend(),word)
                  nextlevel - 1
                  If nextlevel < #SC_FOLDLEVELBASE
                    nextlevel = #SC_FOLDLEVELBASE
                  EndIf  
                EndIf
                
                ;Indention
                If FindMapElement(after(),word)
                  IndentAfter+after()
                EndIf
                If FindMapElement(before(),word)
                  If before()<0 And IndentAfter=>-before()
                    IndentAfter+before()
                  Else                     
                    IndentBefore+before()
                  EndIf   
                EndIf
                
              Case #lex_Misc
                style=#STYLE_DEFAULT
                If Left(word,1)="["
                  lineStateSubCount=1
                  For i=2 To Len(word)
                    achar=Mid(word,i,1)
                    If achar="="
                      lineStateSubCount+1
                    ElseIf achar="["
                      If lineStateSubCount>255
                        lineStateSubCount=255
                      EndIf
                      endmark="]"+RSet("]",lineStateSubCount,"=")
                      lineState|#linestate_start|#linestate_QuoteBlock
                      force=#force_QuoteBlock
                      style=#style_StringBlock
                      Break
                    Else
                      Break
                    EndIf
                  Next
                  If force<>#force_QuoteBlock
                    lineStateSubCount=0
                  EndIf      
                  
                ElseIf Left(word,2)="--"
                  style=#style_Comment
                  force=#force_Comment
                  
                  If Mid(word,3,1)="["
                    lineStateSubCount=1
                    For i=4 To Len(word)
                      achar=Mid(word,i,1)
                      If achar="="
                        lineStateSubCount+1
                      ElseIf achar="["
                        If lineStateSubCount>255
                          lineStateSubCount=255
                        EndIf
                        endmark="--]"+RSet("]",lineStateSubCount,"=")
                        lineState|#linestate_start|#linestate_CommentBlock
                        force=#force_CommentBlock
                        style=#style_CommentBlock
                        Break
                      Else
                        Break
                      EndIf
                    Next
                    If force<>#force_CommentBlock
                      lineStateSubCount=0
                    EndIf                  
                    
                  EndIf
                EndIf
                forceLabel=#False
                
              Default
                ; First call doesn't have a style
                style=0
                forceLabel=#False
            EndSelect
            ;}
            
          Case #force_Comment;{
            If oldstate=#lex_NewLine
              force=#force_None
            EndIf  
            style=#style_Comment
            ;}
            
        EndSelect
        
        If style
          SCI(Gadget, #SCI_SETSTYLING, currentpos - stylePos, style)
          stylePos = currentpos
        EndIf       
        word=""
        oldState=state
      EndIf
      
      word+Chr(char)
      
      If state=#lex_NewLine Or currentpos=endPos
        ;block quotes and comment
        If linestate &(#linestate_QuoteBlock|#linestate_CommentBlock)=0
          lineStateSubCount=0
        Else
          lineState+ (lineStateSubCount<<#linestate_subshift)
        EndIf  
        
        ;indentaion
        lineState+ ((IndentAfter+#linestate_IndentBase)<<#linestate_IndentAfterShift)&#linestate_IndentAfterMask
        lineState+ ((IndentBefore+#linestate_IndentBase)<<#linestate_IndentbeforeShift)&#linestate_IndentBeforeMask
        
        IndentLevel+IndentBefore:If IndentLevel<0:IndentLevel=0:ElseIf IndentLevel>255:IndentLevel=255:EndIf
        linestate+ IndentLevel&#linestate_IndentLevelMask
        Indentlevel+IndentAfter:If IndentLevel<0:IndentLevel=0:ElseIf IndentLevel>255:IndentLevel=255:EndIf
        
        sci(gadget,#SCI_SETLINESTATE,linenumber,lineState)
        
        linestate=0
        IndentAfter=0
        IndentBefore=0
        
        ;folding
        If doFold
          Sci(Gadget, #SCI_SETFOLDLEVEL, linenumber, thislevel)
        EndIf          
        thislevel = nextlevel
          
        lineNumber+1
      EndIf
      
      currentpos+1
    Wend
    
    If _GetProperty(Gadget,#Property_AutoIndention)=#AutoIndention_complex
      Protected startline,endline,currentline
      startline=sci(gadget,#SCI_LINEFROMPOSITION,startpos)
      endline=sci(gadget,#SCI_LINEFROMPOSITION,endpos)
      currentpos=sci(gadget,#SCI_GETCURRENTPOS)
      currentline=sci(gadget,#SCI_LINEFROMPOSITION,currentPos)
      For i=startline To endline
        If i<>currentline
          CheckLine(gadget,i)
        EndIf
      Next
    EndIf
  EndProcedure
  Procedure FindWordStart(gadget,endpos)
    ;used to find the current word under the position (position=last char in the word).
    ;and detect words connected with . or :
    Protected startpos,char,i
    startpos=endpos
    Repeat      
      startpos=sci(gadget,#SCI_WORDSTARTPOSITION,startpos,#True)
      char=sci(gadget,#SCI_GETCHARAT,startpos-1)
      If char='.' Or char=':'
        startpos-1
      Else
        Break
      EndIf
    ForEver
    ProcedureReturn startpos
  EndProcedure
  Procedure LuaCallback(Gadget, *scinotify.SCNotification)
    ;scintilla-Callback for the lexer
    Protected startPos,lineNumber,mode
    
    Select *scinotify\nmhdr\code
        
      Case #SCN_UPDATEUI
        If *scinotify\updated=#SC_UPDATE_SELECTION
          If _GetPropertyFlag(gadget,#PropertyFlag_Brace)
            _MarkBrace(gadget,"()[]{}")
          EndIf        
        EndIf
        
      Case #SCN_MODIFIED
        If *scinotify\modificationType  & (#SC_MOD_DELETETEXT)
          If _GetPropertyFlag(gadget,#PropertyFlag_Brace)
            _MarkBrace(gadget,"()[]{}")
          EndIf
        EndIf
        
        If *scinotify\length=1 And (*scinotify\modificationType&#SC_MOD_DELETETEXT) And *scinotify\position=sci(gadget,#SCI_GETCURRENTPOS) 
          _Autocomplete(gadget,@FindWordStart(),special())
        EndIf
        
      Case #SCN_CHARADDED
        If (*scinotify\ch>='a' And *scinotify\ch<='z') Or (*scinotify\ch>='A' And *scinotify\ch<='Z') Or (*scinotify\ch>='0' And *scinotify\ch<='9') Or *scinotify\ch='_' Or *scinotify\ch='.' Or *scinotify\ch=':'
          _Autocomplete(gadget,@FindWordStart(),special())
        EndIf
        
        If _GetPropertyFlag(gadget,#PropertyFlag_Brace)
          _MarkBrace(gadget,"()[]{}")
        EndIf
        
        If (*scinotify\ch=10 Or *scinotify\ch=13) 
          mode=sci(gadget,#SCI_GETEOLMODE)
          If (mode=#SC_EOL_CR And *scinotify\ch=13) Or (mode<>#SC_EOL_CR And *scinotify\ch=10)
            Select _GetProperty(gadget,#property_AutoIndention)    
              Case #AutoIndention_simple:  _indention(gadget)
              Case #AutoIndention_complex: luaIndention(gadget)
            EndSelect
            ;correct brace
            If _GetPropertyFlag(gadget,#PropertyFlag_Brace)
              _MarkBrace(gadget,"()[]{}")
            EndIf 
          EndIf          
        EndIf
        
      Case #SCN_AUTOCCHARDELETED
        _Autocomplete(gadget,@FindWordStart(),special())
        
      Case  #SCN_STYLENEEDED
        startPos =SCI(Gadget,#SCI_GETENDSTYLED)
        ;the lua-lexer always style the line before start - so we don't need 
        ;lineNumber = SCI(Gadget,#SCI_LINEFROMPOSITION,startPos)
        ;startPos = SCI(Gadget,#SCI_POSITIONFROMLINE,lineNumber)
        Highlight(Gadget,startPos, *scinotify\position)
        
    EndSelect
    
    ProcedureReturn _callback(gadget,*scinotify)
  EndProcedure  
  
  Procedure luainit(gadget,*style.lexerstyle)
    ;this routine is called from the scintilla::gadget, after the gadget is created
    Protected word.s,i
    ;{ read Wordlists for sytling, folding, indention
    If MapSize(special())=0
      Restore special
      Repeat
        Read.s word
        If word<>""
          special(word)=#True
        Else
          Break
        EndIf
      ForEver
    EndIf
    If MapSize(keyword())=0
      Restore keyword
      Repeat
        Read.s word
        If word<>""
          keyword(word)=#True
        Else
          Break
        EndIf
      ForEver
    EndIf
    If MapSize(foldstart())=0
      Restore foldstart
      Repeat
        Read.s word
        If word<>""
          foldstart(word)=#True
        Else
          Break
        EndIf
      ForEver
    EndIf
    If MapSize(foldend())=0
      Restore foldend
      Repeat
        Read.s word
        If word<>""
          foldend(word)=#True
        Else
          Break
        EndIf
      ForEver
    EndIf
    If MapSize(after())=0
      Restore after
      Repeat
        Read.s word
        Read.i i
        If word<>""
          after(word)=i
        Else
          Break
        EndIf
      ForEver
    EndIf
    If MapSize(before())=0
      Restore before
      Repeat
        Read.s word
        Read.i i
        If word<>""
          before(word)=i
        Else
          Break
        EndIf
      ForEver
    EndIf
    ;}
    
    ;Set Lexer to Container
    ScintillaSendMessage(Gadget,#SCI_SETLEXER,#SCLEX_CONTAINER)
    
    ;Chars for Word-Detection
    sci(gadget,#SCI_SETWORDCHARS,0, _fastUTF8("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_"))
    ;Autocomplete-select-chars
    sci(gadget,#SCI_AUTOCSETFILLUPS,0,_fastUTF8(~".:[("))    
    
    ;the style
    SetStyle(Gadget,*style)
    
    ProcedureReturn #True
    ;wordlists
    DataSection
      special:
      Data.s "load", "rawlen", "select", "package", "utf8", "_ENV", "assert", "collectgarbage", "dofile", "error", "_G", "getmetatable", "ipairs", "loadfile", "next", "pairs", "pcall", "print", "rawequal", "rawget", "rawset", "setmetatable", "tonumber", "tostring", "type", "_VERSION", "xpcall", "string", "table", "math", "coroutine", "io", "os", "debug",
             "utf8.char", "utf8.charpattern", "utf8.codes", "utf8.codepoint", "utf8.len", "utf8.offset",
             "string.byte", "string.char", "string.dump", "string.find", "string.format", "string.gmatch", "string.gsub", "string.len", "string.lower", "string.match", "string.pack", "string.packsize", "string.rep", "string.reverse", "string.sub", "string.unpack", "string.upper", 
             "table.concat", "table.insert", "table.move", "table.pack", "table.remove", "table.sort", "table.unpack", 
             "math.abs", "math.acos", "math.asin", "math.atan", "math.ceil", "math.cos", "math.deg", "math.exp", "math.floor", "math.fmod", "math.huge", "math.log", "math.max", "math.maxinteger", "math.min", "math.mininteger", "math.modf", "math.pi", "math.rad", "math.random", "math.randomseed", "math.sin", "math.sqrt", "math.tan", "math.tointeger", "math.type", "math.ult", 
             "coroutine.create", "coroutine.resume", "coroutine.status", "coroutine.wrap", "coroutine.yield", 
             "io.close", "io.flush", "io.input", "io.lines", "io.open", "io.output", "io.read", "io.tmpfile", "io.type", "io.write", "io.stdin", "io.stdout", "io.stderr", 
             "os.clock", "os.date", "os.difftime", "os.execute", "os.exit", "os.getenv", "os.remove", "os.rename", "os.setlocale", "os.time", "os.tmpname",
             "coroutine.isyieldable", "coroutine.running", "io.popen", 
             "package.config", "package.searchers", "package.searchpath", "require", "package.cpath", "package.loaded", "package.loadlib", "package.path", "package.preload",
             "debug.debug", "debug.gethook", "debug.getinfo", "debug.getlocal", "debug.getupvalue", "debug.setlocal", "debug.setupvalue", "debug.sethook", "debug.traceback", "debug.getmetatable", "debug.getregistry", "debug.setmetatable", "debug.getuservalue", "debug.setuservalue", "debug.upvalueid", "debug.upvaluejoin",
             ""
      keyword:
      Data.s "and","break","do","else","elseif","end","false","for","function","goto",
             "if","in","local","nil","not","or","repeat","return","then","true","until",
             "while",""
      foldstart:
      Data.s "function","do","if",""
      after:
      Data.s "function"
      Data.i 1
      Data.s "if"
      Data.i 1
      Data.s "do"
      Data.i 1
      Data.s "else"
      Data.i 1
      Data.s "elseif"
      Data.i 1
      Data.s ""
      Data.i 0
      
      before:
      Data.s "else"
      Data.i -1
      Data.s "elseif"
      Data.i -1
      Data.s "end"
      Data.i -1
      Data.s ""
      Data.i 0
      
      foldend:
      Data.s "end",""
    EndDataSection
  EndProcedure
    
  Procedure use()
    ;used for selection the lexer. return a lexerinfo structure
    ProcedureReturn ?ret
    DataSection
      ret:
      Data.i @LuaCallback(),@luainit()
    EndDataSection
  EndProcedure
  
  
EndModule

;-
;-##################################################################
;-
;- Demo Code
CompilerIf #PB_Compiler_IsMainFile
  Define hWnd
  Define text.s
  Define event
  Define gadget
  
  scintilla::init()
  
  hWnd = OpenWindow(0,#PB_Ignore,#PB_Ignore,640,480,"leer",#PB_Window_SystemMenu|#PB_Window_SizeGadget)
  ;SmartWindowRefresh(0,1)
  RemoveKeyboardShortcut(0,#PB_Shortcut_Tab)
  RemoveKeyboardShortcut(0,#PB_Shortcut_Tab|#PB_Shortcut_Shift)
  
  gadget= scintilla::gadget(#PB_Any,0,0,640,480,Lua_Lexer::use())
  
  text.s = "function test()"        + #LF$
  text.s + "  'blub' äöüß "               + #LF$
  text.s + "  for [[something]] in 1 do " + #LF$
  text.s + "  end " + #LF$
  
  text.s + "end" +#LF$
  
  Scintilla::SetText(gadget,text)
  Scintilla::AppendText(gadget,text)
  Scintilla::AppendText(gadget,text)
  Scintilla::AppendText(gadget,text)
  Scintilla::AppendText(gadget,text)
  Scintilla::AppendText(gadget,text)
  
  
  Repeat
    event = WaitWindowEvent()
    
    If event = #PB_Event_SizeWindow
      ResizeGadget(gadget,#PB_Ignore,#PB_Ignore,WindowWidth(0),WindowHeight(0))
    EndIf
    
  Until event = #PB_Event_CloseWindow
  
  text=Scintilla::GetText(gadget)
  Debug text
  
CompilerEndIf
Bei Fragen fragen!
CodeArchiv Rebirth: Deutsches Forum Github Hilfe ist immer gern gesehen!
GPI
Beiträge: 1511
Registriert: 29.08.2004 13:18
Kontaktdaten:

Re: Scintilla mit Universal-lexer (MAC/LINUX/WIN)

Beitrag von GPI »

http://game.gpihome.eu/PureBasic/scinti ... llaText.7z

Ich hab das obrige Beispiel stark überarbeitet. Zum einen hab ich so ziemlich jedes Feature von Scintilla (Calltip, Tooltip, Einrücken, Autovervollständigungslisten, Falten, Einrück-Hilfs-Linien) eingebaut. Für die Autovervollständigung werden sogar sämtliche Eingaben überprüft und erweitert die Listen entsprechend.

Wichtig: Die Lexer sind sehr universell. Es werden keine großartigen Analysen wie Gültigkeitsbereiche durchgeführt oder welche Wörter in STRUCTURE/ENDSTRUCTURE vorhanden sind.

Die Aufgaben sind in grob drei Module verteilt:
Scintilla -Modul, dass das Gadget erstellt
_Base_Lexer - Module, das lauter allgemeine Functionen für die Lexer enthält
*_Lexer - Modul, das den eigentlichen Lexer beschreibt und den _base_lexer nutzt.

Das Einfärben erfolgt grob in zwei schritten.
Mit "Regular Expression" wird eine Vorauswahl getroffen. Bspw. beginnent mit # und darauf folgend Buchstaben/Zahlen muss eine Konstante sein. Zahlen kann man auch so leicht finden. Oder es wird einfach ein "Keyword" selectiert (Alphanumerische Ansammlung von Zeichen). Diese Vorauswahl erledigen die "Searcher"

Für jede Vorauswahl wird anschließend in der passenden Remap-Tabelle nach den Word gesucht. Dort ist dann hinterlegt, wie das Wort richtig eingeordnet werden muss, ob es ein Falten-Auslöse-Kennwort ist, ob es einen Calltip/Tooltip hat und so weiter und so fort.

Anschließend wird eingefärbt.

Ich hab zwei Lexer mal vordefiniert, einen für LUA und einen für PB.
Der Lua-Lexer ist recht einfach, der für PB ist schon ein bischen komplizierter.

Wenn man nicht so kompliziert wirklich *ALLES* einfärben will, dann könnte man die PB-Lexer natürlich sehr stark vereinfachen. Siehe den Lua-Lexer, der färbt prinzipiell nur die "Schlüsselwörter" der Sprache ein. Man sollte sich überlegen was man will.

Wie gesagt, die Lexer machen keinerlei Analyse des Quellcodes, wie es bspw. der PureBasic-Editor macht.

In Quellcode werden keine OS-Spezifischen Befehle genutzt und sollten unter MAC, Linux und Windows laufen.
CodeArchiv Rebirth: Deutsches Forum Github Hilfe ist immer gern gesehen!
Benutzeravatar
Sicro
Beiträge: 955
Registriert: 11.08.2005 19:08
Kontaktdaten:

Re: Scintilla mit Universal-lexer (MAC/LINUX/WIN)

Beitrag von Sicro »

Ziemlich cool :allright:

Bei der Code-Zeile 94 funktioniert das Falten nicht korrekt. Es wird fälschlicherweise angenommen, dass bei der Code-Zeile 198 das Ende der Faltung ist.
Bild
Warum OpenSource eine Lizenz haben sollte :: PB-CodeArchiv-Rebirth :: Pleasant-Dark (Syntax-Farbschema) :: RegEx-Engine (kompiliert RegExes zu NFA/DFA)
Manjaro Xfce x64 (Hauptsystem) :: Windows 10 Home (VirtualBox) :: Neueste PureBasic-Version
GPI
Beiträge: 1511
Registriert: 29.08.2004 13:18
Kontaktdaten:

Re: Scintilla mit Universal-lexer (MAC/LINUX/WIN)

Beitrag von GPI »

danke

Das liegt daran, das ich eine Faltenstart mit ";-{" in den PureBasic-Optionen eingefügt hab. Bei mir wird das korrekt gemacht.

Ich überleg gerade, wie man den Lexer beschleunigen kann. Wenn man per Copy&Paste den eigenen Quellcode einfügt, dann brauch er über eine Sekunde auf diesen PC. PB ist etwas schneller. Ich denke mal das die Regex-Aufrufe viel zeit kosten.
CodeArchiv Rebirth: Deutsches Forum Github Hilfe ist immer gern gesehen!
Antworten