Upper and Lower Case Mapping for Unicode

Share your advanced PureBasic knowledge/code with the community.
User avatar
Sicro
Enthusiast
Enthusiast
Posts: 538
Joined: Wed Jun 25, 2014 5:25 pm
Location: Germany
Contact:

Re: Upper and Lower Case Mapping for Unicode

Post by Sicro »

@idle:

In your previous post, did you update the code as well? Because the only difference between your previous and last code is now the debug line at the end of the code, which was removed in the last code.

Anyway, your previous and last code now shows the correct result:

Code: Select all

; returns 'B5' and this is wrong
t = Chr($00B5)
pUpCase(t)
Debug Hex(Asc(t)) ; Shows '39C'
; correct is '39C'

But if you run your code, you will notice that the result is now incomplete:

Code: Select all

t.s = "abcdef 0123456789, äöü, äöü, áóú" 
pUpCase(t) 
pLowCase(t) 
t = "ABCDEF" + Mid(t,7) 
Debug t
shows:
Debug wrote:ABCDEF
instead of:
Debug wrote:ABCDEF 0123456789, äöü, äöü, áóú
The problem is that to fix the previous problem in the _pUpCase() procedure you added:

Code: Select all

Else    
  *char\u = UNI_LCase(*char\u)
which produces the new problem.

Maybe the data structure would be better this way:

Code: Select all

Structure CaseMappingStruc
  lowerCase.u
  upperCase.u
  titleCase.u
EndStructure

Global.CaseMappingStruc NewMap caseMapping()

; Fill the map with the table from the data section
; ...
Read.u code
With caseMapping(Chr(code))
  Read.u \lowerCase
  Read.u \upperCase
  Read.u \titleCase
EndWith
; ...

Procedure$ GetUpperCase(char$)
  If FindMapElement(caseMapping(), char$)
    ProcedureReturn Chr(caseMapping()\upperCase)
  Else
    ProcedureReturn char$
  EndIf
EndProcedure
or

Code: Select all

Structure CaseMappingEntryStruc
  lowerCase.u
  upperCase.u
  titleCase.u
EndStructure

Structure CaseMappingStruc
  entries.CaseMappingEntryStruc[0]
EndStructure

Procedure$ GetUpperCase(char$)
  Protected.CaseMappingStruc *caseMapping = ?caseMappingTable
  Protected charCode = Asc(char$)
  
  ProcedureReturn Chr(*caseMapping\entries[charCode]\upperCase)
EndProcedure

DataSection
  caseMappingTable:
  ; ...
  ; upperCase,lowerCase,titleCase
  Data.u $039C,$00B5,$039C
  ; ...
EndDataSection
The advantage of the last code is that the data is read directly from the data section and thus does not have to be read additionally into a map or array at the beginning.
Image
Why OpenSource should have a license :: PB-CodeArchiv-Rebirth :: Pleasant-Dark (syntax color scheme) :: RegEx-Engine (compiles RegExes to NFA/DFA)
Manjaro Xfce x64 (Main system) :: Windows 10 Home (VirtualBox) :: Newest PureBasic version
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: Upper and Lower Case Mapping for Unicode

Post by idle »

This seems to be right now.

Code: Select all

Global Dim Uni_UCase($FF5A) 
Global Dim Uni_LCase($FF5A) 

Procedure Uni_Init() 
  Protected lchar,uchar,tchar  
  
  Restore UNI
  
  Repeat 
    Read.u lchar 
    Read.u uchar 
    Read.u tchar 
    
    If lchar   
      If uchar 
        UNI_LCase(lchar) = uchar  
        UNI_UCase(uchar) = lchar 
      Else   
        UNI_LCase(lchar) = lchar  
        UNI_UCase(lchar) = tchar 
      EndIf 
    EndIf     
  Until lchar = 0 
  
  DataSection: UNI: 
    
    Data.w $0041,$0061,$0000,$0042,$0062,$0000,$0043,$0063
    Data.w $0000,$0044,$0064,$0000,$0045,$0065,$0000,$0046
    Data.w $0066,$0000,$0047,$0067,$0000,$0048,$0068,$0000
    Data.w $0049,$0069,$0000,$004A,$006A,$0000,$004B,$006B
    Data.w $0000,$004C,$006C,$0000,$004D,$006D,$0000,$004E
    Data.w $006E,$0000,$004F,$006F,$0000,$0050,$0070,$0000
    Data.w $0051,$0071,$0000,$0052,$0072,$0000,$0053,$0073
    Data.w $0000,$0054,$0074,$0000,$0055,$0075,$0000,$0056
    Data.w $0076,$0000,$0057,$0077,$0000,$0058,$0078,$0000
    Data.w $0059,$0079,$0000,$005A,$007A,$0000,$0061,$0000
    Data.w $0041,$0062,$0000,$0042,$0063,$0000,$0043,$0064
    Data.w $0000,$0044,$0065,$0000,$0045,$0066,$0000,$0046
    Data.w $0067,$0000,$0047,$0068,$0000,$0048,$0069,$0000
    Data.w $0049,$006A,$0000,$004A,$006B,$0000,$004B,$006C
    Data.w $0000,$004C,$006D,$0000,$004D,$006E,$0000,$004E
    Data.w $006F,$0000,$004F,$0070,$0000,$0050,$0071,$0000
    Data.w $0051,$0072,$0000,$0052,$0073,$0000,$0053,$0074
    Data.w $0000,$0054,$0075,$0000,$0055,$0076,$0000,$0056
    Data.w $0077,$0000,$0057,$0078,$0000,$0058,$0079,$0000
    Data.w $0059,$007A,$0000,$005A,$00B5,$0000,$039C,$00C0
    Data.w $00E0,$0000,$00C1,$00E1,$0000,$00C2,$00E2,$0000
    Data.w $00C3,$00E3,$0000,$00C4,$00E4,$0000,$00C5,$00E5
    Data.w $0000,$00C6,$00E6,$0000,$00C7,$00E7,$0000,$00C8
    Data.w $00E8,$0000,$00C9,$00E9,$0000,$00CA,$00EA,$0000
    Data.w $00CB,$00EB,$0000,$00CC,$00EC,$0000,$00CD,$00ED
    Data.w $0000,$00CE,$00EE,$0000,$00CF,$00EF,$0000,$00D0
    Data.w $00F0,$0000,$00D1,$00F1,$0000,$00D2,$00F2,$0000
    Data.w $00D3,$00F3,$0000,$00D4,$00F4,$0000,$00D5,$00F5
    Data.w $0000,$00D6,$00F6,$0000,$00D8,$00F8,$0000,$00D9
    Data.w $00F9,$0000,$00DA,$00FA,$0000,$00DB,$00FB,$0000
    Data.w $00DC,$00FC,$0000,$00DD,$00FD,$0000,$00DE,$00FE
    Data.w $0000,$00E0,$0000,$00C0,$00E1,$0000,$00C1,$00E2
    Data.w $0000,$00C2,$00E3,$0000,$00C3,$00E4,$0000,$00C4
    Data.w $00E5,$0000,$00C5,$00E6,$0000,$00C6,$00E7,$0000
    Data.w $00C7,$00E8,$0000,$00C8,$00E9,$0000,$00C9,$00EA
    Data.w $0000,$00CA,$00EB,$0000,$00CB,$00EC,$0000,$00CC
    Data.w $00ED,$0000,$00CD,$00EE,$0000,$00CE,$00EF,$0000
    Data.w $00CF,$00F0,$0000,$00D0,$00F1,$0000,$00D1,$00F2
    Data.w $0000,$00D2,$00F3,$0000,$00D3,$00F4,$0000,$00D4
    Data.w $00F5,$0000,$00D5,$00F6,$0000,$00D6,$00F8,$0000
    Data.w $00D8,$00F9,$0000,$00D9,$00FA,$0000,$00DA,$00FB
    Data.w $0000,$00DB,$00FC,$0000,$00DC,$00FD,$0000,$00DD
    Data.w $00FE,$0000,$00DE,$00FF,$0000,$0178,$0100,$0101
    Data.w $0000,$0101,$0000,$0100,$0102,$0103,$0000,$0103
    Data.w $0000,$0102,$0104,$0105,$0000,$0105,$0000,$0104
    Data.w $0106,$0107,$0000,$0107,$0000,$0106,$0108,$0109
    Data.w $0000,$0109,$0000,$0108,$010A,$010B,$0000,$010B
    Data.w $0000,$010A,$010C,$010D,$0000,$010D,$0000,$010C
    Data.w $010E,$010F,$0000,$010F,$0000,$010E,$0110,$0111
    Data.w $0000,$0111,$0000,$0110,$0112,$0113,$0000,$0113
    Data.w $0000,$0112,$0114,$0115,$0000,$0115,$0000,$0114
    Data.w $0116,$0117,$0000,$0117,$0000,$0116,$0118,$0119
    Data.w $0000,$0119,$0000,$0118,$011A,$011B,$0000,$011B
    Data.w $0000,$011A,$011C,$011D,$0000,$011D,$0000,$011C
    Data.w $011E,$011F,$0000,$011F,$0000,$011E,$0120,$0121
    Data.w $0000,$0121,$0000,$0120,$0122,$0123,$0000,$0123
    Data.w $0000,$0122,$0124,$0125,$0000,$0125,$0000,$0124
    Data.w $0126,$0127,$0000,$0127,$0000,$0126,$0128,$0129
    Data.w $0000,$0129,$0000,$0128,$012A,$012B,$0000,$012B
    Data.w $0000,$012A,$012C,$012D,$0000,$012D,$0000,$012C
    Data.w $012E,$012F,$0000,$012F,$0000,$012E,$0130,$0069
    Data.w $0000,$0131,$0000,$0049,$0132,$0133,$0000,$0133
    Data.w $0000,$0132,$0134,$0135,$0000,$0135,$0000,$0134
    Data.w $0136,$0137,$0000,$0137,$0000,$0136,$0139,$013A
    Data.w $0000,$013A,$0000,$0139,$013B,$013C,$0000,$013C
    Data.w $0000,$013B,$013D,$013E,$0000,$013E,$0000,$013D
    Data.w $013F,$0140,$0000,$0140,$0000,$013F,$0141,$0142
    Data.w $0000,$0142,$0000,$0141,$0143,$0144,$0000,$0144
    Data.w $0000,$0143,$0145,$0146,$0000,$0146,$0000,$0145
    Data.w $0147,$0148,$0000,$0148,$0000,$0147,$014A,$014B
    Data.w $0000,$014B,$0000,$014A,$014C,$014D,$0000,$014D
    Data.w $0000,$014C,$014E,$014F,$0000,$014F,$0000,$014E
    Data.w $0150,$0151,$0000,$0151,$0000,$0150,$0152,$0153
    Data.w $0000,$0153,$0000,$0152,$0154,$0155,$0000,$0155
    Data.w $0000,$0154,$0156,$0157,$0000,$0157,$0000,$0156
    Data.w $0158,$0159,$0000,$0159,$0000,$0158,$015A,$015B
    Data.w $0000,$015B,$0000,$015A,$015C,$015D,$0000,$015D
    Data.w $0000,$015C,$015E,$015F,$0000,$015F,$0000,$015E
    Data.w $0160,$0161,$0000,$0161,$0000,$0160,$0162,$0163
    Data.w $0000,$0163,$0000,$0162,$0164,$0165,$0000,$0165
    Data.w $0000,$0164,$0166,$0167,$0000,$0167,$0000,$0166
    Data.w $0168,$0169,$0000,$0169,$0000,$0168,$016A,$016B
    Data.w $0000,$016B,$0000,$016A,$016C,$016D,$0000,$016D
    Data.w $0000,$016C,$016E,$016F,$0000,$016F,$0000,$016E
    Data.w $0170,$0171,$0000,$0171,$0000,$0170,$0172,$0173
    Data.w $0000,$0173,$0000,$0172,$0174,$0175,$0000,$0175
    Data.w $0000,$0174,$0176,$0177,$0000,$0177,$0000,$0176
    Data.w $0178,$00FF,$0000,$0179,$017A,$0000,$017A,$0000
    Data.w $0179,$017B,$017C,$0000,$017C,$0000,$017B,$017D
    Data.w $017E,$0000,$017E,$0000,$017D,$017F,$0000,$0053
    Data.w $0180,$0000,$0243,$0181,$0253,$0000,$0182,$0183
    Data.w $0000,$0183,$0000,$0182,$0184,$0185,$0000,$0185
    Data.w $0000,$0184,$0186,$0254,$0000,$0187,$0188,$0000
    Data.w $0188,$0000,$0187,$0189,$0256,$0000,$018A,$0257
    Data.w $0000,$018B,$018C,$0000,$018C,$0000,$018B,$018E
    Data.w $01DD,$0000,$018F,$0259,$0000,$0190,$025B,$0000
    Data.w $0191,$0192,$0000,$0192,$0000,$0191,$0193,$0260
    Data.w $0000,$0194,$0263,$0000,$0195,$0000,$01F6,$0196
    Data.w $0269,$0000,$0197,$0268,$0000,$0198,$0199,$0000
    Data.w $0199,$0000,$0198,$019A,$0000,$023D,$019C,$026F
    Data.w $0000,$019D,$0272,$0000,$019E,$0000,$0220,$019F
    Data.w $0275,$0000,$01A0,$01A1,$0000,$01A1,$0000,$01A0
    Data.w $01A2,$01A3,$0000,$01A3,$0000,$01A2,$01A4,$01A5
    Data.w $0000,$01A5,$0000,$01A4,$01A6,$0280,$0000,$01A7
    Data.w $01A8,$0000,$01A8,$0000,$01A7,$01A9,$0283,$0000
    Data.w $01AC,$01AD,$0000,$01AD,$0000,$01AC,$01AE,$0288
    Data.w $0000,$01AF,$01B0,$0000,$01B0,$0000,$01AF,$01B1
    Data.w $028A,$0000,$01B2,$028B,$0000,$01B3,$01B4,$0000
    Data.w $01B4,$0000,$01B3,$01B5,$01B6,$0000,$01B6,$0000
    Data.w $01B5,$01B7,$0292,$0000,$01B8,$01B9,$0000,$01B9
    Data.w $0000,$01B8,$01BC,$01BD,$0000,$01BD,$0000,$01BC
    Data.w $01BF,$0000,$01F7,$01C4,$01C6,$01C5,$01C5,$01C6
    Data.w $01C5,$01C6,$0000,$01C5,$01C7,$01C9,$01C8,$01C8
    Data.w $01C9,$01C8,$01C9,$0000,$01C8,$01CA,$01CC,$01CB
    Data.w $01CB,$01CC,$01CB,$01CC,$0000,$01CB,$01CD,$01CE
    Data.w $0000,$01CE,$0000,$01CD,$01CF,$01D0,$0000,$01D0
    Data.w $0000,$01CF,$01D1,$01D2,$0000,$01D2,$0000,$01D1
    Data.w $01D3,$01D4,$0000,$01D4,$0000,$01D3,$01D5,$01D6
    Data.w $0000,$01D6,$0000,$01D5,$01D7,$01D8,$0000,$01D8
    Data.w $0000,$01D7,$01D9,$01DA,$0000,$01DA,$0000,$01D9
    Data.w $01DB,$01DC,$0000,$01DC,$0000,$01DB,$01DD,$0000
    Data.w $018E,$01DE,$01DF,$0000,$01DF,$0000,$01DE,$01E0
    Data.w $01E1,$0000,$01E1,$0000,$01E0,$01E2,$01E3,$0000
    Data.w $01E3,$0000,$01E2,$01E4,$01E5,$0000,$01E5,$0000
    Data.w $01E4,$01E6,$01E7,$0000,$01E7,$0000,$01E6,$01E8
    Data.w $01E9,$0000,$01E9,$0000,$01E8,$01EA,$01EB,$0000
    Data.w $01EB,$0000,$01EA,$01EC,$01ED,$0000,$01ED,$0000
    Data.w $01EC,$01EE,$01EF,$0000,$01EF,$0000,$01EE,$01F1
    Data.w $01F3,$01F2,$01F2,$01F3,$01F2,$01F3,$0000,$01F2
    Data.w $01F4,$01F5,$0000,$01F5,$0000,$01F4,$01F6,$0195
    Data.w $0000,$01F7,$01BF,$0000,$01F8,$01F9,$0000,$01F9
    Data.w $0000,$01F8,$01FA,$01FB,$0000,$01FB,$0000,$01FA
    Data.w $01FC,$01FD,$0000,$01FD,$0000,$01FC,$01FE,$01FF
    Data.w $0000,$01FF,$0000,$01FE,$0200,$0201,$0000,$0201
    Data.w $0000,$0200,$0202,$0203,$0000,$0203,$0000,$0202
    Data.w $0204,$0205,$0000,$0205,$0000,$0204,$0206,$0207
    Data.w $0000,$0207,$0000,$0206,$0208,$0209,$0000,$0209
    Data.w $0000,$0208,$020A,$020B,$0000,$020B,$0000,$020A
    Data.w $020C,$020D,$0000,$020D,$0000,$020C,$020E,$020F
    Data.w $0000,$020F,$0000,$020E,$0210,$0211,$0000,$0211
    Data.w $0000,$0210,$0212,$0213,$0000,$0213,$0000,$0212
    Data.w $0214,$0215,$0000,$0215,$0000,$0214,$0216,$0217
    Data.w $0000,$0217,$0000,$0216,$0218,$0219,$0000,$0219
    Data.w $0000,$0218,$021A,$021B,$0000,$021B,$0000,$021A
    Data.w $021C,$021D,$0000,$021D,$0000,$021C,$021E,$021F
    Data.w $0000,$021F,$0000,$021E,$0220,$019E,$0000,$0222
    Data.w $0223,$0000,$0223,$0000,$0222,$0224,$0225,$0000
    Data.w $0225,$0000,$0224,$0226,$0227,$0000,$0227,$0000
    Data.w $0226,$0228,$0229,$0000,$0229,$0000,$0228,$022A
    Data.w $022B,$0000,$022B,$0000,$022A,$022C,$022D,$0000
    Data.w $022D,$0000,$022C,$022E,$022F,$0000,$022F,$0000
    Data.w $022E,$0230,$0231,$0000,$0231,$0000,$0230,$0232
    Data.w $0233,$0000,$0233,$0000,$0232,$023A,$2C65,$0000
    Data.w $023B,$023C,$0000,$023C,$0000,$023B,$023D,$019A
    Data.w $0000,$023E,$2C66,$0000,$023F,$0000,$2C7E,$0240
    Data.w $0000,$2C7F,$0241,$0242,$0000,$0242,$0000,$0241
    Data.w $0243,$0180,$0000,$0244,$0289,$0000,$0245,$028C
    Data.w $0000,$0246,$0247,$0000,$0247,$0000,$0246,$0248
    Data.w $0249,$0000,$0249,$0000,$0248,$024A,$024B,$0000
    Data.w $024B,$0000,$024A,$024C,$024D,$0000,$024D,$0000
    Data.w $024C,$024E,$024F,$0000,$024F,$0000,$024E,$0250
    Data.w $0000,$2C6F,$0251,$0000,$2C6D,$0252,$0000,$2C70
    Data.w $0253,$0000,$0181,$0254,$0000,$0186,$0256,$0000
    Data.w $0189,$0257,$0000,$018A,$0259,$0000,$018F,$025B
    Data.w $0000,$0190,$025C,$0000,$A7AB,$0260,$0000,$0193
    Data.w $0261,$0000,$A7AC,$0263,$0000,$0194,$0265,$0000
    Data.w $A78D,$0266,$0000,$A7AA,$0268,$0000,$0197,$0269
    Data.w $0000,$0196,$026A,$0000,$A7AE,$026B,$0000,$2C62
    Data.w $026C,$0000,$A7AD,$026F,$0000,$019C,$0271,$0000
    Data.w $2C6E,$0272,$0000,$019D,$0275,$0000,$019F,$027D
    Data.w $0000,$2C64,$0280,$0000,$01A6,$0282,$0000,$A7C5
    Data.w $0283,$0000,$01A9,$0287,$0000,$A7B1,$0288,$0000
    Data.w $01AE,$0289,$0000,$0244,$028A,$0000,$01B1,$028B
    Data.w $0000,$01B2,$028C,$0000,$0245,$0292,$0000,$01B7
    Data.w $029D,$0000,$A7B2,$029E,$0000,$A7B0,$0345,$0000
    Data.w $0399,$0370,$0371,$0000,$0371,$0000,$0370,$0372
    Data.w $0373,$0000,$0373,$0000,$0372,$0376,$0377,$0000
    Data.w $0377,$0000,$0376,$037B,$0000,$03FD,$037C,$0000
    Data.w $03FE,$037D,$0000,$03FF,$037F,$03F3,$0000,$0386
    Data.w $03AC,$0000,$0388,$03AD,$0000,$0389,$03AE,$0000
    Data.w $038A,$03AF,$0000,$038C,$03CC,$0000,$038E,$03CD
    Data.w $0000,$038F,$03CE,$0000,$0391,$03B1,$0000,$0392
    Data.w $03B2,$0000,$0393,$03B3,$0000,$0394,$03B4,$0000
    Data.w $0395,$03B5,$0000,$0396,$03B6,$0000,$0397,$03B7
    Data.w $0000,$0398,$03B8,$0000,$0399,$03B9,$0000,$039A
    Data.w $03BA,$0000,$039B,$03BB,$0000,$039C,$03BC,$0000
    Data.w $039D,$03BD,$0000,$039E,$03BE,$0000,$039F,$03BF
    Data.w $0000,$03A0,$03C0,$0000,$03A1,$03C1,$0000,$03A3
    Data.w $03C3,$0000,$03A4,$03C4,$0000,$03A5,$03C5,$0000
    Data.w $03A6,$03C6,$0000,$03A7,$03C7,$0000,$03A8,$03C8
    Data.w $0000,$03A9,$03C9,$0000,$03AA,$03CA,$0000,$03AB
    Data.w $03CB,$0000,$03AC,$0000,$0386,$03AD,$0000,$0388
    Data.w $03AE,$0000,$0389,$03AF,$0000,$038A,$03B1,$0000
    Data.w $0391,$03B2,$0000,$0392,$03B3,$0000,$0393,$03B4
    Data.w $0000,$0394,$03B5,$0000,$0395,$03B6,$0000,$0396
    Data.w $03B7,$0000,$0397,$03B8,$0000,$0398,$03B9,$0000
    Data.w $0399,$03BA,$0000,$039A,$03BB,$0000,$039B,$03BC
    Data.w $0000,$039C,$03BD,$0000,$039D,$03BE,$0000,$039E
    Data.w $03BF,$0000,$039F,$03C0,$0000,$03A0,$03C1,$0000
    Data.w $03A1,$03C2,$0000,$03A3,$03C3,$0000,$03A3,$03C4
    Data.w $0000,$03A4,$03C5,$0000,$03A5,$03C6,$0000,$03A6
    Data.w $03C7,$0000,$03A7,$03C8,$0000,$03A8,$03C9,$0000
    Data.w $03A9,$03CA,$0000,$03AA,$03CB,$0000,$03AB,$03CC
    Data.w $0000,$038C,$03CD,$0000,$038E,$03CE,$0000,$038F
    Data.w $03CF,$03D7,$0000,$03D0,$0000,$0392,$03D1,$0000
    Data.w $0398,$03D5,$0000,$03A6,$03D6,$0000,$03A0,$03D7
    Data.w $0000,$03CF,$03D8,$03D9,$0000,$03D9,$0000,$03D8
    Data.w $03DA,$03DB,$0000,$03DB,$0000,$03DA,$03DC,$03DD
    Data.w $0000,$03DD,$0000,$03DC,$03DE,$03DF,$0000,$03DF
    Data.w $0000,$03DE,$03E0,$03E1,$0000,$03E1,$0000,$03E0
    Data.w $03E2,$03E3,$0000,$03E3,$0000,$03E2,$03E4,$03E5
    Data.w $0000,$03E5,$0000,$03E4,$03E6,$03E7,$0000,$03E7
    Data.w $0000,$03E6,$03E8,$03E9,$0000,$03E9,$0000,$03E8
    Data.w $03EA,$03EB,$0000,$03EB,$0000,$03EA,$03EC,$03ED
    Data.w $0000,$03ED,$0000,$03EC,$03EE,$03EF,$0000,$03EF
    Data.w $0000,$03EE,$03F0,$0000,$039A,$03F1,$0000,$03A1
    Data.w $03F2,$0000,$03F9,$03F3,$0000,$037F,$03F4,$03B8
    Data.w $0000,$03F5,$0000,$0395,$03F7,$03F8,$0000,$03F8
    Data.w $0000,$03F7,$03F9,$03F2,$0000,$03FA,$03FB,$0000
    Data.w $03FB,$0000,$03FA,$03FD,$037B,$0000,$03FE,$037C
    Data.w $0000,$03FF,$037D,$0000,$0400,$0450,$0000,$0401
    Data.w $0451,$0000,$0402,$0452,$0000,$0403,$0453,$0000
    Data.w $0404,$0454,$0000,$0405,$0455,$0000,$0406,$0456
    Data.w $0000,$0407,$0457,$0000,$0408,$0458,$0000,$0409
    Data.w $0459,$0000,$040A,$045A,$0000,$040B,$045B,$0000
    Data.w $040C,$045C,$0000,$040D,$045D,$0000,$040E,$045E
    Data.w $0000,$040F,$045F,$0000,$0410,$0430,$0000,$0411
    Data.w $0431,$0000,$0412,$0432,$0000,$0413,$0433,$0000
    Data.w $0414,$0434,$0000,$0415,$0435,$0000,$0416,$0436
    Data.w $0000,$0417,$0437,$0000,$0418,$0438,$0000,$0419
    Data.w $0439,$0000,$041A,$043A,$0000,$041B,$043B,$0000
    Data.w $041C,$043C,$0000,$041D,$043D,$0000,$041E,$043E
    Data.w $0000,$041F,$043F,$0000,$0420,$0440,$0000,$0421
    Data.w $0441,$0000,$0422,$0442,$0000,$0423,$0443,$0000
    Data.w $0424,$0444,$0000,$0425,$0445,$0000,$0426,$0446
    Data.w $0000,$0427,$0447,$0000,$0428,$0448,$0000,$0429
    Data.w $0449,$0000,$042A,$044A,$0000,$042B,$044B,$0000
    Data.w $042C,$044C,$0000,$042D,$044D,$0000,$042E,$044E
    Data.w $0000,$042F,$044F,$0000,$0430,$0000,$0410,$0431
    Data.w $0000,$0411,$0432,$0000,$0412,$0433,$0000,$0413
    Data.w $0434,$0000,$0414,$0435,$0000,$0415,$0436,$0000
    Data.w $0416,$0437,$0000,$0417,$0438,$0000,$0418,$0439
    Data.w $0000,$0419,$043A,$0000,$041A,$043B,$0000,$041B
    Data.w $043C,$0000,$041C,$043D,$0000,$041D,$043E,$0000
    Data.w $041E,$043F,$0000,$041F,$0440,$0000,$0420,$0441
    Data.w $0000,$0421,$0442,$0000,$0422,$0443,$0000,$0423
    Data.w $0444,$0000,$0424,$0445,$0000,$0425,$0446,$0000
    Data.w $0426,$0447,$0000,$0427,$0448,$0000,$0428,$0449
    Data.w $0000,$0429,$044A,$0000,$042A,$044B,$0000,$042B
    Data.w $044C,$0000,$042C,$044D,$0000,$042D,$044E,$0000
    Data.w $042E,$044F,$0000,$042F,$0450,$0000,$0400,$0451
    Data.w $0000,$0401,$0452,$0000,$0402,$0453,$0000,$0403
    Data.w $0454,$0000,$0404,$0455,$0000,$0405,$0456,$0000
    Data.w $0406,$0457,$0000,$0407,$0458,$0000,$0408,$0459
    Data.w $0000,$0409,$045A,$0000,$040A,$045B,$0000,$040B
    Data.w $045C,$0000,$040C,$045D,$0000,$040D,$045E,$0000
    Data.w $040E,$045F,$0000,$040F,$0460,$0461,$0000,$0461
    Data.w $0000,$0460,$0462,$0463,$0000,$0463,$0000,$0462
    Data.w $0464,$0465,$0000,$0465,$0000,$0464,$0466,$0467
    Data.w $0000,$0467,$0000,$0466,$0468,$0469,$0000,$0469
    Data.w $0000,$0468,$046A,$046B,$0000,$046B,$0000,$046A
    Data.w $046C,$046D,$0000,$046D,$0000,$046C,$046E,$046F
    Data.w $0000,$046F,$0000,$046E,$0470,$0471,$0000,$0471
    Data.w $0000,$0470,$0472,$0473,$0000,$0473,$0000,$0472
    Data.w $0474,$0475,$0000,$0475,$0000,$0474,$0476,$0477
    Data.w $0000,$0477,$0000,$0476,$0478,$0479,$0000,$0479
    Data.w $0000,$0478,$047A,$047B,$0000,$047B,$0000,$047A
    Data.w $047C,$047D,$0000,$047D,$0000,$047C,$047E,$047F
    Data.w $0000,$047F,$0000,$047E,$0480,$0481,$0000,$0481
    Data.w $0000,$0480,$048A,$048B,$0000,$048B,$0000,$048A
    Data.w $048C,$048D,$0000,$048D,$0000,$048C,$048E,$048F
    Data.w $0000,$048F,$0000,$048E,$0490,$0491,$0000,$0491
    Data.w $0000,$0490,$0492,$0493,$0000,$0493,$0000,$0492
    Data.w $0494,$0495,$0000,$0495,$0000,$0494,$0496,$0497
    Data.w $0000,$0497,$0000,$0496,$0498,$0499,$0000,$0499
    Data.w $0000,$0498,$049A,$049B,$0000,$049B,$0000,$049A
    Data.w $049C,$049D,$0000,$049D,$0000,$049C,$049E,$049F
    Data.w $0000,$049F,$0000,$049E,$04A0,$04A1,$0000,$04A1
    Data.w $0000,$04A0,$04A2,$04A3,$0000,$04A3,$0000,$04A2
    Data.w $04A4,$04A5,$0000,$04A5,$0000,$04A4,$04A6,$04A7
    Data.w $0000,$04A7,$0000,$04A6,$04A8,$04A9,$0000,$04A9
    Data.w $0000,$04A8,$04AA,$04AB,$0000,$04AB,$0000,$04AA
    Data.w $04AC,$04AD,$0000,$04AD,$0000,$04AC,$04AE,$04AF
    Data.w $0000,$04AF,$0000,$04AE,$04B0,$04B1,$0000,$04B1
    Data.w $0000,$04B0,$04B2,$04B3,$0000,$04B3,$0000,$04B2
    Data.w $04B4,$04B5,$0000,$04B5,$0000,$04B4,$04B6,$04B7
    Data.w $0000,$04B7,$0000,$04B6,$04B8,$04B9,$0000,$04B9
    Data.w $0000,$04B8,$04BA,$04BB,$0000,$04BB,$0000,$04BA
    Data.w $04BC,$04BD,$0000,$04BD,$0000,$04BC,$04BE,$04BF
    Data.w $0000,$04BF,$0000,$04BE,$04C0,$04CF,$0000,$04C1
    Data.w $04C2,$0000,$04C2,$0000,$04C1,$04C3,$04C4,$0000
    Data.w $04C4,$0000,$04C3,$04C5,$04C6,$0000,$04C6,$0000
    Data.w $04C5,$04C7,$04C8,$0000,$04C8,$0000,$04C7,$04C9
    Data.w $04CA,$0000,$04CA,$0000,$04C9,$04CB,$04CC,$0000
    Data.w $04CC,$0000,$04CB,$04CD,$04CE,$0000,$04CE,$0000
    Data.w $04CD,$04CF,$0000,$04C0,$04D0,$04D1,$0000,$04D1
    Data.w $0000,$04D0,$04D2,$04D3,$0000,$04D3,$0000,$04D2
    Data.w $04D4,$04D5,$0000,$04D5,$0000,$04D4,$04D6,$04D7
    Data.w $0000,$04D7,$0000,$04D6,$04D8,$04D9,$0000,$04D9
    Data.w $0000,$04D8,$04DA,$04DB,$0000,$04DB,$0000,$04DA
    Data.w $04DC,$04DD,$0000,$04DD,$0000,$04DC,$04DE,$04DF
    Data.w $0000,$04DF,$0000,$04DE,$04E0,$04E1,$0000,$04E1
    Data.w $0000,$04E0,$04E2,$04E3,$0000,$04E3,$0000,$04E2
    Data.w $04E4,$04E5,$0000,$04E5,$0000,$04E4,$04E6,$04E7
    Data.w $0000,$04E7,$0000,$04E6,$04E8,$04E9,$0000,$04E9
    Data.w $0000,$04E8,$04EA,$04EB,$0000,$04EB,$0000,$04EA
    Data.w $04EC,$04ED,$0000,$04ED,$0000,$04EC,$04EE,$04EF
    Data.w $0000,$04EF,$0000,$04EE,$04F0,$04F1,$0000,$04F1
    Data.w $0000,$04F0,$04F2,$04F3,$0000,$04F3,$0000,$04F2
    Data.w $04F4,$04F5,$0000,$04F5,$0000,$04F4,$04F6,$04F7
    Data.w $0000,$04F7,$0000,$04F6,$04F8,$04F9,$0000,$04F9
    Data.w $0000,$04F8,$04FA,$04FB,$0000,$04FB,$0000,$04FA
    Data.w $04FC,$04FD,$0000,$04FD,$0000,$04FC,$04FE,$04FF
    Data.w $0000,$04FF,$0000,$04FE,$0500,$0501,$0000,$0501
    Data.w $0000,$0500,$0502,$0503,$0000,$0503,$0000,$0502
    Data.w $0504,$0505,$0000,$0505,$0000,$0504,$0506,$0507
    Data.w $0000,$0507,$0000,$0506,$0508,$0509,$0000,$0509
    Data.w $0000,$0508,$050A,$050B,$0000,$050B,$0000,$050A
    Data.w $050C,$050D,$0000,$050D,$0000,$050C,$050E,$050F
    Data.w $0000,$050F,$0000,$050E,$0510,$0511,$0000,$0511
    Data.w $0000,$0510,$0512,$0513,$0000,$0513,$0000,$0512
    Data.w $0514,$0515,$0000,$0515,$0000,$0514,$0516,$0517
    Data.w $0000,$0517,$0000,$0516,$0518,$0519,$0000,$0519
    Data.w $0000,$0518,$051A,$051B,$0000,$051B,$0000,$051A
    Data.w $051C,$051D,$0000,$051D,$0000,$051C,$051E,$051F
    Data.w $0000,$051F,$0000,$051E,$0520,$0521,$0000,$0521
    Data.w $0000,$0520,$0522,$0523,$0000,$0523,$0000,$0522
    Data.w $0524,$0525,$0000,$0525,$0000,$0524,$0526,$0527
    Data.w $0000,$0527,$0000,$0526,$0528,$0529,$0000,$0529
    Data.w $0000,$0528,$052A,$052B,$0000,$052B,$0000,$052A
    Data.w $052C,$052D,$0000,$052D,$0000,$052C,$052E,$052F
    Data.w $0000,$052F,$0000,$052E,$0531,$0561,$0000,$0532
    Data.w $0562,$0000,$0533,$0563,$0000,$0534,$0564,$0000
    Data.w $0535,$0565,$0000,$0536,$0566,$0000,$0537,$0567
    Data.w $0000,$0538,$0568,$0000,$0539,$0569,$0000,$053A
    Data.w $056A,$0000,$053B,$056B,$0000,$053C,$056C,$0000
    Data.w $053D,$056D,$0000,$053E,$056E,$0000,$053F,$056F
    Data.w $0000,$0540,$0570,$0000,$0541,$0571,$0000,$0542
    Data.w $0572,$0000,$0543,$0573,$0000,$0544,$0574,$0000
    Data.w $0545,$0575,$0000,$0546,$0576,$0000,$0547,$0577
    Data.w $0000,$0548,$0578,$0000,$0549,$0579,$0000,$054A
    Data.w $057A,$0000,$054B,$057B,$0000,$054C,$057C,$0000
    Data.w $054D,$057D,$0000,$054E,$057E,$0000,$054F,$057F
    Data.w $0000,$0550,$0580,$0000,$0551,$0581,$0000,$0552
    Data.w $0582,$0000,$0553,$0583,$0000,$0554,$0584,$0000
    Data.w $0555,$0585,$0000,$0556,$0586,$0000,$0561,$0000
    Data.w $0531,$0562,$0000,$0532,$0563,$0000,$0533,$0564
    Data.w $0000,$0534,$0565,$0000,$0535,$0566,$0000,$0536
    Data.w $0567,$0000,$0537,$0568,$0000,$0538,$0569,$0000
    Data.w $0539,$056A,$0000,$053A,$056B,$0000,$053B,$056C
    Data.w $0000,$053C,$056D,$0000,$053D,$056E,$0000,$053E
    Data.w $056F,$0000,$053F,$0570,$0000,$0540,$0571,$0000
    Data.w $0541,$0572,$0000,$0542,$0573,$0000,$0543,$0574
    Data.w $0000,$0544,$0575,$0000,$0545,$0576,$0000,$0546
    Data.w $0577,$0000,$0547,$0578,$0000,$0548,$0579,$0000
    Data.w $0549,$057A,$0000,$054A,$057B,$0000,$054B,$057C
    Data.w $0000,$054C,$057D,$0000,$054D,$057E,$0000,$054E
    Data.w $057F,$0000,$054F,$0580,$0000,$0550,$0581,$0000
    Data.w $0551,$0582,$0000,$0552,$0583,$0000,$0553,$0584
    Data.w $0000,$0554,$0585,$0000,$0555,$0586,$0000,$0556
    Data.w $10A0,$2D00,$0000,$10A1,$2D01,$0000,$10A2,$2D02
    Data.w $0000,$10A3,$2D03,$0000,$10A4,$2D04,$0000,$10A5
    Data.w $2D05,$0000,$10A6,$2D06,$0000,$10A7,$2D07,$0000
    Data.w $10A8,$2D08,$0000,$10A9,$2D09,$0000,$10AA,$2D0A
    Data.w $0000,$10AB,$2D0B,$0000,$10AC,$2D0C,$0000,$10AD
    Data.w $2D0D,$0000,$10AE,$2D0E,$0000,$10AF,$2D0F,$0000
    Data.w $10B0,$2D10,$0000,$10B1,$2D11,$0000,$10B2,$2D12
    Data.w $0000,$10B3,$2D13,$0000,$10B4,$2D14,$0000,$10B5
    Data.w $2D15,$0000,$10B6,$2D16,$0000,$10B7,$2D17,$0000
    Data.w $10B8,$2D18,$0000,$10B9,$2D19,$0000,$10BA,$2D1A
    Data.w $0000,$10BB,$2D1B,$0000,$10BC,$2D1C,$0000,$10BD
    Data.w $2D1D,$0000,$10BE,$2D1E,$0000,$10BF,$2D1F,$0000
    Data.w $10C0,$2D20,$0000,$10C1,$2D21,$0000,$10C2,$2D22
    Data.w $0000,$10C3,$2D23,$0000,$10C4,$2D24,$0000,$10C5
    Data.w $2D25,$0000,$10C7,$2D27,$0000,$10CD,$2D2D,$0000
    Data.w $10D0,$0000,$10D0,$10D1,$0000,$10D1,$10D2,$0000
    Data.w $10D2,$10D3,$0000,$10D3,$10D4,$0000,$10D4,$10D5
    Data.w $0000,$10D5,$10D6,$0000,$10D6,$10D7,$0000,$10D7
    Data.w $10D8,$0000,$10D8,$10D9,$0000,$10D9,$10DA,$0000
    Data.w $10DA,$10DB,$0000,$10DB,$10DC,$0000,$10DC,$10DD
    Data.w $0000,$10DD,$10DE,$0000,$10DE,$10DF,$0000,$10DF
    Data.w $10E0,$0000,$10E0,$10E1,$0000,$10E1,$10E2,$0000
    Data.w $10E2,$10E3,$0000,$10E3,$10E4,$0000,$10E4,$10E5
    Data.w $0000,$10E5,$10E6,$0000,$10E6,$10E7,$0000,$10E7
    Data.w $10E8,$0000,$10E8,$10E9,$0000,$10E9,$10EA,$0000
    Data.w $10EA,$10EB,$0000,$10EB,$10EC,$0000,$10EC,$10ED
    Data.w $0000,$10ED,$10EE,$0000,$10EE,$10EF,$0000,$10EF
    Data.w $10F0,$0000,$10F0,$10F1,$0000,$10F1,$10F2,$0000
    Data.w $10F2,$10F3,$0000,$10F3,$10F4,$0000,$10F4,$10F5
    Data.w $0000,$10F5,$10F6,$0000,$10F6,$10F7,$0000,$10F7
    Data.w $10F8,$0000,$10F8,$10F9,$0000,$10F9,$10FA,$0000
    Data.w $10FA,$10FD,$0000,$10FD,$10FE,$0000,$10FE,$10FF
    Data.w $0000,$10FF,$13A0,$AB70,$0000,$13A1,$AB71,$0000
    Data.w $13A2,$AB72,$0000,$13A3,$AB73,$0000,$13A4,$AB74
    Data.w $0000,$13A5,$AB75,$0000,$13A6,$AB76,$0000,$13A7
    Data.w $AB77,$0000,$13A8,$AB78,$0000,$13A9,$AB79,$0000
    Data.w $13AA,$AB7A,$0000,$13AB,$AB7B,$0000,$13AC,$AB7C
    Data.w $0000,$13AD,$AB7D,$0000,$13AE,$AB7E,$0000,$13AF
    Data.w $AB7F,$0000,$13B0,$AB80,$0000,$13B1,$AB81,$0000
    Data.w $13B2,$AB82,$0000,$13B3,$AB83,$0000,$13B4,$AB84
    Data.w $0000,$13B5,$AB85,$0000,$13B6,$AB86,$0000,$13B7
    Data.w $AB87,$0000,$13B8,$AB88,$0000,$13B9,$AB89,$0000
    Data.w $13BA,$AB8A,$0000,$13BB,$AB8B,$0000,$13BC,$AB8C
    Data.w $0000,$13BD,$AB8D,$0000,$13BE,$AB8E,$0000,$13BF
    Data.w $AB8F,$0000,$13C0,$AB90,$0000,$13C1,$AB91,$0000
    Data.w $13C2,$AB92,$0000,$13C3,$AB93,$0000,$13C4,$AB94
    Data.w $0000,$13C5,$AB95,$0000,$13C6,$AB96,$0000,$13C7
    Data.w $AB97,$0000,$13C8,$AB98,$0000,$13C9,$AB99,$0000
    Data.w $13CA,$AB9A,$0000,$13CB,$AB9B,$0000,$13CC,$AB9C
    Data.w $0000,$13CD,$AB9D,$0000,$13CE,$AB9E,$0000,$13CF
    Data.w $AB9F,$0000,$13D0,$ABA0,$0000,$13D1,$ABA1,$0000
    Data.w $13D2,$ABA2,$0000,$13D3,$ABA3,$0000,$13D4,$ABA4
    Data.w $0000,$13D5,$ABA5,$0000,$13D6,$ABA6,$0000,$13D7
    Data.w $ABA7,$0000,$13D8,$ABA8,$0000,$13D9,$ABA9,$0000
    Data.w $13DA,$ABAA,$0000,$13DB,$ABAB,$0000,$13DC,$ABAC
    Data.w $0000,$13DD,$ABAD,$0000,$13DE,$ABAE,$0000,$13DF
    Data.w $ABAF,$0000,$13E0,$ABB0,$0000,$13E1,$ABB1,$0000
    Data.w $13E2,$ABB2,$0000,$13E3,$ABB3,$0000,$13E4,$ABB4
    Data.w $0000,$13E5,$ABB5,$0000,$13E6,$ABB6,$0000,$13E7
    Data.w $ABB7,$0000,$13E8,$ABB8,$0000,$13E9,$ABB9,$0000
    Data.w $13EA,$ABBA,$0000,$13EB,$ABBB,$0000,$13EC,$ABBC
    Data.w $0000,$13ED,$ABBD,$0000,$13EE,$ABBE,$0000,$13EF
    Data.w $ABBF,$0000,$13F0,$13F8,$0000,$13F1,$13F9,$0000
    Data.w $13F2,$13FA,$0000,$13F3,$13FB,$0000,$13F4,$13FC
    Data.w $0000,$13F5,$13FD,$0000,$13F8,$0000,$13F0,$13F9
    Data.w $0000,$13F1,$13FA,$0000,$13F2,$13FB,$0000,$13F3
    Data.w $13FC,$0000,$13F4,$13FD,$0000,$13F5,$1C80,$0000
    Data.w $0412,$1C81,$0000,$0414,$1C82,$0000,$041E,$1C83
    Data.w $0000,$0421,$1C84,$0000,$0422,$1C85,$0000,$0422
    Data.w $1C86,$0000,$042A,$1C87,$0000,$0462,$1C88,$0000
    Data.w $A64A,$1C90,$10D0,$0000,$1C91,$10D1,$0000,$1C92
    Data.w $10D2,$0000,$1C93,$10D3,$0000,$1C94,$10D4,$0000
    Data.w $1C95,$10D5,$0000,$1C96,$10D6,$0000,$1C97,$10D7
    Data.w $0000,$1C98,$10D8,$0000,$1C99,$10D9,$0000,$1C9A
    Data.w $10DA,$0000,$1C9B,$10DB,$0000,$1C9C,$10DC,$0000
    Data.w $1C9D,$10DD,$0000,$1C9E,$10DE,$0000,$1C9F,$10DF
    Data.w $0000,$1CA0,$10E0,$0000,$1CA1,$10E1,$0000,$1CA2
    Data.w $10E2,$0000,$1CA3,$10E3,$0000,$1CA4,$10E4,$0000
    Data.w $1CA5,$10E5,$0000,$1CA6,$10E6,$0000,$1CA7,$10E7
    Data.w $0000,$1CA8,$10E8,$0000,$1CA9,$10E9,$0000,$1CAA
    Data.w $10EA,$0000,$1CAB,$10EB,$0000,$1CAC,$10EC,$0000
    Data.w $1CAD,$10ED,$0000,$1CAE,$10EE,$0000,$1CAF,$10EF
    Data.w $0000,$1CB0,$10F0,$0000,$1CB1,$10F1,$0000,$1CB2
    Data.w $10F2,$0000,$1CB3,$10F3,$0000,$1CB4,$10F4,$0000
    Data.w $1CB5,$10F5,$0000,$1CB6,$10F6,$0000,$1CB7,$10F7
    Data.w $0000,$1CB8,$10F8,$0000,$1CB9,$10F9,$0000,$1CBA
    Data.w $10FA,$0000,$1CBD,$10FD,$0000,$1CBE,$10FE,$0000
    Data.w $1CBF,$10FF,$0000,$1D79,$0000,$A77D,$1D7D,$0000
    Data.w $2C63,$1D8E,$0000,$A7C6,$1E00,$1E01,$0000,$1E01
    Data.w $0000,$1E00,$1E02,$1E03,$0000,$1E03,$0000,$1E02
    Data.w $1E04,$1E05,$0000,$1E05,$0000,$1E04,$1E06,$1E07
    Data.w $0000,$1E07,$0000,$1E06,$1E08,$1E09,$0000,$1E09
    Data.w $0000,$1E08,$1E0A,$1E0B,$0000,$1E0B,$0000,$1E0A
    Data.w $1E0C,$1E0D,$0000,$1E0D,$0000,$1E0C,$1E0E,$1E0F
    Data.w $0000,$1E0F,$0000,$1E0E,$1E10,$1E11,$0000,$1E11
    Data.w $0000,$1E10,$1E12,$1E13,$0000,$1E13,$0000,$1E12
    Data.w $1E14,$1E15,$0000,$1E15,$0000,$1E14,$1E16,$1E17
    Data.w $0000,$1E17,$0000,$1E16,$1E18,$1E19,$0000,$1E19
    Data.w $0000,$1E18,$1E1A,$1E1B,$0000,$1E1B,$0000,$1E1A
    Data.w $1E1C,$1E1D,$0000,$1E1D,$0000,$1E1C,$1E1E,$1E1F
    Data.w $0000,$1E1F,$0000,$1E1E,$1E20,$1E21,$0000,$1E21
    Data.w $0000,$1E20,$1E22,$1E23,$0000,$1E23,$0000,$1E22
    Data.w $1E24,$1E25,$0000,$1E25,$0000,$1E24,$1E26,$1E27
    Data.w $0000,$1E27,$0000,$1E26,$1E28,$1E29,$0000,$1E29
    Data.w $0000,$1E28,$1E2A,$1E2B,$0000,$1E2B,$0000,$1E2A
    Data.w $1E2C,$1E2D,$0000,$1E2D,$0000,$1E2C,$1E2E,$1E2F
    Data.w $0000,$1E2F,$0000,$1E2E,$1E30,$1E31,$0000,$1E31
    Data.w $0000,$1E30,$1E32,$1E33,$0000,$1E33,$0000,$1E32
    Data.w $1E34,$1E35,$0000,$1E35,$0000,$1E34,$1E36,$1E37
    Data.w $0000,$1E37,$0000,$1E36,$1E38,$1E39,$0000,$1E39
    Data.w $0000,$1E38,$1E3A,$1E3B,$0000,$1E3B,$0000,$1E3A
    Data.w $1E3C,$1E3D,$0000,$1E3D,$0000,$1E3C,$1E3E,$1E3F
    Data.w $0000,$1E3F,$0000,$1E3E,$1E40,$1E41,$0000,$1E41
    Data.w $0000,$1E40,$1E42,$1E43,$0000,$1E43,$0000,$1E42
    Data.w $1E44,$1E45,$0000,$1E45,$0000,$1E44,$1E46,$1E47
    Data.w $0000,$1E47,$0000,$1E46,$1E48,$1E49,$0000,$1E49
    Data.w $0000,$1E48,$1E4A,$1E4B,$0000,$1E4B,$0000,$1E4A
    Data.w $1E4C,$1E4D,$0000,$1E4D,$0000,$1E4C,$1E4E,$1E4F
    Data.w $0000,$1E4F,$0000,$1E4E,$1E50,$1E51,$0000,$1E51
    Data.w $0000,$1E50,$1E52,$1E53,$0000,$1E53,$0000,$1E52
    Data.w $1E54,$1E55,$0000,$1E55,$0000,$1E54,$1E56,$1E57
    Data.w $0000,$1E57,$0000,$1E56,$1E58,$1E59,$0000,$1E59
    Data.w $0000,$1E58,$1E5A,$1E5B,$0000,$1E5B,$0000,$1E5A
    Data.w $1E5C,$1E5D,$0000,$1E5D,$0000,$1E5C,$1E5E,$1E5F
    Data.w $0000,$1E5F,$0000,$1E5E,$1E60,$1E61,$0000,$1E61
    Data.w $0000,$1E60,$1E62,$1E63,$0000,$1E63,$0000,$1E62
    Data.w $1E64,$1E65,$0000,$1E65,$0000,$1E64,$1E66,$1E67
    Data.w $0000,$1E67,$0000,$1E66,$1E68,$1E69,$0000,$1E69
    Data.w $0000,$1E68,$1E6A,$1E6B,$0000,$1E6B,$0000,$1E6A
    Data.w $1E6C,$1E6D,$0000,$1E6D,$0000,$1E6C,$1E6E,$1E6F
    Data.w $0000,$1E6F,$0000,$1E6E,$1E70,$1E71,$0000,$1E71
    Data.w $0000,$1E70,$1E72,$1E73,$0000,$1E73,$0000,$1E72
    Data.w $1E74,$1E75,$0000,$1E75,$0000,$1E74,$1E76,$1E77
    Data.w $0000,$1E77,$0000,$1E76,$1E78,$1E79,$0000,$1E79
    Data.w $0000,$1E78,$1E7A,$1E7B,$0000,$1E7B,$0000,$1E7A
    Data.w $1E7C,$1E7D,$0000,$1E7D,$0000,$1E7C,$1E7E,$1E7F
    Data.w $0000,$1E7F,$0000,$1E7E,$1E80,$1E81,$0000,$1E81
    Data.w $0000,$1E80,$1E82,$1E83,$0000,$1E83,$0000,$1E82
    Data.w $1E84,$1E85,$0000,$1E85,$0000,$1E84,$1E86,$1E87
    Data.w $0000,$1E87,$0000,$1E86,$1E88,$1E89,$0000,$1E89
    Data.w $0000,$1E88,$1E8A,$1E8B,$0000,$1E8B,$0000,$1E8A
    Data.w $1E8C,$1E8D,$0000,$1E8D,$0000,$1E8C,$1E8E,$1E8F
    Data.w $0000,$1E8F,$0000,$1E8E,$1E90,$1E91,$0000,$1E91
    Data.w $0000,$1E90,$1E92,$1E93,$0000,$1E93,$0000,$1E92
    Data.w $1E94,$1E95,$0000,$1E95,$0000,$1E94,$1E9B,$0000
    Data.w $1E60,$1E9E,$00DF,$0000,$1EA0,$1EA1,$0000,$1EA1
    Data.w $0000,$1EA0,$1EA2,$1EA3,$0000,$1EA3,$0000,$1EA2
    Data.w $1EA4,$1EA5,$0000,$1EA5,$0000,$1EA4,$1EA6,$1EA7
    Data.w $0000,$1EA7,$0000,$1EA6,$1EA8,$1EA9,$0000,$1EA9
    Data.w $0000,$1EA8,$1EAA,$1EAB,$0000,$1EAB,$0000,$1EAA
    Data.w $1EAC,$1EAD,$0000,$1EAD,$0000,$1EAC,$1EAE,$1EAF
    Data.w $0000,$1EAF,$0000,$1EAE,$1EB0,$1EB1,$0000,$1EB1
    Data.w $0000,$1EB0,$1EB2,$1EB3,$0000,$1EB3,$0000,$1EB2
    Data.w $1EB4,$1EB5,$0000,$1EB5,$0000,$1EB4,$1EB6,$1EB7
    Data.w $0000,$1EB7,$0000,$1EB6,$1EB8,$1EB9,$0000,$1EB9
    Data.w $0000,$1EB8,$1EBA,$1EBB,$0000,$1EBB,$0000,$1EBA
    Data.w $1EBC,$1EBD,$0000,$1EBD,$0000,$1EBC,$1EBE,$1EBF
    Data.w $0000,$1EBF,$0000,$1EBE,$1EC0,$1EC1,$0000,$1EC1
    Data.w $0000,$1EC0,$1EC2,$1EC3,$0000,$1EC3,$0000,$1EC2
    Data.w $1EC4,$1EC5,$0000,$1EC5,$0000,$1EC4,$1EC6,$1EC7
    Data.w $0000,$1EC7,$0000,$1EC6,$1EC8,$1EC9,$0000,$1EC9
    Data.w $0000,$1EC8,$1ECA,$1ECB,$0000,$1ECB,$0000,$1ECA
    Data.w $1ECC,$1ECD,$0000,$1ECD,$0000,$1ECC,$1ECE,$1ECF
    Data.w $0000,$1ECF,$0000,$1ECE,$1ED0,$1ED1,$0000,$1ED1
    Data.w $0000,$1ED0,$1ED2,$1ED3,$0000,$1ED3,$0000,$1ED2
    Data.w $1ED4,$1ED5,$0000,$1ED5,$0000,$1ED4,$1ED6,$1ED7
    Data.w $0000,$1ED7,$0000,$1ED6,$1ED8,$1ED9,$0000,$1ED9
    Data.w $0000,$1ED8,$1EDA,$1EDB,$0000,$1EDB,$0000,$1EDA
    Data.w $1EDC,$1EDD,$0000,$1EDD,$0000,$1EDC,$1EDE,$1EDF
    Data.w $0000,$1EDF,$0000,$1EDE,$1EE0,$1EE1,$0000,$1EE1
    Data.w $0000,$1EE0,$1EE2,$1EE3,$0000,$1EE3,$0000,$1EE2
    Data.w $1EE4,$1EE5,$0000,$1EE5,$0000,$1EE4,$1EE6,$1EE7
    Data.w $0000,$1EE7,$0000,$1EE6,$1EE8,$1EE9,$0000,$1EE9
    Data.w $0000,$1EE8,$1EEA,$1EEB,$0000,$1EEB,$0000,$1EEA
    Data.w $1EEC,$1EED,$0000,$1EED,$0000,$1EEC,$1EEE,$1EEF
    Data.w $0000,$1EEF,$0000,$1EEE,$1EF0,$1EF1,$0000,$1EF1
    Data.w $0000,$1EF0,$1EF2,$1EF3,$0000,$1EF3,$0000,$1EF2
    Data.w $1EF4,$1EF5,$0000,$1EF5,$0000,$1EF4,$1EF6,$1EF7
    Data.w $0000,$1EF7,$0000,$1EF6,$1EF8,$1EF9,$0000,$1EF9
    Data.w $0000,$1EF8,$1EFA,$1EFB,$0000,$1EFB,$0000,$1EFA
    Data.w $1EFC,$1EFD,$0000,$1EFD,$0000,$1EFC,$1EFE,$1EFF
    Data.w $0000,$1EFF,$0000,$1EFE,$1F00,$0000,$1F08,$1F01
    Data.w $0000,$1F09,$1F02,$0000,$1F0A,$1F03,$0000,$1F0B
    Data.w $1F04,$0000,$1F0C,$1F05,$0000,$1F0D,$1F06,$0000
    Data.w $1F0E,$1F07,$0000,$1F0F,$1F08,$1F00,$0000,$1F09
    Data.w $1F01,$0000,$1F0A,$1F02,$0000,$1F0B,$1F03,$0000
    Data.w $1F0C,$1F04,$0000,$1F0D,$1F05,$0000,$1F0E,$1F06
    Data.w $0000,$1F0F,$1F07,$0000,$1F10,$0000,$1F18,$1F11
    Data.w $0000,$1F19,$1F12,$0000,$1F1A,$1F13,$0000,$1F1B
    Data.w $1F14,$0000,$1F1C,$1F15,$0000,$1F1D,$1F18,$1F10
    Data.w $0000,$1F19,$1F11,$0000,$1F1A,$1F12,$0000,$1F1B
    Data.w $1F13,$0000,$1F1C,$1F14,$0000,$1F1D,$1F15,$0000
    Data.w $1F20,$0000,$1F28,$1F21,$0000,$1F29,$1F22,$0000
    Data.w $1F2A,$1F23,$0000,$1F2B,$1F24,$0000,$1F2C,$1F25
    Data.w $0000,$1F2D,$1F26,$0000,$1F2E,$1F27,$0000,$1F2F
    Data.w $1F28,$1F20,$0000,$1F29,$1F21,$0000,$1F2A,$1F22
    Data.w $0000,$1F2B,$1F23,$0000,$1F2C,$1F24,$0000,$1F2D
    Data.w $1F25,$0000,$1F2E,$1F26,$0000,$1F2F,$1F27,$0000
    Data.w $1F30,$0000,$1F38,$1F31,$0000,$1F39,$1F32,$0000
    Data.w $1F3A,$1F33,$0000,$1F3B,$1F34,$0000,$1F3C,$1F35
    Data.w $0000,$1F3D,$1F36,$0000,$1F3E,$1F37,$0000,$1F3F
    Data.w $1F38,$1F30,$0000,$1F39,$1F31,$0000,$1F3A,$1F32
    Data.w $0000,$1F3B,$1F33,$0000,$1F3C,$1F34,$0000,$1F3D
    Data.w $1F35,$0000,$1F3E,$1F36,$0000,$1F3F,$1F37,$0000
    Data.w $1F40,$0000,$1F48,$1F41,$0000,$1F49,$1F42,$0000
    Data.w $1F4A,$1F43,$0000,$1F4B,$1F44,$0000,$1F4C,$1F45
    Data.w $0000,$1F4D,$1F48,$1F40,$0000,$1F49,$1F41,$0000
    Data.w $1F4A,$1F42,$0000,$1F4B,$1F43,$0000,$1F4C,$1F44
    Data.w $0000,$1F4D,$1F45,$0000,$1F51,$0000,$1F59,$1F53
    Data.w $0000,$1F5B,$1F55,$0000,$1F5D,$1F57,$0000,$1F5F
    Data.w $1F59,$1F51,$0000,$1F5B,$1F53,$0000,$1F5D,$1F55
    Data.w $0000,$1F5F,$1F57,$0000,$1F60,$0000,$1F68,$1F61
    Data.w $0000,$1F69,$1F62,$0000,$1F6A,$1F63,$0000,$1F6B
    Data.w $1F64,$0000,$1F6C,$1F65,$0000,$1F6D,$1F66,$0000
    Data.w $1F6E,$1F67,$0000,$1F6F,$1F68,$1F60,$0000,$1F69
    Data.w $1F61,$0000,$1F6A,$1F62,$0000,$1F6B,$1F63,$0000
    Data.w $1F6C,$1F64,$0000,$1F6D,$1F65,$0000,$1F6E,$1F66
    Data.w $0000,$1F6F,$1F67,$0000,$1F70,$0000,$1FBA,$1F71
    Data.w $0000,$1FBB,$1F72,$0000,$1FC8,$1F73,$0000,$1FC9
    Data.w $1F74,$0000,$1FCA,$1F75,$0000,$1FCB,$1F76,$0000
    Data.w $1FDA,$1F77,$0000,$1FDB,$1F78,$0000,$1FF8,$1F79
    Data.w $0000,$1FF9,$1F7A,$0000,$1FEA,$1F7B,$0000,$1FEB
    Data.w $1F7C,$0000,$1FFA,$1F7D,$0000,$1FFB,$1F80,$0000
    Data.w $1F88,$1F81,$0000,$1F89,$1F82,$0000,$1F8A,$1F83
    Data.w $0000,$1F8B,$1F84,$0000,$1F8C,$1F85,$0000,$1F8D
    Data.w $1F86,$0000,$1F8E,$1F87,$0000,$1F8F,$1F88,$1F80
    Data.w $0000,$1F89,$1F81,$0000,$1F8A,$1F82,$0000,$1F8B
    Data.w $1F83,$0000,$1F8C,$1F84,$0000,$1F8D,$1F85,$0000
    Data.w $1F8E,$1F86,$0000,$1F8F,$1F87,$0000,$1F90,$0000
    Data.w $1F98,$1F91,$0000,$1F99,$1F92,$0000,$1F9A,$1F93
    Data.w $0000,$1F9B,$1F94,$0000,$1F9C,$1F95,$0000,$1F9D
    Data.w $1F96,$0000,$1F9E,$1F97,$0000,$1F9F,$1F98,$1F90
    Data.w $0000,$1F99,$1F91,$0000,$1F9A,$1F92,$0000,$1F9B
    Data.w $1F93,$0000,$1F9C,$1F94,$0000,$1F9D,$1F95,$0000
    Data.w $1F9E,$1F96,$0000,$1F9F,$1F97,$0000,$1FA0,$0000
    Data.w $1FA8,$1FA1,$0000,$1FA9,$1FA2,$0000,$1FAA,$1FA3
    Data.w $0000,$1FAB,$1FA4,$0000,$1FAC,$1FA5,$0000,$1FAD
    Data.w $1FA6,$0000,$1FAE,$1FA7,$0000,$1FAF,$1FA8,$1FA0
    Data.w $0000,$1FA9,$1FA1,$0000,$1FAA,$1FA2,$0000,$1FAB
    Data.w $1FA3,$0000,$1FAC,$1FA4,$0000,$1FAD,$1FA5,$0000
    Data.w $1FAE,$1FA6,$0000,$1FAF,$1FA7,$0000,$1FB0,$0000
    Data.w $1FB8,$1FB1,$0000,$1FB9,$1FB3,$0000,$1FBC,$1FB8
    Data.w $1FB0,$0000,$1FB9,$1FB1,$0000,$1FBA,$1F70,$0000
    Data.w $1FBB,$1F71,$0000,$1FBC,$1FB3,$0000,$1FBE,$0000
    Data.w $0399,$1FC3,$0000,$1FCC,$1FC8,$1F72,$0000,$1FC9
    Data.w $1F73,$0000,$1FCA,$1F74,$0000,$1FCB,$1F75,$0000
    Data.w $1FCC,$1FC3,$0000,$1FD0,$0000,$1FD8,$1FD1,$0000
    Data.w $1FD9,$1FD8,$1FD0,$0000,$1FD9,$1FD1,$0000,$1FDA
    Data.w $1F76,$0000,$1FDB,$1F77,$0000,$1FE0,$0000,$1FE8
    Data.w $1FE1,$0000,$1FE9,$1FE5,$0000,$1FEC,$1FE8,$1FE0
    Data.w $0000,$1FE9,$1FE1,$0000,$1FEA,$1F7A,$0000,$1FEB
    Data.w $1F7B,$0000,$1FEC,$1FE5,$0000,$1FF3,$0000,$1FFC
    Data.w $1FF8,$1F78,$0000,$1FF9,$1F79,$0000,$1FFA,$1F7C
    Data.w $0000,$1FFB,$1F7D,$0000,$1FFC,$1FF3,$0000,$2126
    Data.w $03C9,$0000,$212A,$006B,$0000,$212B,$00E5,$0000
    Data.w $2132,$214E,$0000,$214E,$0000,$2132,$2160,$2170
    Data.w $0000,$2161,$2171,$0000,$2162,$2172,$0000,$2163
    Data.w $2173,$0000,$2164,$2174,$0000,$2165,$2175,$0000
    Data.w $2166,$2176,$0000,$2167,$2177,$0000,$2168,$2178
    Data.w $0000,$2169,$2179,$0000,$216A,$217A,$0000,$216B
    Data.w $217B,$0000,$216C,$217C,$0000,$216D,$217D,$0000
    Data.w $216E,$217E,$0000,$216F,$217F,$0000,$2170,$0000
    Data.w $2160,$2171,$0000,$2161,$2172,$0000,$2162,$2173
    Data.w $0000,$2163,$2174,$0000,$2164,$2175,$0000,$2165
    Data.w $2176,$0000,$2166,$2177,$0000,$2167,$2178,$0000
    Data.w $2168,$2179,$0000,$2169,$217A,$0000,$216A,$217B
    Data.w $0000,$216B,$217C,$0000,$216C,$217D,$0000,$216D
    Data.w $217E,$0000,$216E,$217F,$0000,$216F,$2183,$2184
    Data.w $0000,$2184,$0000,$2183,$24B6,$24D0,$0000,$24B7
    Data.w $24D1,$0000,$24B8,$24D2,$0000,$24B9,$24D3,$0000
    Data.w $24BA,$24D4,$0000,$24BB,$24D5,$0000,$24BC,$24D6
    Data.w $0000,$24BD,$24D7,$0000,$24BE,$24D8,$0000,$24BF
    Data.w $24D9,$0000,$24C0,$24DA,$0000,$24C1,$24DB,$0000
    Data.w $24C2,$24DC,$0000,$24C3,$24DD,$0000,$24C4,$24DE
    Data.w $0000,$24C5,$24DF,$0000,$24C6,$24E0,$0000,$24C7
    Data.w $24E1,$0000,$24C8,$24E2,$0000,$24C9,$24E3,$0000
    Data.w $24CA,$24E4,$0000,$24CB,$24E5,$0000,$24CC,$24E6
    Data.w $0000,$24CD,$24E7,$0000,$24CE,$24E8,$0000,$24CF
    Data.w $24E9,$0000,$24D0,$0000,$24B6,$24D1,$0000,$24B7
    Data.w $24D2,$0000,$24B8,$24D3,$0000,$24B9,$24D4,$0000
    Data.w $24BA,$24D5,$0000,$24BB,$24D6,$0000,$24BC,$24D7
    Data.w $0000,$24BD,$24D8,$0000,$24BE,$24D9,$0000,$24BF
    Data.w $24DA,$0000,$24C0,$24DB,$0000,$24C1,$24DC,$0000
    Data.w $24C2,$24DD,$0000,$24C3,$24DE,$0000,$24C4,$24DF
    Data.w $0000,$24C5,$24E0,$0000,$24C6,$24E1,$0000,$24C7
    Data.w $24E2,$0000,$24C8,$24E3,$0000,$24C9,$24E4,$0000
    Data.w $24CA,$24E5,$0000,$24CB,$24E6,$0000,$24CC,$24E7
    Data.w $0000,$24CD,$24E8,$0000,$24CE,$24E9,$0000,$24CF
    Data.w $2C00,$2C30,$0000,$2C01,$2C31,$0000,$2C02,$2C32
    Data.w $0000,$2C03,$2C33,$0000,$2C04,$2C34,$0000,$2C05
    Data.w $2C35,$0000,$2C06,$2C36,$0000,$2C07,$2C37,$0000
    Data.w $2C08,$2C38,$0000,$2C09,$2C39,$0000,$2C0A,$2C3A
    Data.w $0000,$2C0B,$2C3B,$0000,$2C0C,$2C3C,$0000,$2C0D
    Data.w $2C3D,$0000,$2C0E,$2C3E,$0000,$2C0F,$2C3F,$0000
    Data.w $2C10,$2C40,$0000,$2C11,$2C41,$0000,$2C12,$2C42
    Data.w $0000,$2C13,$2C43,$0000,$2C14,$2C44,$0000,$2C15
    Data.w $2C45,$0000,$2C16,$2C46,$0000,$2C17,$2C47,$0000
    Data.w $2C18,$2C48,$0000,$2C19,$2C49,$0000,$2C1A,$2C4A
    Data.w $0000,$2C1B,$2C4B,$0000,$2C1C,$2C4C,$0000,$2C1D
    Data.w $2C4D,$0000,$2C1E,$2C4E,$0000,$2C1F,$2C4F,$0000
    Data.w $2C20,$2C50,$0000,$2C21,$2C51,$0000,$2C22,$2C52
    Data.w $0000,$2C23,$2C53,$0000,$2C24,$2C54,$0000,$2C25
    Data.w $2C55,$0000,$2C26,$2C56,$0000,$2C27,$2C57,$0000
    Data.w $2C28,$2C58,$0000,$2C29,$2C59,$0000,$2C2A,$2C5A
    Data.w $0000,$2C2B,$2C5B,$0000,$2C2C,$2C5C,$0000,$2C2D
    Data.w $2C5D,$0000,$2C2E,$2C5E,$0000,$2C2F,$2C5F,$0000
    Data.w $2C30,$0000,$2C00,$2C31,$0000,$2C01,$2C32,$0000
    Data.w $2C02,$2C33,$0000,$2C03,$2C34,$0000,$2C04,$2C35
    Data.w $0000,$2C05,$2C36,$0000,$2C06,$2C37,$0000,$2C07
    Data.w $2C38,$0000,$2C08,$2C39,$0000,$2C09,$2C3A,$0000
    Data.w $2C0A,$2C3B,$0000,$2C0B,$2C3C,$0000,$2C0C,$2C3D
    Data.w $0000,$2C0D,$2C3E,$0000,$2C0E,$2C3F,$0000,$2C0F
    Data.w $2C40,$0000,$2C10,$2C41,$0000,$2C11,$2C42,$0000
    Data.w $2C12,$2C43,$0000,$2C13,$2C44,$0000,$2C14,$2C45
    Data.w $0000,$2C15,$2C46,$0000,$2C16,$2C47,$0000,$2C17
    Data.w $2C48,$0000,$2C18,$2C49,$0000,$2C19,$2C4A,$0000
    Data.w $2C1A,$2C4B,$0000,$2C1B,$2C4C,$0000,$2C1C,$2C4D
    Data.w $0000,$2C1D,$2C4E,$0000,$2C1E,$2C4F,$0000,$2C1F
    Data.w $2C50,$0000,$2C20,$2C51,$0000,$2C21,$2C52,$0000
    Data.w $2C22,$2C53,$0000,$2C23,$2C54,$0000,$2C24,$2C55
    Data.w $0000,$2C25,$2C56,$0000,$2C26,$2C57,$0000,$2C27
    Data.w $2C58,$0000,$2C28,$2C59,$0000,$2C29,$2C5A,$0000
    Data.w $2C2A,$2C5B,$0000,$2C2B,$2C5C,$0000,$2C2C,$2C5D
    Data.w $0000,$2C2D,$2C5E,$0000,$2C2E,$2C5F,$0000,$2C2F
    Data.w $2C60,$2C61,$0000,$2C61,$0000,$2C60,$2C62,$026B
    Data.w $0000,$2C63,$1D7D,$0000,$2C64,$027D,$0000,$2C65
    Data.w $0000,$023A,$2C66,$0000,$023E,$2C67,$2C68,$0000
    Data.w $2C68,$0000,$2C67,$2C69,$2C6A,$0000,$2C6A,$0000
    Data.w $2C69,$2C6B,$2C6C,$0000,$2C6C,$0000,$2C6B,$2C6D
    Data.w $0251,$0000,$2C6E,$0271,$0000,$2C6F,$0250,$0000
    Data.w $2C70,$0252,$0000,$2C72,$2C73,$0000,$2C73,$0000
    Data.w $2C72,$2C75,$2C76,$0000,$2C76,$0000,$2C75,$2C7E
    Data.w $023F,$0000,$2C7F,$0240,$0000,$2C80,$2C81,$0000
    Data.w $2C81,$0000,$2C80,$2C82,$2C83,$0000,$2C83,$0000
    Data.w $2C82,$2C84,$2C85,$0000,$2C85,$0000,$2C84,$2C86
    Data.w $2C87,$0000,$2C87,$0000,$2C86,$2C88,$2C89,$0000
    Data.w $2C89,$0000,$2C88,$2C8A,$2C8B,$0000,$2C8B,$0000
    Data.w $2C8A,$2C8C,$2C8D,$0000,$2C8D,$0000,$2C8C,$2C8E
    Data.w $2C8F,$0000,$2C8F,$0000,$2C8E,$2C90,$2C91,$0000
    Data.w $2C91,$0000,$2C90,$2C92,$2C93,$0000,$2C93,$0000
    Data.w $2C92,$2C94,$2C95,$0000,$2C95,$0000,$2C94,$2C96
    Data.w $2C97,$0000,$2C97,$0000,$2C96,$2C98,$2C99,$0000
    Data.w $2C99,$0000,$2C98,$2C9A,$2C9B,$0000,$2C9B,$0000
    Data.w $2C9A,$2C9C,$2C9D,$0000,$2C9D,$0000,$2C9C,$2C9E
    Data.w $2C9F,$0000,$2C9F,$0000,$2C9E,$2CA0,$2CA1,$0000
    Data.w $2CA1,$0000,$2CA0,$2CA2,$2CA3,$0000,$2CA3,$0000
    Data.w $2CA2,$2CA4,$2CA5,$0000,$2CA5,$0000,$2CA4,$2CA6
    Data.w $2CA7,$0000,$2CA7,$0000,$2CA6,$2CA8,$2CA9,$0000
    Data.w $2CA9,$0000,$2CA8,$2CAA,$2CAB,$0000,$2CAB,$0000
    Data.w $2CAA,$2CAC,$2CAD,$0000,$2CAD,$0000,$2CAC,$2CAE
    Data.w $2CAF,$0000,$2CAF,$0000,$2CAE,$2CB0,$2CB1,$0000
    Data.w $2CB1,$0000,$2CB0,$2CB2,$2CB3,$0000,$2CB3,$0000
    Data.w $2CB2,$2CB4,$2CB5,$0000,$2CB5,$0000,$2CB4,$2CB6
    Data.w $2CB7,$0000,$2CB7,$0000,$2CB6,$2CB8,$2CB9,$0000
    Data.w $2CB9,$0000,$2CB8,$2CBA,$2CBB,$0000,$2CBB,$0000
    Data.w $2CBA,$2CBC,$2CBD,$0000,$2CBD,$0000,$2CBC,$2CBE
    Data.w $2CBF,$0000,$2CBF,$0000,$2CBE,$2CC0,$2CC1,$0000
    Data.w $2CC1,$0000,$2CC0,$2CC2,$2CC3,$0000,$2CC3,$0000
    Data.w $2CC2,$2CC4,$2CC5,$0000,$2CC5,$0000,$2CC4,$2CC6
    Data.w $2CC7,$0000,$2CC7,$0000,$2CC6,$2CC8,$2CC9,$0000
    Data.w $2CC9,$0000,$2CC8,$2CCA,$2CCB,$0000,$2CCB,$0000
    Data.w $2CCA,$2CCC,$2CCD,$0000,$2CCD,$0000,$2CCC,$2CCE
    Data.w $2CCF,$0000,$2CCF,$0000,$2CCE,$2CD0,$2CD1,$0000
    Data.w $2CD1,$0000,$2CD0,$2CD2,$2CD3,$0000,$2CD3,$0000
    Data.w $2CD2,$2CD4,$2CD5,$0000,$2CD5,$0000,$2CD4,$2CD6
    Data.w $2CD7,$0000,$2CD7,$0000,$2CD6,$2CD8,$2CD9,$0000
    Data.w $2CD9,$0000,$2CD8,$2CDA,$2CDB,$0000,$2CDB,$0000
    Data.w $2CDA,$2CDC,$2CDD,$0000,$2CDD,$0000,$2CDC,$2CDE
    Data.w $2CDF,$0000,$2CDF,$0000,$2CDE,$2CE0,$2CE1,$0000
    Data.w $2CE1,$0000,$2CE0,$2CE2,$2CE3,$0000,$2CE3,$0000
    Data.w $2CE2,$2CEB,$2CEC,$0000,$2CEC,$0000,$2CEB,$2CED
    Data.w $2CEE,$0000,$2CEE,$0000,$2CED,$2CF2,$2CF3,$0000
    Data.w $2CF3,$0000,$2CF2,$2D00,$0000,$10A0,$2D01,$0000
    Data.w $10A1,$2D02,$0000,$10A2,$2D03,$0000,$10A3,$2D04
    Data.w $0000,$10A4,$2D05,$0000,$10A5,$2D06,$0000,$10A6
    Data.w $2D07,$0000,$10A7,$2D08,$0000,$10A8,$2D09,$0000
    Data.w $10A9,$2D0A,$0000,$10AA,$2D0B,$0000,$10AB,$2D0C
    Data.w $0000,$10AC,$2D0D,$0000,$10AD,$2D0E,$0000,$10AE
    Data.w $2D0F,$0000,$10AF,$2D10,$0000,$10B0,$2D11,$0000
    Data.w $10B1,$2D12,$0000,$10B2,$2D13,$0000,$10B3,$2D14
    Data.w $0000,$10B4,$2D15,$0000,$10B5,$2D16,$0000,$10B6
    Data.w $2D17,$0000,$10B7,$2D18,$0000,$10B8,$2D19,$0000
    Data.w $10B9,$2D1A,$0000,$10BA,$2D1B,$0000,$10BB,$2D1C
    Data.w $0000,$10BC,$2D1D,$0000,$10BD,$2D1E,$0000,$10BE
    Data.w $2D1F,$0000,$10BF,$2D20,$0000,$10C0,$2D21,$0000
    Data.w $10C1,$2D22,$0000,$10C2,$2D23,$0000,$10C3,$2D24
    Data.w $0000,$10C4,$2D25,$0000,$10C5,$2D27,$0000,$10C7
    Data.w $2D2D,$0000,$10CD,$A640,$A641,$0000,$A641,$0000
    Data.w $A640,$A642,$A643,$0000,$A643,$0000,$A642,$A644
    Data.w $A645,$0000,$A645,$0000,$A644,$A646,$A647,$0000
    Data.w $A647,$0000,$A646,$A648,$A649,$0000,$A649,$0000
    Data.w $A648,$A64A,$A64B,$0000,$A64B,$0000,$A64A,$A64C
    Data.w $A64D,$0000,$A64D,$0000,$A64C,$A64E,$A64F,$0000
    Data.w $A64F,$0000,$A64E,$A650,$A651,$0000,$A651,$0000
    Data.w $A650,$A652,$A653,$0000,$A653,$0000,$A652,$A654
    Data.w $A655,$0000,$A655,$0000,$A654,$A656,$A657,$0000
    Data.w $A657,$0000,$A656,$A658,$A659,$0000,$A659,$0000
    Data.w $A658,$A65A,$A65B,$0000,$A65B,$0000,$A65A,$A65C
    Data.w $A65D,$0000,$A65D,$0000,$A65C,$A65E,$A65F,$0000
    Data.w $A65F,$0000,$A65E,$A660,$A661,$0000,$A661,$0000
    Data.w $A660,$A662,$A663,$0000,$A663,$0000,$A662,$A664
    Data.w $A665,$0000,$A665,$0000,$A664,$A666,$A667,$0000
    Data.w $A667,$0000,$A666,$A668,$A669,$0000,$A669,$0000
    Data.w $A668,$A66A,$A66B,$0000,$A66B,$0000,$A66A,$A66C
    Data.w $A66D,$0000,$A66D,$0000,$A66C,$A680,$A681,$0000
    Data.w $A681,$0000,$A680,$A682,$A683,$0000,$A683,$0000
    Data.w $A682,$A684,$A685,$0000,$A685,$0000,$A684,$A686
    Data.w $A687,$0000,$A687,$0000,$A686,$A688,$A689,$0000
    Data.w $A689,$0000,$A688,$A68A,$A68B,$0000,$A68B,$0000
    Data.w $A68A,$A68C,$A68D,$0000,$A68D,$0000,$A68C,$A68E
    Data.w $A68F,$0000,$A68F,$0000,$A68E,$A690,$A691,$0000
    Data.w $A691,$0000,$A690,$A692,$A693,$0000,$A693,$0000
    Data.w $A692,$A694,$A695,$0000,$A695,$0000,$A694,$A696
    Data.w $A697,$0000,$A697,$0000,$A696,$A698,$A699,$0000
    Data.w $A699,$0000,$A698,$A69A,$A69B,$0000,$A69B,$0000
    Data.w $A69A,$A722,$A723,$0000,$A723,$0000,$A722,$A724
    Data.w $A725,$0000,$A725,$0000,$A724,$A726,$A727,$0000
    Data.w $A727,$0000,$A726,$A728,$A729,$0000,$A729,$0000
    Data.w $A728,$A72A,$A72B,$0000,$A72B,$0000,$A72A,$A72C
    Data.w $A72D,$0000,$A72D,$0000,$A72C,$A72E,$A72F,$0000
    Data.w $A72F,$0000,$A72E,$A732,$A733,$0000,$A733,$0000
    Data.w $A732,$A734,$A735,$0000,$A735,$0000,$A734,$A736
    Data.w $A737,$0000,$A737,$0000,$A736,$A738,$A739,$0000
    Data.w $A739,$0000,$A738,$A73A,$A73B,$0000,$A73B,$0000
    Data.w $A73A,$A73C,$A73D,$0000,$A73D,$0000,$A73C,$A73E
    Data.w $A73F,$0000,$A73F,$0000,$A73E,$A740,$A741,$0000
    Data.w $A741,$0000,$A740,$A742,$A743,$0000,$A743,$0000
    Data.w $A742,$A744,$A745,$0000,$A745,$0000,$A744,$A746
    Data.w $A747,$0000,$A747,$0000,$A746,$A748,$A749,$0000
    Data.w $A749,$0000,$A748,$A74A,$A74B,$0000,$A74B,$0000
    Data.w $A74A,$A74C,$A74D,$0000,$A74D,$0000,$A74C,$A74E
    Data.w $A74F,$0000,$A74F,$0000,$A74E,$A750,$A751,$0000
    Data.w $A751,$0000,$A750,$A752,$A753,$0000,$A753,$0000
    Data.w $A752,$A754,$A755,$0000,$A755,$0000,$A754,$A756
    Data.w $A757,$0000,$A757,$0000,$A756,$A758,$A759,$0000
    Data.w $A759,$0000,$A758,$A75A,$A75B,$0000,$A75B,$0000
    Data.w $A75A,$A75C,$A75D,$0000,$A75D,$0000,$A75C,$A75E
    Data.w $A75F,$0000,$A75F,$0000,$A75E,$A760,$A761,$0000
    Data.w $A761,$0000,$A760,$A762,$A763,$0000,$A763,$0000
    Data.w $A762,$A764,$A765,$0000,$A765,$0000,$A764,$A766
    Data.w $A767,$0000,$A767,$0000,$A766,$A768,$A769,$0000
    Data.w $A769,$0000,$A768,$A76A,$A76B,$0000,$A76B,$0000
    Data.w $A76A,$A76C,$A76D,$0000,$A76D,$0000,$A76C,$A76E
    Data.w $A76F,$0000,$A76F,$0000,$A76E,$A779,$A77A,$0000
    Data.w $A77A,$0000,$A779,$A77B,$A77C,$0000,$A77C,$0000
    Data.w $A77B,$A77D,$1D79,$0000,$A77E,$A77F,$0000,$A77F
    Data.w $0000,$A77E,$A780,$A781,$0000,$A781,$0000,$A780
    Data.w $A782,$A783,$0000,$A783,$0000,$A782,$A784,$A785
    Data.w $0000,$A785,$0000,$A784,$A786,$A787,$0000,$A787
    Data.w $0000,$A786,$A78B,$A78C,$0000,$A78C,$0000,$A78B
    Data.w $A78D,$0265,$0000,$A790,$A791,$0000,$A791,$0000
    Data.w $A790,$A792,$A793,$0000,$A793,$0000,$A792,$A794
    Data.w $0000,$A7C4,$A796,$A797,$0000,$A797,$0000,$A796
    Data.w $A798,$A799,$0000,$A799,$0000,$A798,$A79A,$A79B
    Data.w $0000,$A79B,$0000,$A79A,$A79C,$A79D,$0000,$A79D
    Data.w $0000,$A79C,$A79E,$A79F,$0000,$A79F,$0000,$A79E
    Data.w $A7A0,$A7A1,$0000,$A7A1,$0000,$A7A0,$A7A2,$A7A3
    Data.w $0000,$A7A3,$0000,$A7A2,$A7A4,$A7A5,$0000,$A7A5
    Data.w $0000,$A7A4,$A7A6,$A7A7,$0000,$A7A7,$0000,$A7A6
    Data.w $A7A8,$A7A9,$0000,$A7A9,$0000,$A7A8,$A7AA,$0266
    Data.w $0000,$A7AB,$025C,$0000,$A7AC,$0261,$0000,$A7AD
    Data.w $026C,$0000,$A7AE,$026A,$0000,$A7B0,$029E,$0000
    Data.w $A7B1,$0287,$0000,$A7B2,$029D,$0000,$A7B3,$AB53
    Data.w $0000,$A7B4,$A7B5,$0000,$A7B5,$0000,$A7B4,$A7B6
    Data.w $A7B7,$0000,$A7B7,$0000,$A7B6,$A7B8,$A7B9,$0000
    Data.w $A7B9,$0000,$A7B8,$A7BA,$A7BB,$0000,$A7BB,$0000
    Data.w $A7BA,$A7BC,$A7BD,$0000,$A7BD,$0000,$A7BC,$A7BE
    Data.w $A7BF,$0000,$A7BF,$0000,$A7BE,$A7C0,$A7C1,$0000
    Data.w $A7C1,$0000,$A7C0,$A7C2,$A7C3,$0000,$A7C3,$0000
    Data.w $A7C2,$A7C4,$A794,$0000,$A7C5,$0282,$0000,$A7C6
    Data.w $1D8E,$0000,$A7C7,$A7C8,$0000,$A7C8,$0000,$A7C7
    Data.w $A7C9,$A7CA,$0000,$A7CA,$0000,$A7C9,$A7D0,$A7D1
    Data.w $0000,$A7D1,$0000,$A7D0,$A7D6,$A7D7,$0000,$A7D7
    Data.w $0000,$A7D6,$A7D8,$A7D9,$0000,$A7D9,$0000,$A7D8
    Data.w $A7F5,$A7F6,$0000,$A7F6,$0000,$A7F5,$AB53,$0000
    Data.w $A7B3,$AB70,$0000,$13A0,$AB71,$0000,$13A1,$AB72
    Data.w $0000,$13A2,$AB73,$0000,$13A3,$AB74,$0000,$13A4
    Data.w $AB75,$0000,$13A5,$AB76,$0000,$13A6,$AB77,$0000
    Data.w $13A7,$AB78,$0000,$13A8,$AB79,$0000,$13A9,$AB7A
    Data.w $0000,$13AA,$AB7B,$0000,$13AB,$AB7C,$0000,$13AC
    Data.w $AB7D,$0000,$13AD,$AB7E,$0000,$13AE,$AB7F,$0000
    Data.w $13AF,$AB80,$0000,$13B0,$AB81,$0000,$13B1,$AB82
    Data.w $0000,$13B2,$AB83,$0000,$13B3,$AB84,$0000,$13B4
    Data.w $AB85,$0000,$13B5,$AB86,$0000,$13B6,$AB87,$0000
    Data.w $13B7,$AB88,$0000,$13B8,$AB89,$0000,$13B9,$AB8A
    Data.w $0000,$13BA,$AB8B,$0000,$13BB,$AB8C,$0000,$13BC
    Data.w $AB8D,$0000,$13BD,$AB8E,$0000,$13BE,$AB8F,$0000
    Data.w $13BF,$AB90,$0000,$13C0,$AB91,$0000,$13C1,$AB92
    Data.w $0000,$13C2,$AB93,$0000,$13C3,$AB94,$0000,$13C4
    Data.w $AB95,$0000,$13C5,$AB96,$0000,$13C6,$AB97,$0000
    Data.w $13C7,$AB98,$0000,$13C8,$AB99,$0000,$13C9,$AB9A
    Data.w $0000,$13CA,$AB9B,$0000,$13CB,$AB9C,$0000,$13CC
    Data.w $AB9D,$0000,$13CD,$AB9E,$0000,$13CE,$AB9F,$0000
    Data.w $13CF,$ABA0,$0000,$13D0,$ABA1,$0000,$13D1,$ABA2
    Data.w $0000,$13D2,$ABA3,$0000,$13D3,$ABA4,$0000,$13D4
    Data.w $ABA5,$0000,$13D5,$ABA6,$0000,$13D6,$ABA7,$0000
    Data.w $13D7,$ABA8,$0000,$13D8,$ABA9,$0000,$13D9,$ABAA
    Data.w $0000,$13DA,$ABAB,$0000,$13DB,$ABAC,$0000,$13DC
    Data.w $ABAD,$0000,$13DD,$ABAE,$0000,$13DE,$ABAF,$0000
    Data.w $13DF,$ABB0,$0000,$13E0,$ABB1,$0000,$13E1,$ABB2
    Data.w $0000,$13E2,$ABB3,$0000,$13E3,$ABB4,$0000,$13E4
    Data.w $ABB5,$0000,$13E5,$ABB6,$0000,$13E6,$ABB7,$0000
    Data.w $13E7,$ABB8,$0000,$13E8,$ABB9,$0000,$13E9,$ABBA
    Data.w $0000,$13EA,$ABBB,$0000,$13EB,$ABBC,$0000,$13EC
    Data.w $ABBD,$0000,$13ED,$ABBE,$0000,$13EE,$ABBF,$0000
    Data.w $13EF,$FF21,$FF41,$0000,$FF22,$FF42,$0000,$FF23
    Data.w $FF43,$0000,$FF24,$FF44,$0000,$FF25,$FF45,$0000
    Data.w $FF26,$FF46,$0000,$FF27,$FF47,$0000,$FF28,$FF48
    Data.w $0000,$FF29,$FF49,$0000,$FF2A,$FF4A,$0000,$FF2B
    Data.w $FF4B,$0000,$FF2C,$FF4C,$0000,$FF2D,$FF4D,$0000
    Data.w $FF2E,$FF4E,$0000,$FF2F,$FF4F,$0000,$FF30,$FF50
    Data.w $0000,$FF31,$FF51,$0000,$FF32,$FF52,$0000,$FF33
    Data.w $FF53,$0000,$FF34,$FF54,$0000,$FF35,$FF55,$0000
    Data.w $FF36,$FF56,$0000,$FF37,$FF57,$0000,$FF38,$FF58
    Data.w $0000,$FF39,$FF59,$0000,$FF3A,$FF5A,$0000,$FF41
    Data.w $0000,$FF21,$FF42,$0000,$FF22,$FF43,$0000,$FF23
    Data.w $FF44,$0000,$FF24,$FF45,$0000,$FF25,$FF46,$0000
    Data.w $FF26,$FF47,$0000,$FF27,$FF48,$0000,$FF28,$FF49
    Data.w $0000,$FF29,$FF4A,$0000,$FF2A,$FF4B,$0000,$FF2B
    Data.w $FF4C,$0000,$FF2C,$FF4D,$0000,$FF2D,$FF4E,$0000
    Data.w $FF2E,$FF4F,$0000,$FF2F,$FF50,$0000,$FF30,$FF51
    Data.w $0000,$FF31,$FF52,$0000,$FF32,$FF53,$0000,$FF33
    Data.w $FF54,$0000,$FF34,$FF55,$0000,$FF35,$FF56,$0000
    Data.w $FF36,$FF57,$0000,$FF37,$FF58,$0000,$FF38,$FF59
    Data.w $0000,$FF39,$FF5A,$0000,$FF3A,$00,$00,$00
    
  EndDataSection 
  
EndProcedure 

UNI_Init() 

Procedure _UpCase(*in.Unicode)  ;changes the case of the string inplace 
  
  Protected *char.Unicode, t.u 
  *char = *in 
  While *char\u  
    tchar = UNI_UCase(*char\u) 
    If tchar 
      *char\u = tchar 
    EndIf   
    *char+2 
  Wend 
EndProcedure 

Procedure _LowCase(*in.Unicode)  ;changes the case of the string inplace 
  Protected *char.Unicode, t.u 
  *char = *in 
  While *char\u  
    tchar = UNI_LCase(*char\u) 
    If tchar 
      *char\u = tchar 
    EndIf   
    *char+2 
  Wend 
EndProcedure 

Procedure _StrCmpNC(*a.Unicode,*b.Unicode)  ;compare two strings with no case  
    
  While UNI_LCase(*a\u) = UNI_LCase(*b\u)  
    *a+2
    *b+2 
    If (*a\u = 0 And *b\u = 0) 
      ProcedureReturn 1 
    ElseIf (*a\u = 0 Or *b\u = 0) 
      ProcedureReturn 0 
    EndIf   
  Wend   
  
EndProcedure   

Prototype UpCase(in.p-unicode)   
Prototype LowCase(in.p-unicode) 
Prototype StrCmpNC(a.p-unicode,b.p-unicode) 
Global UpCase.UpCase = @_upcase()
Global LowCase.LowCase = @_lowcase()
Global StrCmpNC.StrCmpNC = @_StrCmpNC() 

t.s = "abcdef 0123456789, äöü, äöü, áóú" 
UpCase(t) 
Debug "upper case  " + t 
tU.s = t 
LowCase(t) 
Debug "lower case  " + t 
t = "ABCDEF" + Mid(t,7) 

Cherokee_Letter_A$ = Chr($13A0)
LowCase(Cherokee_Letter_A$)
Debug Cherokee_Letter_A$ 
UpCase(Cherokee_Letter_A$)  
Debug Cherokee_Letter_A$ 

Cherokee_Small_Letter_A$ = Chr($AB70)
UpCase(Cherokee_Small_Letter_A$) 
Debug Cherokee_Small_Letter_A$ 
LowCase(Cherokee_Small_Letter_A$)
Debug Cherokee_Small_Letter_A$

Debug "is equal "  + strcmpnc(t,tu)

t = Chr($00B5)
Debug t 
UpCase(t)
Debug t + " " + Hex(Asc(t))
; correct is '39C'


juergenkulow
Enthusiast
Enthusiast
Posts: 544
Joined: Wed Sep 25, 2019 10:18 am

Re: Upper and Lower Case Mapping for Unicode

Post by juergenkulow »

Highest Unicode value is U+10FFFD.
Affected case-sensitive fonts are DESERET U+10400, OSAGE U+104B0,
VITHKUQI U+10570, OLD HUNGARIAN U+10C80, WARANG CITI U+118A0, MEDEFAIDRIN U+16E40, MATHEMATICAL U+1D400 and ADLAM U+1E900.
https://www.unicode.org/Public/14.0.0/u ... deData.txt
Please ask your questions, because switch on the cognition apparatus decides on the only known life in the universe.Wersten :DDüsseldorf NRW Germany Europe Earth Solar System Flake Bubble Orionarm
Milky Way Local_Group Virgo Supercluster Laniakea Universe
User avatar
Demivec
Addict
Addict
Posts: 4086
Joined: Mon Jul 25, 2005 3:51 pm
Location: Utah, USA

Re: Upper and Lower Case Mapping for Unicode

Post by Demivec »

I second the post juergenkulow made. My point being that in the code samples posted thus far, arrays and data being read and stored in datasections for unicode code points is using only 16 bit word and unicode data types. They should be 32 bit longs or quads (to avoid signs).

The procedures for converting to uppercase and lowercase need to account for characters outside the Basic Multilingual Plane and handle the possibility of characters taking two PureBasic unicode string characters to express. This same issue involves handling a 2 byte code being replaced by a 4 byte code and vice versa. Both of these things complicate the in place string conversion and would really necessitate the possibility of creating a new string instead.
User avatar
Sicro
Enthusiast
Enthusiast
Posts: 538
Joined: Wed Jun 25, 2014 5:25 pm
Location: Germany
Contact:

Re: Upper and Lower Case Mapping for Unicode

Post by Sicro »

idle wrote: Sun Aug 14, 2022 10:48 pm This seems to be right now.
Thanks.
juergenkulow wrote: Mon Aug 15, 2022 2:50 pm Highest Unicode value is U+10FFFD.
PureBasic officially uses UCS-2 for Unicode, which only supports characters from 0x0000 to 0xFFFF:
https://www.purebasic.com/documentation ... icode.html

Although UCS-2 and UTF-16 are identical in this area and UTF-16 "hacks" may work, it is not officially supported according to the PureBasic help.
Image
Why OpenSource should have a license :: PB-CodeArchiv-Rebirth :: Pleasant-Dark (syntax color scheme) :: RegEx-Engine (compiles RegExes to NFA/DFA)
Manjaro Xfce x64 (Main system) :: Windows 10 Home (VirtualBox) :: Newest PureBasic version
juergenkulow
Enthusiast
Enthusiast
Posts: 544
Joined: Wed Sep 25, 2019 10:18 am

Re: Upper and Lower Case Mapping for Unicode

Post by juergenkulow »

Operating systems before Windows 2000 supported only UCS-2.
UTF-16 is supported since Windows 2000.
But Fred decided in 2015: Re: PureBASIC internal encoding of unicode, UCS-2 or UTF-16?
Fred wrote: Thu Mar 05, 2015 3:54 pm For the record, PB uses UCS2 string encoding internally when unicode mode is ON (it doesn't support multibyte UTF16 encoding).
User avatar
Demivec
Addict
Addict
Posts: 4086
Joined: Mon Jul 25, 2005 3:51 pm
Location: Utah, USA

Re: Upper and Lower Case Mapping for Unicode

Post by Demivec »

juergenkulow wrote: Wed Aug 17, 2022 5:27 pm Operating systems before Windows 2000 supported only UCS-2.
UTF-16 is supported since Windows 2000.
But Fred decided in 2015: Re: PureBASIC internal encoding of unicode, UCS-2 or UTF-16?
Fred wrote: Thu Mar 05, 2015 3:54 pm For the record, PB uses UCS2 string encoding internally when unicode mode is ON (it doesn't support multibyte UTF16 encoding).
Here is a link to another post with discussion and demonstration code to highlight how PureBasic uses USC-2:
https://www.purebasic.fr/english/viewtopic.php?p=569391

The main point is that PureBasic uses USC-2 for string functions only. The strings used as parameters by other functions typically accept UTF-16.
User avatar
mk-soft
Always Here
Always Here
Posts: 5335
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Upper and Lower Case Mapping for Unicode

Post by mk-soft »

Perhaps work this ...

Update v1.02.1

Code: Select all

;-TOP by mk-soft, v1.02.1

Procedure.s uppercaseString(String.s)
  Protected NSPool, NSLocale, NSString, NSString2, NSString3, *String
  Protected result.s
  
  If String
    NSPool = CocoaMessage(0, 0, "NSAutoreleasePool new")
    NSLocale = CocoaMessage(0, 0, "NSLocale currentLocale")
    NSString = CocoaMessage(0, 0, "NSString alloc")
    NSString2 = CocoaMessage(0, NSString, "initWithBytes:", @String, "length:", StringByteLength(String), "encoding:", #NSUTF16LittleEndianStringEncoding)
    NSString3 = CocoaMessage(0, NSString2, "uppercaseStringWithLocale:", NSLocale) 
    *String = CocoaMessage(0, NSString3, "cStringUsingEncoding:", #NSUTF16LittleEndianStringEncoding)
    If *String
      result = PeekS(*String)
    EndIf
    CocoaMessage(0, NSPool, "release")
  EndIf
  ProcedureReturn result
EndProcedure

Procedure.s lowercaseString(String.s)
  Protected NSPool, NSLocale, NSString, NSString2, NSString3, *String
  Protected result.s
  
  If String
    NSPool = CocoaMessage(0, 0, "NSAutoreleasePool new")
    NSLocale = CocoaMessage(0, 0, "NSLocale currentLocale")
    NSString = CocoaMessage(0, 0, "NSString alloc")
    NSString2 = CocoaMessage(0, NSString, "initWithBytes:", @String, "length:", StringByteLength(String), "encoding:", #NSUTF16LittleEndianStringEncoding)
    NSString3 = CocoaMessage(0, NSString2, "lowercaseStringWithLocale:", NSLocale) 
    *String = CocoaMessage(0, NSString3, "cStringUsingEncoding:", #NSUTF16LittleEndianStringEncoding)
    If *String
      result = PeekS(*String)
    EndIf
    CocoaMessage(0, NSPool, "release")
  EndIf
  ProcedureReturn result
EndProcedure

;sVal.s = "Hello World! äöü"
sVal.s = "abcdef 0123456789, äöü, äöü, áóú"

r1.s = uppercaseString(sVal)
Debug r1

r1.s = lowercaseString(r1)
Debug r1
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
Sicro
Enthusiast
Enthusiast
Posts: 538
Joined: Wed Jun 25, 2014 5:25 pm
Location: Germany
Contact:

Re: Upper and Lower Case Mapping for Unicode

Post by Sicro »

Demivec wrote: Wed Aug 17, 2022 7:59 pm Here is a link to another post with discussion and demonstration code to highlight how PureBasic uses USC-2:
viewtopic.php?p=569391

The main point is that PureBasic uses USC-2 for string functions only. The strings used as parameters by other functions typically accept UTF-16.
Yes, as I wrote in the discussion you linked, these are the OS API functions that evaluate the strings as UTF-16.

On my Linux, the demonstration code you posted in the discussion also works.

@mk-soft: Does the demonstration code also work on your macOS?

If now also on macOS the PB strings are interpreted as UTF-16 when they are displayed, then we should definitely write a case mapping code that supports UTF-16 instead of only UCS-2. I agree.

mk-soft wrote: Wed Aug 17, 2022 10:09 pm Perhaps work this ...

[macOS API code]
I think we should stay with the UnicodeData file to create a solution that will give the same results under all OS. As known, there are different versions of the UnicodeData file (currently version 14.0, version 15.0 is already being worked on). When we use the OS functions for case mapping, we don't know which UnicodeData file version the WinAPI, Linux API and macOS API use, and if we do know, these versions may be different under the different OS APIs.
Image
Why OpenSource should have a license :: PB-CodeArchiv-Rebirth :: Pleasant-Dark (syntax color scheme) :: RegEx-Engine (compiles RegExes to NFA/DFA)
Manjaro Xfce x64 (Main system) :: Windows 10 Home (VirtualBox) :: Newest PureBasic version
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: Upper and Lower Case Mapping for Unicode

Post by idle »

I modified Demivecs _Chr() procedure

Code: Select all

Procedure.s _Chr(v.i) ;return a proper surrogate pair for unicode values outside the BMP (Basic Multilingual Plane)
  Protected high, low
  If v < $10000
    ProcedureReturn Chr(v)
  Else
    ;calculate surrogate pair of unicode codepoints to represent value in UTF-16
    v - $10000
    high = v >> 10 + $D800 ;high/lead surrogate value
    low = v & $3FF + $DC00  ;low/tail surrogate value
    ProcedureReturn Chr(high) + Chr(low)
  EndIf
EndProcedure 
I'm having a brain fart about reversing it to _UNC()
User avatar
Demivec
Addict
Addict
Posts: 4086
Joined: Mon Jul 25, 2005 3:51 pm
Location: Utah, USA

Re: Upper and Lower Case Mapping for Unicode

Post by Demivec »

idle wrote:I'm having a brain fart about reversing it to _UNC()
I'm guessing you are trying to perform something like a UTF-16 version of the Asc() function. If so you will find it in the thread linked below. Simply rename it or modify it if necessary to suit your purposes.

https://www.purebasic.fr/english/viewtopic.php?t=64947
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: Upper and Lower Case Mapping for Unicode

Post by idle »

Demivec wrote: Sun Aug 21, 2022 4:42 am
idle wrote:I'm having a brain fart about reversing it to _UNC()
I'm guessing you are trying to perform something like a UTF-16 version of the Asc() function. If so you will find it in the thread linked below. Simply rename it or modify it if necessary to suit your purposes.

https://www.purebasic.fr/english/viewtopic.php?t=64947
Thank you, I almost had it right.
User avatar
mk-soft
Always Here
Always Here
Posts: 5335
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Upper and Lower Case Mapping for Unicode

Post by mk-soft »

Update v1.07
- Change table format
- New unicode version v15.0 (lastest)

The table should now be correct.
Had to split the code.

Data section builder for new unicode version...

Code: Select all

; Case mapping table file:
; https://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt

; UnicodeData file structure documentation:
; https://www.unicode.org/L2/L1999/UnicodeData.html

; Base by Sicro, 26.06.2022

; DataSection builder by mk-soft

EnableExplicit

Define file$, line$, code$, upperCase$, lowerCase$, titleCase$
Define code

file$ = GetHomeDirectory() + "UnicodeData.txt"

If MessageRequester("Question", "Load UnicodeData.txt", #PB_MessageRequester_Warning | #PB_MessageRequester_YesNo) = #PB_MessageRequester_Yes
 If Not ReceiveHTTPFile("https://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt", file$)
   Debug "Download error"
    End
  EndIf
EndIf

If Not ReadFile(0, file$)
  Debug "Read file error"
  End
EndIf

Debug "  DataSection"
Debug "    ; Code, Upper, Lower, Title"
Debug "    CaseFolding:"

While Not Eof(0)
  line$ = ReadString(0)
  code$ = StringField(line$, 1, ";")
  code = Val("$" + code$)
  If code > $FFFF
    ; PureBasic supports only character codes up to $FFFF
    Break
  EndIf
  upperCase$ = StringField(line$, 13, ";")
  lowerCase$ = StringField(line$, 14, ";")
  titleCase$ = StringField(line$, 15, ";")
  If upperCase$ Or lowerCase$ Or titleCase$
    If titleCase$ = ""
      titleCase$ = code$
    EndIf
    If upperCase$ = ""
      upperCase$ = code$
    EndIf
    If lowerCase$ = ""
      lowerCase$ = code$
    EndIf
    Debug "    Data.c $" + code$ + ",$" + upperCase$ + ",$" + lowerCase$ + ",$" + titleCase$
  EndIf
Wend

Debug "    Data.c $0000"
Debug "  EndDataSection"

CloseFile(0)
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
Sicro
Enthusiast
Enthusiast
Posts: 538
Joined: Wed Jun 25, 2014 5:25 pm
Location: Germany
Contact:

Re: Upper and Lower Case Mapping for Unicode

Post by Sicro »

Your codes seem to work correctly now. Thanks.

@idle: Strictly speaking, your function _StrCmpNC() is not correct because case-folding is usually applied for comparing, but as your function is, I think it can be useful as well.
Image
Why OpenSource should have a license :: PB-CodeArchiv-Rebirth :: Pleasant-Dark (syntax color scheme) :: RegEx-Engine (compiles RegExes to NFA/DFA)
Manjaro Xfce x64 (Main system) :: Windows 10 Home (VirtualBox) :: Newest PureBasic version
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: Upper and Lower Case Mapping for Unicode

Post by idle »

Sicro wrote: Sun Nov 13, 2022 12:53 pm Your codes seem to work correctly now. Thanks.

@idle: Strictly speaking, your function _StrCmpNC() is not correct because case-folding is usually applied for comparing, but as your function is, I think it can be useful as well.
It only does the simple case, I would need to implement
https://www.unicode.org/Public/UCD/late ... olding.txt
the goal for example will say that "MASSE" and "Maße" are equal.
Post Reply