Another Webcam Demo Program

Share your advanced PureBasic knowledge/code with the community.
chris319
Enthusiast
Enthusiast
Posts: 782
Joined: Mon Oct 24, 2005 1:05 pm

Another Webcam Demo Program

Post by chris319 »

Here is a webcam demo program I have written. It uses escapi and processes each pixel individually. The processing code runs fast enough to keep up with the frame rate provided the debugger is off. I get a full 30 fps if the screen resolution is 640 x 360. The frame rate goes down if the resolution is 1280 x 720. The camera itself seems to be the frame-rate bottleneck, depending on the resolution.

Press CTL-Q to exit the program.

The program uses look-up tables rather than performing floating-point calculations on the fly. It also draws to an image gadget.

For testing purposes you can skip the pixel-by-pixel processing code by uncommenting line 86.

http://sol.gfxile.net/escapi/

Code: Select all

;WORKS WITH ESCAPI
;UPDATED 8/28/2017
;PRESS CTL-Q TO QUIT
;INCORPORATES PROC AMP

ExamineDesktops()

gamma.f = 1.34 ;WEBCAM OUTPUT IS ALREADY GAMMA CORRECTED
#PEDESTAL = 16
gain.f = (235-16)/235
gain * 0.97

factor.f = 255/Pow(255,gamma)

;SET WEBCAM TO DEFAULTS AND SHARPNESS TO 255

;MAKE LUTS
Dim gammaTable.f(256)
For cnt = 0 To 255
cnf.f = cnt
gammaTable.f(cnt)=cnt;(Pow(cnf,gamma) * factor * gain) + #PEDESTAL
Next

Dim x3table.l(1921)
For cnt = 0 To 1920
x3table(cnt) = cnt * 3
Next

LoadFont(1,"Arial",24)

IncludeFile "escapi.pbi"
Global WIDTH = 640;DesktopWidth(0)
Global HEIGHT = 360;DesktopHeight(0)
Global WIDTHM1 = WIDTH - 1
Global HEIGHTM1 = HEIGHT - 1
Global pixCount = (WIDTH * HEIGHT) - 2

Global Dim pixcolor.l(WIDTH, HEIGHT): Global Dim unsmoothedY.d(WIDTH, HEIGHT)
Global Dim Cr.d(WIDTH, HEIGHT): Global Dim Y.d(WIDTH, HEIGHT): Global Dim Cb.d(WIDTH, HEIGHT)
Global imHeight, imWidth, xCoord, yCoord,Rd,Gd,Bd

#DEVICE = 0    
If setupESCAPI() = #Null
      MessageRequester("Error", "Unable to initialize ESCAPI.")
    End
EndIf

    bufSize = WIDTH * HEIGHT * 4
    scp.SimpleCapParams
    scp\mWidth = WIDTH
    scp\mHeight = HEIGHT
    scp\mTargetBuf = AllocateMemory(bufSize)
    *buf = scp\mTargetBuf

    If initCapture(#DEVICE, @scp)
     
image = CreateImage(1, WIDTH, HEIGHT, 24)
OpenWindow(1, 0, 0, WIDTH, HEIGHT,"",#PB_Window_BorderLess)
AddKeyboardShortcut(1, #PB_Shortcut_Control|#PB_Shortcut_Q, 113);CTL Q TO QUIT
ImageGadget(0, 0, 0, WIDTH, HEIGHT, ImageID(1))
Quit = #False

StartDrawing(ImageOutput(1))
*writeBuffer = DrawingBuffer()
pitch = DrawingBufferPitch()
StopDrawing()

Repeat
If WindowEvent() = #PB_Event_Menu ;KEYBOARD INPUT
If EventMenu() = 113
  Quit = #True
EndIf
EndIf       
     
doCapture(#DEVICE)

Repeat: Delay(1):Until isCaptureDone(#DEVICE) <> #False

;If isCaptureDone(#DEVICE) <> #False
;If isCaptureDone(#DEVICE) = #False

;PIXEL-BY-PIXEL READING AND WRITING
hm1 = *writebuffer + (HEIGHTM1 * pitch)
*bufoff = *buf

;Goto skip
For y = 0 To HEIGHTM1
For x = 0 To WIDTHM1
x3 = hm1 + x3table(x)

p1.l = PeekL(*bufoff)

PokeA(x3,gammaTable(p1 & 255))
PokeA(x3+1,gammaTable(p1 >> 8 & 255))
PokeA(x3+2,gammaTable(p1 >> 16))

*bufoff + 4
Next
hm1 - pitch
Next

skip:
SetGadgetState(0, ImageID(1))

StartDrawing(WindowOutput(1))
DrawingFont(FontID(1))

now.f = ElapsedMilliseconds()
fps.f = now.f-then.f

If fps > 0:fps$ = StrF((1/fps)*1000,2) ;:EndIf
If (1/fps)*1000 < 10: fps$ = "0" + fps$: EndIf
EndIf

DrawText(100, 200, fps$+" fps",#White)
then.f = ElapsedMilliseconds()
StopDrawing()

Until Quit = #True

deinitCapture(#DEVICE)
FreeImage(1)
FreeMemory(scp\mTargetBuf)
CloseWindow(1)

    Else
      Debug "Init capture failed."
    EndIf

    End
walbus
Addict
Addict
Posts: 929
Joined: Sat Mar 02, 2013 9:17 am

Re: Another Webcam Demo Program

Post by walbus »

Hi,
i have maked a quick look to your code (Before the first coffee)
Primary, i think
The frames text output is immediately, this is not good
This make also immediately a start and stop drawing in the main loop

I think, its recommended to change this, at sample so :
Set a windowed counter all ~500ms, then make this output
And looking before you make this output, is the frame rate changed
Is the frame rate not changed make this text output not !
The output directly on the window is also not a good idea, i think

The PokeA method is mostly fast enough for a image sized about 1024*768, but with more, its to slow
Older machines, i think, can not output this 30 frames
So, i think, further, a other buffer handling is recommended

The Delay(1) construction inside the main loop is a handbrake and looking a little strange
(I have not looking the pbi to time)
chris319
Enthusiast
Enthusiast
Posts: 782
Joined: Mon Oct 24, 2005 1:05 pm

Re: Another Webcam Demo Program

Post by chris319 »

This is a demo program. In actual use you wouldn't display the fps text over video.
Set a windowed counter all ~500ms, then make this output
You can't do that with video. The program must synchronize with an external device (the camera).
Older machines, i think, can not output this 30 frames
Then they'll either have to get a better computer or live with a lower frame rate.
walbus
Addict
Addict
Posts: 929
Joined: Sat Mar 02, 2013 9:17 am

Re: Another Webcam Demo Program

Post by walbus »

I self think, your primary main loop construction is the problem, never this can works good

And the world is full older machines, we should try making software for all
Not all users have money for new machines $
Also i see not a problem for a frames/s output
IdeasVacuum
Always Here
Always Here
Posts: 6425
Joined: Fri Oct 23, 2009 2:33 am
Location: Wales, UK
Contact:

Re: Another Webcam Demo Program

Post by IdeasVacuum »

Thanks for sharing your code chris319 8)

A couple of questions:
Where is file escapi.pbi?
What version of escapi is required?
IdeasVacuum
If it sounds simple, you have not grasped the complexity.
chris319
Enthusiast
Enthusiast
Posts: 782
Joined: Mon Oct 24, 2005 1:05 pm

Re: Another Webcam Demo Program

Post by chris319 »

Here are some links for escapi. I'm using the latest version, 3.0.

http://sol.gfxile.net/escapi/

https://github.com/jarikomppa/escapi

https://github.com/jarikomppa/escapi/tr ... /purebasic

Kudos to Jari for supporting pb.

There are several posts regarding pb and escapi on this forum.

escapi.pbi can be found here, but it needs to be renamed with the .pbi extension. https://github.com/jarikomppa/escapi/bl ... /escapi.pb

Here are the contents of escapi.pbi:

Code: Select all

;/* Extremely Simple Capture API */

Structure SimpleCapParams
  *mTargetBuf ; Must be at least mWidth * mHeight * SizeOf(int) of size!
  mWidth.l
  mHeight.l
EndStructure

;/* Return the number of capture devices found */
PrototypeC countCaptureDevicesProc()

; /* initCapture tries To open the video capture device.
;  * Returns 0 on failure, 1 on success.
;  * Note: Capture parameter values must Not change While capture device
;  *       is in use (i.e. between initCapture And deinitCapture).
;  *       Do *Not* free the target buffer, or change its pointer!
;  */
PrototypeC initCaptureProc(deviceno, *aParams.SimpleCapParams)

;/* deinitCapture closes the video capture device. */
PrototypeC deinitCaptureProc(deviceno)

;/* doCapture requests video frame To be captured. */
PrototypeC doCaptureProc(deviceno)

;/* isCaptureDone returns 1 when the requested frame has been captured.*/
PrototypeC isCaptureDoneProc(deviceno)

;/* Get the user-friendly name of a capture device. */
PrototypeC getCaptureDeviceNameProc(deviceno, *namebuffer, bufferlength)

;/* Returns the ESCAPI DLL version. 0x200 For 2.0 */
PrototypeC ESCAPIDLLVersionProc()

; marked as "internal" in the example
PrototypeC initCOMProc()

Global countCaptureDevices.countCaptureDevicesProc
Global initCapture.initCaptureProc
Global deinitCapture.deinitCaptureProc
Global doCapture.doCaptureProc
Global isCaptureDone.isCaptureDoneProc
Global getCaptureDeviceName.getCaptureDeviceNameProc
Global ESCAPIDLLVersion.ESCAPIDLLVersionProc

Procedure setupESCAPI()
  
  ; load library
  capdll = OpenLibrary(#PB_Any, "escapi.dll")
  If capdll = 0
    ProcedureReturn 0
  EndIf
  
  ;/* Fetch function entry points */
  countCaptureDevices = GetFunction(capdll, "countCaptureDevices")
  initCapture = GetFunction(capdll, "initCapture")
  deinitCapture = GetFunction(capdll, "deinitCapture")
  doCapture = GetFunction(capdll, "doCapture")
  isCaptureDone = GetFunction(capdll, "isCaptureDone")
  initCOM.initCOMProc = GetFunction(capdll, "initCOM")
  getCaptureDeviceName = GetFunction(capdll, "getCaptureDeviceName")
  ESCAPIDLLVersion = GetFunction(capdll, "ESCAPIDLLVersion")
  
  If countCaptureDevices = 0 Or initCapture = 0 Or deinitCapture = 0 Or doCapture = 0 Or isCaptureDone = 0 Or initCOM = 0 Or getCaptureDeviceName = 0 Or ESCAPIDLLVersion = 0
    ProcedureReturn 0
  EndIf
  
  ;/* Verify DLL version */
  If ESCAPIDLLVersion() < $200
    ProcedureReturn 0
  EndIf
  
  ;/* Initialize COM.. */
  initCOM();
  
  ; returns number of devices found
  ProcedureReturn countCaptureDevices()
EndProcedure
Last edited by chris319 on Tue Aug 29, 2017 8:15 pm, edited 1 time in total.
chris319
Enthusiast
Enthusiast
Posts: 782
Joined: Mon Oct 24, 2005 1:05 pm

Re: Another Webcam Demo Program

Post by chris319 »

walbus wrote:I self think, your primary main loop construction is the problem, never this can works good

And the world is full older machines, we should try making software for all
Not all users have money for new machines $
Also i see not a problem for a frames/s output
See the other thread in Coding Questions in which you have participated. The bottleneck is the webcam itself. Even the OpenCV code slows down when it has to draw a 1280 x 720 bitmap. It slows down in my program even if you skip all of my processing code.

If a 30 or even 60 fps frame rate is desired, the hardware has to keep up. That includes the webcam and the host computer. Again, the frame rate is dictated by an external device.
chris319
Enthusiast
Enthusiast
Posts: 782
Joined: Mon Oct 24, 2005 1:05 pm

Re: Another Webcam Demo Program

Post by chris319 »

Here is a version of the program with a smaller 640 x 360 image and incorporating Walbus's suggestion for not calling StartDrawing() repeatedly. In addition, in creating the lookup table, any pixel value greater than 254 is forced to 254, and any pixel value less than 1 is forced to 1. In professional video, the values of 0 and 255 are reserved for sync. This code can be eliminated if desired.

Code: Select all

;WORKS WITH ESCAPI
;UPDATED 8/28/2017
;PRESS CTL-Q TO QUIT
;INCORPORATES PROC AMP

ExamineDesktops()

gamma.f = 1.34 ;WEBCAM OUTPUT IS ALREADY GAMMA CORRECTED
#PEDESTAL = 16
gain.f = (235-16)/235
gain * 0.97

factor.f = 255/Pow(255,gamma)

;SET WEBCAM TO DEFAULTS AND SHARPNESS TO 255

;MAKE LUTS
Dim gammaTable.f(256)
For cnt = 0 To 255
cnf.f = cnt
gammaTable.f(cnt)=cnt;(Pow(cnf,gamma) * factor * gain) + #PEDESTAL
If gammaTable(cnt) > 254: gammaTable(cnt) = 254
ElseIf gammaTable(cnt) < 1: gammaTable(cnt) = 1
EndIf
Next

Dim x3table.l(1921)
For cnt = 0 To 1920
x3table(cnt) = cnt * 3
Next

LoadFont(1,"Arial",24)

IncludeFile "escapi.pbi"
Global WIDTH = 640;DesktopWidth(0)
Global HEIGHT = 360;DesktopHeight(0)
Global WIDTHM1 = WIDTH - 1
Global HEIGHTM1 = HEIGHT - 1
Global pixCount = (WIDTH * HEIGHT) - 2

Global Dim pixcolor.l(WIDTH, HEIGHT): Global Dim unsmoothedY.d(WIDTH, HEIGHT)
Global Dim Cr.d(WIDTH, HEIGHT): Global Dim Y.d(WIDTH, HEIGHT): Global Dim Cb.d(WIDTH, HEIGHT)

Global imHeight, imWidth, xCoord, yCoord,Rd,Gd,Bd

#DEVICE = 0    
If setupESCAPI() = #Null
      MessageRequester("Error", "Unable to initialize ESCAPI.")
    End
EndIf

    bufSize = WIDTH * HEIGHT * 4
    scp.SimpleCapParams
    scp\mWidth = WIDTH
    scp\mHeight = HEIGHT
    scp\mTargetBuf = AllocateMemory(bufSize)
    *buf = scp\mTargetBuf

    If initCapture(#DEVICE, @scp)
     
image = CreateImage(1, WIDTH, HEIGHT, 24)
OpenWindow(1, 0, 0, WIDTH, HEIGHT,"",#PB_Window_BorderLess)

AddKeyboardShortcut(1, #PB_Shortcut_Control|#PB_Shortcut_Q, 113);CTL Q TO QUIT
ImageGadget(0, 0, 0, WIDTH, HEIGHT, ImageID(1))
Quit = #False

StartDrawing(ImageOutput(1))
*writeBuffer = DrawingBuffer()
pitch = DrawingBufferPitch()
StopDrawing()

StartDrawing(WindowOutput(1))
;StartDrawing(ImageOutput(1))
DrawingFont(FontID(1))

Repeat
If WindowEvent() = #PB_Event_Menu ;KEYBOARD INPUT
If EventMenu() = 113
  Quit = #True
EndIf
EndIf       
     
doCapture(#DEVICE)

Repeat: Delay(1):Until isCaptureDone(#DEVICE) <> #False

;If isCaptureDone(#DEVICE) <> #False
;If isCaptureDone(#DEVICE) = #False

;PIXEL-BY-PIXEL READING AND WRITING
hm1 = *writebuffer + (HEIGHTM1 * pitch)
*bufoff = *buf

;Goto skip
For y = 0 To HEIGHTM1
For x = 0 To WIDTHM1
x3 = hm1 + x3table(x)

p1.l = PeekL(*bufoff)

PokeA(x3,gammaTable(p1 & 255))
PokeA(x3+1,gammaTable(p1 >> 8 & 255))
PokeA(x3+2,gammaTable(p1 >> 16))

*bufoff + 4
Next
hm1 - pitch
Next

skip:
SetGadgetState(0, ImageID(1))

;StartDrawing(WindowOutput(1))
;DrawingFont(FontID(1))

now.f = ElapsedMilliseconds()
fps.f = now.f-then.f

If fps > 0:fps$ = StrF((1/fps)*1000,2) ;:EndIf
If (1/fps)*1000 < 10: fps$ = "0" + fps$: EndIf
EndIf

DrawText(100, 200, fps$+" fps",#White)
then.f = ElapsedMilliseconds()
;StopDrawing()

Until Quit = #True
StopDrawing()
deinitCapture(#DEVICE)
FreeImage(1)
FreeMemory(scp\mTargetBuf)
CloseWindow(1)

    Else
      Debug "Init capture failed."
    EndIf

    End
User avatar
djes
Addict
Addict
Posts: 1806
Joined: Sat Feb 19, 2005 2:46 pm
Location: Pas-de-Calais, France

Re: Another Webcam Demo Program

Post by djes »

I've tried a simple copymemory(), as I don't understand why every pixel should be converted, if the image given by ESCAPI is already in 32 bits. As the buffer is upside down, I've used vectordrawing lib to reverse the whole thing. I've also used your fps() counter.

Code: Select all

XIncludeFile "escapi.pbi"

device = 0

count = setupESCAPI()
Debug "init: " + Str(count)

If count = 0
  End
EndIf

*name = AllocateMemory(1000)
getCaptureDeviceName(device, *name, 1000)
name$ = PeekS(*name, -1, #PB_Ascii)
Debug "name: " + name$
FreeMemory(*name)

scp.SimpleCapParams
scp\mWidth = 640
scp\mHeight = 360
scp\mTargetBuf = AllocateMemory (scp\mWidth * scp\mHeight * 4)

If initCapture(device, @scp)
  Debug "cap init successful" 
  
  gadgetimage = CreateImage(#PB_Any, scp\mWidth, scp\mHeight)
  webcamimage = CreateImage(#PB_Any, scp\mWidth, scp\mHeight, 32)
  CopyImage(gadgetimage, renderedimage)
  
  OpenWindow(0, 0, 0, scp\mWidth, scp\mHeight, name$, #PB_Window_ScreenCentered|#PB_Window_SystemMenu)
  ImageGadget(0, 0, 0, scp\mWidth, scp\mHeight, ImageID(gadgetimage))
  LoadFont(0, "Impact", 20, #PB_Font_Bold)
  
  Quit = 0
  Repeat
    
    doCapture(device)
    While isCaptureDone(device) = 0
      If WaitWindowEvent(1) = #PB_Event_CloseWindow
        Quit = 1
        Break
      EndIf       
    Wend
    
    If StartDrawing(ImageOutput(webcamimage))
      Buffer      = DrawingBuffer()             ; Get the start address of the screen buffer
      Pitch       = DrawingBufferPitch()        ; Get the length (in byte) took by one horizontal line
      PixelFormat = DrawingBufferPixelFormat()  ; Get the pixel format. 
      CopyMemory(scp\mTargetBuf, Buffer, scp\mWidth * scp\mHeight * 4)
      StopDrawing()
    EndIf
    
    If StartDrawing(ImageOutput(renderedimage))
      DrawImage(ImageID(webcamimage), 0, 0)
      StopDrawing()
    EndIf
    
    now = ElapsedMilliseconds()
    fps.f = now - then
    If fps > 0 : fps$ = StrF((1/fps)*1000, 2) ;:EndIf
      If (1 / fps) * 1000 < 10: fps$ = "0" + fps$: EndIf
    EndIf  
    then = ElapsedMilliseconds()
     
    If StartVectorDrawing(ImageVectorOutput(gadgetimage, #PB_Unit_Pixel))
      VectorSourceColor(RGBA(0, 0, 0, 255))
      FillVectorOutput()
      RotateCoordinates(scp\mWidth/2, scp\mHeight/2, now / 10)
      MovePathCursor(0, 0)      
      DrawVectorImage(ImageID(renderedimage), 128)
      ResetCoordinates()
      MovePathCursor(10, 10)      
      VectorFont(FontID(0), 25)
      VectorSourceColor(RGBA(255, 255, 0, 255))
      DrawVectorText(fps$ + " fps")
      StopVectorDrawing()
    EndIf
    SetGadgetState(0, ImageID(gadgetimage))
    
  Until Quit
  
  deinitCapture(device)
Else
  Debug "init capture failed!"
EndIf

End
walbus
Addict
Addict
Posts: 929
Joined: Sat Mar 02, 2013 9:17 am

Re: Another Webcam Demo Program

Post by walbus »

@djes
Unfortunately, your rotating is also pixel based, primary your method is ever a slower solution as a simple copy method from a to b
User avatar
djes
Addict
Addict
Posts: 1806
Joined: Sat Feb 19, 2005 2:46 pm
Location: Pas-de-Calais, France

Re: Another Webcam Demo Program

Post by djes »

walbus wrote:@djes
Unfortunately, your rotating is also pixel based, primary your method is ever a slower solution as a simple copy method from a to b
I just showed that a pixel based loop is not necessary, as the frame buffer is directly available in a compatible format (see the C code). A direct copy is faster, as long as you don't have to modify the picture. The rotation was only for fun.
There's also in escapi an example using opengl, that should be the faster method to achieve a large frame update at full speed.
AAT
Enthusiast
Enthusiast
Posts: 256
Joined: Sun Jun 15, 2008 3:13 am
Location: Russia

Re: Another Webcam Demo Program

Post by AAT »

chris319 wrote:... The bottleneck is the webcam itself. Even the OpenCV code slows down when it has to draw a 1280 x 720 bitmap. It slows down in my program even if you skip all of my processing code.
Perhaps, this information will be useful to you.
https://translate.google.com/translate? ... edit-text=
chris319
Enthusiast
Enthusiast
Posts: 782
Joined: Mon Oct 24, 2005 1:05 pm

Re: Another Webcam Demo Program

Post by chris319 »

Hi AAT -

Thank you for posting that. It is very interesting.
Working with the camera in Windows is implemented in cap_dshow.cpp. There in
VideoInput :: videoInput ()
The camera modes in the mediaSubtypes array are defined

Here are the standard modes

MediaSubtypes [0] = MEDIASUBTYPE_RGB24;
MediaSubtypes [1] = MEDIASUBTYPE_RGB32;
MediaSubtypes [2] = MEDIASUBTYPE_RGB555;
MediaSubtypes [3] = MEDIASUBTYPE_RGB565;
MediaSubtypes [4] = MEDIASUBTYPE_YUY2;
MediaSubtypes [5] = MEDIASUBTYPE_YVYU;
MediaSubtypes [6] = MEDIASUBTYPE_YUYV;
MediaSubtypes [7] = MEDIASUBTYPE_IYUV;
MediaSubtypes [8] = MEDIASUBTYPE_UYVY;
MediaSubtypes [9] = MEDIASUBTYPE_YV12;
MediaSubtypes [10] = MEDIASUBTYPE_YVU9;
MediaSubtypes [11] = MEDIASUBTYPE_Y411;
MediaSubtypes [12] = MEDIASUBTYPE_Y41P;
MediaSubtypes [13] = MEDIASUBTYPE_Y211;
MediaSubtypes [14] = MEDIASUBTYPE_AYUV;
MediaSubtypes [15] = MEDIASUBTYPE_MJPG; // MGB
all you had to do was raise in the array MEDIASUBTYPE_MJPG over MEDIASUBTYPE_YUY2.
After that, after compiling the library, the normal result was 30 FPS for all cameras and now they worked
What do you make of that, AAT? Recompile cap_dshow.cpp?

So MediaSubtypes [4] should be MJPG and move the others down one slot?

I have passed this information along to the developer of escapi.
AAT
Enthusiast
Enthusiast
Posts: 256
Joined: Sun Jun 15, 2008 3:13 am
Location: Russia

Re: Another Webcam Demo Program

Post by AAT »

I did not do anything, it's just a link. :)

I think you need to put first the value of MediaSubtypes, which gives the maximum value of FPS for you. Then, recompile the entire package of OpenCV. In my opinion, this will be easier than recompiling only one DLL from the package.
I have passed this information along to the developer of escapi.
If it becomes possible to specify the video format (mediasubtype) when the function is called, this will be the best solution.
chris319
Enthusiast
Enthusiast
Posts: 782
Joined: Mon Oct 24, 2005 1:05 pm

Re: Another Webcam Demo Program

Post by chris319 »

I've downloaded the OpenCV source code and have Visual Studio, but am looking for directions on how to compile it.
Post Reply