Get item dimensions with AccessibleObjectFromPoint

[BarryG]

Hi guys, I'm trying to determine the dimensions (rectangle) of the item under the mouse using AccessibleObjectFromPoint. There's info about it here -> https://docs.microsoft.com/en-us/window ... cclocation but I don't know how to fit that into the following code template. Can somebody please help if they know? Thank you.

Note: The goal of this is to get the dimensions of Google's "Search" box, which can't normally be obtained with GetWindowRect_() as it's not a Windows item. I've read online that AccessibleObjectFromPoint can probably do it, hence this request for help. It should be possible because I can get the text in the Google "Search" box with AccessibleObjectFromPoint, but not the dimensions of the box.

Code: Select all

CoInitialize_(0)
Prototype.l ProtoAccessibleObjectFromPoint(x.l,y.l,*ia,*var)
Global AccessibleObjectFromPoint.ProtoAccessibleObjectFromPoint
AccessibleObjectFromPoint=GetFunction(OpenLibrary(#PB_Any,"Oleacc.dll"),"AccessibleObjectFromPoint")

Procedure GetItemDimensions()
  
  Protected CursorPos.Point
  Protected handle
  Protected *pIAcc.IAccessible
  Protected vt.VARIANT
  Protected s.s
  
  GetCursorPos_(@CursorPos)
  
  If AccessibleObjectFromPoint(CursorPos\x,CursorPos\y,@*pIAcc,@vt)=#S_OK
    ; Get the item dimensions here, somehow... :(
    
  EndIf
  
  *pIAcc\Release()
  
  ProcedureReturn #Null
  
EndProcedure

Sleep_(2000) ; Give user time to hover over the "Search" box.
GetItemDimensions()

[IdeasVacuum]

Hi Barry

Not sure it can do it because the Google Search Box is part of a web page (HTML) and does not have to comply with anything Microsoft.

You might be able to download the HTML and extract the box dimensions, but the height auto-expands to display recent searches.

[BarryG]

There'd have to be a way to do it, because PhraseExpress can pop up an auto-complete box in any web browser field, such as Google's "Search" field; so somehow they know where the caret is. I'm trying to do it too, but GetCaretPos_() doesn't work with web browsers. I'm asking in relation to this post -> viewtopic.php?f=13&t=72738

[IdeasVacuum]

That's different Barry

You probably don't need the box size (or fake box size) or even the caret position, you probably do need to know the id of the box and "inject" your text into the box - it's javascript territory (sorry, I'm not a javascript person).

You might find something useful on the Spider Basic Forum, someone like Peter or Falsam that can help. I can link to something about injection - it's used by Android Apps to fill in Web form boxes:

https://puravidaapps.com/inject.php

[BarryG]

I don't think it's an injection thing because PhraseExpress doesn't do anything like that and it works.

[IdeasVacuum]

How do you know how PhraseExpress does what it does?

It's said that Clive Sinclair, the man who developed the prototype of the Amstrad PC, once tried to explain the computer's technical specifications to Alan Sugar. 'I don't care if they have rubber bands in them,' replied Sugar, the Amstrad boss, 'as long as they work.'

By the way, it has nothing to do with sharp needles. Like a lot of javascript terms, it's just a different, more wacky name for something.

[BarryG]

How do you know how PhraseExpress does what it does?

Obviously I don't know for sure, but do you really think it's parsing the loaded web page source and getting the element info?

[IdeasVacuum]

... yes I do, if it's done on a smartphone in milliseconds, it can be done on a desktop.

[BarryG]

I may have to investigate that route, then. But can it get the web elements from an external browser? I'm not talking about using a PureBasic WebGadget(), because that link you gave (https://puravidaapps.com/inject.php) is injecting code into its own loaded webview into its own app, which of course is easy. I'm talking about a PureBasic exe reading Firefox.exe's web elements of any given page; a totally independent and separate process.

Wouldn't it be a whole lot easier just to use AccessibleObjectFromPoint to get the current focussed item, since Windows already knows what that is for everything on-screen (both apps and web-based items)? Like how the Windows Narrator knows.

[IdeasVacuum]

Wouldn't it be a whole lot easier just to use AccessibleObjectFromPoint to get the current focussed item, since Windows already knows what that is for everything on-screen (both apps and web-based items)? Like how the Windows Narrator knows.

As far as I can tell from the MS link, it does not know everything that is on-screen, it only knows the things that have been built to be compatible, which websites built with C# etc probably are but most things Google definitely not.