Invalid Form coordinates on Windows 10 tablet

Hi,
I have a program that's been running on Windows 7, 8 and 10 and I just implemented an on screen keyboard (OSK) feature for tablet use that's integrated with one of my dialogs because using the OSK that came with windows was too clumsy to use as I had to constantly keep moving it around in order to get to the fields that I was trying to fill out.

My OKS is integrated with a dialog that pops up after clicking a button on a parent dialog (so it's main window > dialog1 > dialog2 > OSK_Ctrl > button_label) and all works OK on my Windows 7 development machine (with or without a 30" monitor) and my Surface 3 (Windows 8) but NOT on my Surface 4 (Windows 10) or a Windows 10 laptop. On my Surface 4 (and the Windows 10 laptop), depending on the position of my OSK dialog on the screen, I have to click about 60mm above the actual button in order for the button to be identified correctly.

I then moved the dialog to the top & left edges (to remove any possible offsets) but that didn't help however if connect my Surface 4 to the 30" monitor than all is OK, i.e. the button is identified correctly, although connecting my Windows 10 laptop to a monitor didn't make any difference, the problem was still there.

To identify a button I basically compare the X & Y provided by the Hook event with each of the button coordinates and if they match it's considered a hit. Button coordinates are basically 'dialog1.left + dialog2.left + OSK_Ctrl.Left + button_label.left' and similar for the Top property.

I then decided to get the hook out of the equation and just printed a message of my dialog position (Form.Top & .Left properties) and I get a different result on my Surface 4 depending on whether it's connected to the 30" monitor or not.

I'm really at a loss as to what it could be and and would really appreciate some help.


Thanks,
Mike

Use a Spy++ tool to caprtue the window positions and see if it will shed any light on the issues you are running into. It should come with your installation of VB6.

Hi Arjay, sorry for not responding sooner. I have in fact been carrying out some tests utilizing the Spy++ tool (that I used in the past but forgot all about this time so thanks for the reminder) and I've also been trying to simplify the scenario but I've been running a bit in circles. A few times I thought I may be onto something but then it was nothing. I'm still trying to get some meaningful results and (unless it all works out) will come back soon.

You talk about buttons but also hooks and using x,y to identify the buttons which sounds like you are not using buttons at all so not really sure what you are doing. If you really are using buttons then is there a reason you are not using the click event? How are you positioning the dialog and buttons? Are you taking into account the scaling that may be done on different screens?

I wonder if the issue is differences between dialog units and screen units. Read more with a bing 9r google search for "dialog units vs screen units".

Thank you DataMiser and Arjay for your response. I tend to give too much info so I tried to keep it simple but looks like I didn't give enough.

I basically have a dialog/form with a frame and an ActiveX control (.ocx) within that frame. I am using labels for button to show the button character(s) but I can't use button events as that would take the focus away from an active control and give it to my keyboard button/label control whereas using the hook intercepts the mouse 'button down' event without taking the focus away from the active control.

Hook event gives me X & Y in pixels and I then need to determine if those correspond to any of my keyboard buttons. To do that I scan through all my keyboard buttons and convert the form+frame+button 'Left & Top properties that are in twips into pixels and do the comparison. All that works 100% on my Windows 7 machine (whether connected to my monitor or not) and my (Windows 8) Surface 3 tablet but not on my Surface 4 (Windows 10) tablet or my Windows 10 laptop.

On Surface 4 I have to click about 60mm above the button in order for the button coordinates to match the hook X & Y values but if I connect Surface 4 to a monitor then all works ok. Also if I connect my Windows 10 laptop to a monitor, unlike Surface 4 though, the problem is still there.

I've created a little app with a frame and the ActiveX keyboard and used Spy++ to get some readings and (for example) the Spy shows me the same coordinates for the form.Top & Left as I get from the form properties themselves when reading in code and outputting to a message box but Spy gives me slightly different coordinates for the Form>Frame.Top & Left than when adding 'Form.Top+Frame.Top' values and printing them using message box and I'm still ptrying to make sense out of that.

Sorry if it's all a bit confusing but it feels like that there is a setting in Windows 4 that may even exist in Windows 7 & 8 that is set to a different value on each of my machines and I get a different behavior depending on whether the monitor is attached or not.

/Mike

I forgot to say that once I determine which key the hook X & Y correspond to I execute SendKeys with appropriate ASCII code otherwise I effectively ignore the hook event and it gets handles by windows as per normal.

Could it be a DPI issue?

Thanks for the suggestion. I read up a fair bit on that earlier on and was in two minds as to whether it could be although I didn't have as much info as I have now and you might be right.

After a fair bit of testing (and getting past my antivirus apps that detected possible harmful virus activity on account of Spy++ and me using hooks) I got the following results:

Surface 4 screen; DOES NOT WORK
~~~~~~~~~~~~~~~~~~~
- Spy++ Rectangle: (11, 98) - (668, 235) This represents the ActiveX keyboard control rectangle/area, that is correct
- Hook: X=27, Y=199 Hook event values after I clicked in top left corner of the ActiveX keyboard control (digit 1)

Surface 4 connected to a large monitor; WORKS OK
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Spy++ Rectangle: (11, 98) - (668, 235) This represents the ActiveX keyboard control rectangle/area, that is correct (same as above)
- Hook: X=12, Y=98 Hook event values after after I connected an external monitor and clicked in top left corner of the ActiveX keyboard, same as above


Looking at the values above, with a monitor connected to the tablet the Hook event X & Y values are 12 & 98, and fall within the ActiveX keyboard control rectangle and identify the correct key.

When I then do nothing else other than simply connect an external monitor to the tablet and click in exactly the same spot as above the Hook event values are 27 & 199 and (in this case) instead of identifying the key in the top left corner of the keyboard rectangle it identified the key that is at the bottom left side of the keyboard, which that makes sense because the hook Y value (199) is just within the bottom part of the rectangle, being 90-235.

Looking at the above results and me doing absolutely nothing other than connecting a monitor to the tablet, I still feel that some Windows setting comes into play with the external monitor.

I can (just for now) deduct 11 & 98 from the Hook values and see if that works consistently but of course that is unlikely to solve the problem as not knowing what's causing it I have no idea how different tablets/windows settings will be and what offset will they have so this should be done properly.

I was going to take a look at SystemParametersInfo API to see if it shows any parameter that relate to the monitor, virtual displays, DPI or anything along those lines and of course any input or suggestions will be most welcome.

There are apis that allow you to determine if you have multiple monitors. Look in msdn (I'm on a phone having breakfast now, otherwise I'd do it for you).

I have not had my hands on a windows tablet as of yet so am pretty much in the dark as to any quirks they may have. I am thinking font size may have an effect. I have ran into some issues with Android devices where the end user had fonts set to a larger size. Different issues than you are seeing but issues anyway.

I would think that there is a good chance that the res changes when attached to a different monitor and I may be wrong but I am thinking that when the res changes then the scaling goes back to 100% even if it was set to 125% or higher on the previous screen. I assume that the version of Windows 10 on the tablet has the option for scaling above 100%. If so you may want to take a look and if not set at 100% try changing it and see what results you get.

Thanks for your earlier suggestion Arjay, I was typing you a response earlier today when I had to quickly go out and I didn't get a chance to respond till now so I'll comment on your suggestion first. I'm getting a mixed behavior in that one Windows 10 device (the tablet) doesn't work properly by itself but it does with an external monitor whereas the other Windows 10 device (the laptop) doesn't work properly with or without the external monitor so I'm not exactly sure how it would help me finding out whether a monitor is/isn't attached but I'll keep that in mind.

DataMiser I'm fairly sure that I checked that several days ago but I'm not sure any more so I just double checked it now and the scaling and font size didn't change when I connected the external monitor compared to when I didn't have the external monitor connected so I'm assuming that it's not playing an active role here.

I also did a bit more checking of my 'calibration' function and after I adjust the Hook X & Y values (by the difference found during the calibration process, i.e. being the difference between the raw Hook X & Y values and the absolute center point of the calibration button coordinates) I'm finding that if I click 2-3 keys either side of the 'calibration button center' all is OK but going further starts miss-identifying the clicked keys.

Hi guys, sorry for not getting back to this issue but as it's proving more difficult than I anticipated and I have to urgently complete something else, I'll post the solution once I get back to this.

If using a Surface Tablet Windows 10, consider checking the display scale (right click on desktop -> Display Settings)