Drawing with OpenGL in an OpenCV window

Hi everyone!!

First of all, sorry if my english is not perfect, I am not a native.

I've been working with OpenCV to capture images from a webcam and using its functions to proccess this images. I needed to detect a marker for which I used ARToolKit, but just for that, detecting the marker. Everything is fine until here.

Now, I need to draw a kind of blackboard according to the marker, for which OpenGL is perfect for perspective transformation and so on. My problem is that I can't achieve that. I don't know how to say to OpenGL "you guy, draw that blackboard within this opencv window"

Is it impossible what I want to do? I am desperate... any help would be great. Thanks in advance :)

By reading the Red Book I couldn't understand what glDrawPixels and glReadPixels were for.

But now I know they should help with my problem, don't they?

I have this no-working code, where dataPtr is the array that contains the pixel data to be drawn, an Image captured from a webcam:

Code:

---

glDrawPixels(320,240,GL_RGBA,GL_UNSIGNED_BYTE,dataPtr);

glClearColor (0.0, 0.0, 0.0, 0.0);
glClear (GL_COLOR_BUFFER_BIT);
glColor3f (1.0, 1.0, 1.0);
glOrtho(0.0, 1.0, 0.0, 1.0, -1.0, 1.0);
glBegin(GL_POLYGON);
        glVertex3f (0.25, 0.25, 0.0);
        glVertex3f (0.75, 0.25, 0.0);
        glVertex3f (0.75, 0.75, 0.0);
        glVertex3f (0.25, 0.75, 0.0);
glEnd();
glFlush();

glReadPixels(0,0,320,240,GL_RGBA,GL_UNSIGNED_BYTE,dataPtr);

---

When I display the image is exactly the same as before running this code.

What's wrong? What's missing?

Thanks in advance.

This might help you:

http://www.robthebloke.org/opengl_ex...screengrab.zip

Thanks a lot, but that's not what I am looking for, but you seem to know a lot about OpenGL.

See, I wonder if it's possible with glDrawPixels to draw on an image given by that function, and then returning it to another image with glReadPixels without displaying anything with GLUT. 'Cos, to display with GLUT I have to enter the "mainloop", and I don't want because I already have a loop of my own, does it make any sense?

you cant draw on an image you draw with objects on objects..
u can draw on image by manipulating the image's data array before output on opengl

As Mitsukai said you can manipulate the pixels array of the image. If you do not want to see the image you can draw it at XY(-10000,-10000) and then read the pixels since glReadPixels need pixels drawn in the buffer. I do not understand what you need, maybe to read pixels without drawing them? That's impossible.

Thank you both for your replies :) This subject is driving me crazy, because I don't know what can and can't be done with OpenGL. I'd need you to answer a few questions, please.

As I said, I am using OpenCV to capture and display images from a webcam. I want OpenGL to draw something in my image and display it with OpenCV.

1. is glDrawPixels the appropiate function to draw with openGL on a non-blank canvas??

2. If it's not, how can you draw on an existing image??

3.Can OpenGL "ReadPixels" from an OpenCV window?

4. You already told me you can't "ReadPixels" without drawing them, fine, is it there a way to display OpenGL drawings without using GLUT and its mainloop??

In short, I need OpenGL to draw an object on an image captured with openCV and only displaying the result and I don't know how to do it.

I am sorry, it's just so difficult for me to make myself clear talking about things I don't fully understand in another language. I appreciate your help and patience.

If you want to draw something in the image you should use PhotoShop, OGL can't interact with other windows that aren't OGL ones, or if the application uses OGL for drawing maybe you can write a PlugIn, it depends on the application. It's the same thing that you want to draw with OpenGL in the PowerDVD window or you want to read pixels from Firefox with OGL, it make no sense. glReadPixels can be used in a OGL window, where you have setup OpenGL. If OpenCV uses OpenGL to draw images on the window and if is it possible, you can write a PlugIn to access to the buffer and use glReadPixels, but only if OGL is enabled on that window and if the API used is OpenGL and if is it possible to write PlugIn or is it possible to play with OGL in that window etc...

Quote:

---

If you want to draw something in the image you should use PhotoShop, OGL can't interact with other windows that aren't OGL ones, or if the application uses OGL for drawing maybe you can write a PlugIn, it depends on the application.

---

Oh, come on! This can't be that difficult. If I wanted to draw I would use PhotoShop, but I want MY application to draw by itself, not ME.

So, fine, OpenGL cannot draw in a non-openGL window. I think it should. If you had more than one openGL window, how would you select one or another to draw on it? Can't you use the same procedure to draw on an OpenCV window?

That's OK, I'll have to give up on this issue...

I don't know how OCV works, I have never used it. Of course you can have multiple OpenGL windows. OGL link a rendering context to each window's device context, thus the OGL calls will redirect to that window:

MakeCurrent(RC1, DC1)

//here OGL calls will draw on window 1

MakeCurrent(RC2, DC2)

//here OGL calls will draw on window 2

Maybe it's better you have a glance at OGL and how it works before start to write code.

Hi there! :)

I finally made it, I have the OpenCV image in an OpenGL window. That's not what I wanted, but that's the best I can get. I didn't use GLUT, as GLUT has its own mainloop, so I used freeGLUT instead. Here is the code.
Initializing function:

Code:

---

int init(void)
{
//general initialization:
        char cadena[25]="GestionMarcaAR\0";
        char **argv;
        int argc = 1;

        argv=(char **)malloc(sizeof(char *));
        *argv=(char *)malloc(sizeof(char)*10);
        strcpy(*argv, cadena);

        glutInit(&argc,argv); //we cheat it ;P

        glutInitDisplayMode(GLUT_DOUBLE | GLUT_RGB | GLUT_DEPTH);
        glutInitWindowPosition(800, 400);
        glutInitWindowSize(320,240);
        glutCreateWindow("Name");

        glutDisplayFunc(draw);
        glutIdleFunc(draw);

//initializating the texture mapping
        GLuint mTexture;
        glGenTextures(1, &mTexture);
        glBindTexture(GL_TEXTURE_2D, mTexture);
        glEnable(GL_TEXTURE_2D);
        glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP);       
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP);

       
        glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_DECAL);

        glClearColor(1.0f, 1.0f, 1.0f, 1.0f); //color blanco de fondo
        glColor3f(0.0f,0.0f,0.0f);

        return 0;
}

---

Drawing function:

Code:

---

void draw()
{
        glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB,320,240,0,GL_BGR,GL_UNSIGNED_BYTE, imagen->imageData);
       
        glBegin(GL_POLYGON);
                glTexCoord2f(0.0f, 0.0f); glVertex2f(-1.0f, -1.0f);
                glTexCoord2f(1.0f, 0.0f); glVertex2f( 1.0f, -1.0f);
                glTexCoord2f(1.0f, 1.0f); glVertex2f( 1.0f,  1.0f);
                glTexCoord2f(0.0f, 1.0f); glVertex2f(-1.0f,  1.0f);
        glEnd();
       
        glFlush();
       
        glutSwapBuffers();
}

---

And what is needed to be anywhere in the mainloop of openCV:

Code:

---

glutPostRedisplay();  //to force "draw" function to be called
glutMainLoopEvent();  //freeglut function

---

What's imagen->imageData? If it's always the same texture you must call it within the initialization function and not every frame.

It is the image I am getting from the webcam. I want it to change with every frame.

I am using the glTexImage2D function cos I've been told that its faster than using glDrawPixels.

;D

bettynet

I need to do the same... do you can send me a code files for this example? Please.

thanks.

Hi bettyNet,
Im having the same difficulties you had and i think you may help me.
Im grabbing a frame from cam using OpenCV and send it to ARToolkitPLUS.
now, after i recognize my marker i wanna draw on it using OpenGL.
now, about The code you wrote... I have a main loop that grabs a frame sends it to ARTP and than should draw on it.
where should i put all those functions, and how do i use the freeglut?
Thanks in advance.
eyalalo.

Dear Bettynet,

Thank YOU for posting your solution to this forum. Like many others, I foud your code excerpt very useful to understand your solution. THANK YOU!

Like other posters (such as Danielmerkel, and Eyalalo) I am also getting started with OpenCV, ARToolkitPLUS along with OpenGL. (I am proficient in OpenGL programming)

Can you recommend me (and to many others) from where to start in order to implement marker detection on live video and create OpenGL overlays between markers?

Can you also give me (and to many others) example codes or your code excerpt showing
* OpenCV and ARTToolkitPLUS initialisation
* OpenGL window creation
* Live video acquisition via OpenCV
* Marker detection
* Displaying live video with OpenGL overlay between detected markers (Your previous posting shows how to display live video as texture in an OpenGL window. However it does not show you how you draw between markers.)

Thank you for being open and sharing your experience.

Miem Chan

miemchan @ gmail . com

hi,
@Bettynet: please send me your code example. I am stuck with the same problem as you were

do you need my email id??

thanks in advance