[RESOLVED] Processing Time For PVOID Into Byte Array Conversion

Hi,

In my project I'm using PVOID pointer to store real time data.

After getting this data I will convert the data into byte array, like below

Code:

---

byte *bPoint = NULL;
PVOID pvData;
byte TempArr[1024];
bPoint = (byte*) pvData;
for(int i=0;i<1024;i++)
    TempArr[i] = (byte) (*bPoint + i);

---

Processing time for the above code takes 9500 to 9900 microseconds (Used QueryPerformanceCounter).

Code:

---

TempArr[0] = ((BYTE*) pvData) [0];

---

This code takes 1100 to 1200 microseconds.

My doubt is, The processing time of PVOID data into byte array conversion takes time like above?

Or any other easy way(PVOID data into byte array conversion) to reduce the processing time?

Please clear me.

Quote:

---

Originally Posted by saraswathisrinath

Hi,

In my project I'm using PVOID pointer to store real time data.

After getting this data I will convert the data into byte array, like below

---

Why aren't you using memcpy()?

Regards,

Paul McKenzie

Quote:

---

Originally Posted by saraswathisrinath

After getting this data I will convert the data into byte array, like below

---

Also, you are not converting anything. What you're doing is copying from one memory block to another. Mass copying is done efficiently using memcpy(), and not by writing a for-loop.

On the other hand, converting implies you're using some sort of algorithm or function to transform from one type to another, and you're not doing that here.

Regards,

Paul McKenzie

Quote:

---

Originally Posted by saraswathisrinath

Processing time for the above code takes[B] 9500 to 9900 microseconds (Used QueryPerformanceCounter).

---

9500 microseconds is 9.5 milliseconds, or 0.0095seconds... copying 1024 bytes does not take that long, not even on an original IBM XT would it take that long.

Note that QueryPerformanceCounter does NOT return microseconds.
What it returns is a computer dependant value that indicates a discrete time interval for that specific computer. You need to use QueryPerformanceFrequency() to convert the results from QueryPerformanceCounter() into actual second-based values.

Quote:

---

Originally Posted by Paul McKenzie

Mass copying is done efficiently using memcpy(), and not by writing a for-loop.

---

Not necessarily. A smart compiler optimizer will see the sequence and convert it to a memcpy on it's own, or potentially even tailor a specific optimal bit of copying code. Don't assume the compiler will simply make a loop and copy bytes one by one.

Then again, don't assume the compiler WILL make this optimisation either ;-)

Quote:

---

Or any other easy way(PVOID data into byte array conversion) to reduce the processing time?

---

First, your code has a problem

Code:

---

TempArr[i] = (byte) (*bPoint + i);

---

should be

Code:

---

TempArr[i] = (byte) *(bPoint + i);

---

to correctly index memory pointed to by bPoint

Using an array index within the loop is the most expensive way of performing the copy. On my computer using VS fully optimised for speed, this gives an average of 4140 (using QueryPerformanceCounter)

Code:

---

PVOID pvData = (PVOID)data;
byte *bPoint = (byte*) pvData;

byte TempArr[1024];
byte *bTo = TempArr;

        for(int i = 0; i < 1024; i++)
                TempArr[i] = (byte) *(bPoint + i);

---

Just changing the assignment to use pointers rather than index reduces the value to average 880

Code:

---

PVOID pvData = (PVOID)data;
byte *bPoint = (byte*) pvData;

byte TempArr[1024];
byte *bTo = TempArr;

        for(int i = 0; i < 1024; i++)
                *bTo++ = *bPoint++;

---

And using memcpy reduces this further to average 850

Code:

---

PVOID pvData = (PVOID)data;
byte *bPoint = (byte*) pvData;

byte TempArr[1024];
byte *bTo = TempArr;

        memcpy(bTo, bPoint, 1024);

---

Quote:

---

Originally Posted by Paul McKenzie

Why aren't you using memcpy()?

---

I have PVOID data only. But I like to draw real time image using byte array. So I convert this into byte array. I get that method from net and I used. After that only I noticed, my program getting delay for this data conversion.

Quote:

---

Originally Posted by Paul McKenzie

On the other hand, converting implies you're using some sort of algorithm or function to transform from one type to another, and you're not doing that here.

---

I never used before any algorithm for data conversion. Please give any reference, after that I will try in my project.

Quote:

---

Originally Posted by OReubens

You need to use QueryPerformanceFrequency() to convert the results from QueryPerformanceCounter() into actual second-based values.

---

I'm processing in the speed of Milliseconds. If I return second based means it must return 0 only.

Code:

---

LARGE_INTEGER m_lnStart;
    LARGE_INTEGER m_lnEnd;
    LARGE_INTEGER m_lnFreq;

    QueryPerformanceFrequency(&m_lnFreq);//Retrieves the frequency of the high-resolution performance counter
    QueryPerformanceCounter( &m_lnStart);//Retrieves the current value of the high-resolution performance counter
 
        byte *bPoint = (byte*)pvData;

        for(int i = 0; i < 1024; i++)
                TempArr[i] = (byte)(*(bPoint + i));

          QueryPerformanceCounter( &m_lnEnd );

        __int64 nDiff(0) ;
    double dMircoSecond( 0 ) ;
    if( m_lnStart.QuadPart !=0 && m_lnEnd.QuadPart != 0)
    {
        dMircoSecond = (double)(m_lnFreq.QuadPart/1000000);
        nDiff = m_lnEnd.QuadPart-m_lnStart.QuadPart ;
        dMircoSecond = nDiff/dMircoSecond;
                CString str;
                str.Format("%d", nDiff);
                GetDlgItem(IDC_TIMEDIF)->SetWindowText(str);               
    }

---

Quote:

---

Originally Posted by 2kaud

First, your code has a problem

---

Sorry Mr.2kaud, that's my mistake. But I'm correctly used in my project.

Quote:

---

Originally Posted by 2kaud

And using memcpy reduces this further to average 850

---

850 microseconds or Milliseconds?

My requirement is, I'm getting PVOID data continuously, using that i convert into 2D byte array & draw an image.Important note is these process are done by minimum 1 millisecond. If process time exceed then i miss some data in real time.

Quote:

---

Originally Posted by saraswathisrinath

I have PVOID data only.

---

There is no such thing as "PVOID data".

You have a pointer, and starting at that pointer is data in one contiguous block. How you interpret that contiguous block could be anything -- it could be characters, floating point, integers, etc. But just because the data is interpreted differently doesn't change the data. The data at that memory location stays exactly the same.

That's why I'm stating to you that you are not converting anything. You're interpreting the data differently, but you are not converting. You're simply copying the same data from one memory block to another memory block, and then at the destination block, you're interpreting that data as pointing to a series of "bytes".

An example of a real conversion would be going from ANSI to Unicode, or from binary to hexadecimal, or going from little-endian to big-endian where you need to actually change the data by swapping bytes, etc.

Regards,

Paul McKenzie

Quote:

---

Originally Posted by saraswathisrinath

I never used before any algorithm for data conversion. Please give any reference, after that I will try in my project.

---

I'm not saying to use an algorithm. What I'm saying is that what you posted is not conversion. It is copying data from one memory location to another. I don't know how I can put it more simply.

So your goal is to copy contiguous data from one memory location to another -- that is what everyone responding to you is giving you advice on. The fastest way is to write the copying in assembly language using the fastest mass-copying assembly instructions for your CPU. You can't get any faster than that, regardless of your requirements.

Regards,

Paul McKenzie

Is the only reason you're copying it because you think you're converting it? If so, all you need to do is change your pointer type.

Quote:

---

850 microseconds or Milliseconds?

---

The numbers are the difference between two calls to QueryPerformanceCounter. For the sake of the comparisons I did it doesn't really matter what are the units of measurement - just the relative numbers - which as expected shows that using memcpy is the fastest and array indexing the slowest.

Quote:

---

Originally Posted by saraswathisrinath

Code:

---

        dMircoSecond = (double)(m_lnFreq.QuadPart/1000000);

---

---

This is wrong.
If you have to go this somewhat awkward way about it, you need to do a floating point divide. you're here doing an integer divide, throwing away the decimal part and storing only the integer result into dMircoSecond.

This will work and give you a better result.

Code:

---

        dMircoSecond = (double)m_lnFreq.QuadPart/1000000;

---

Quote:

---

Originally Posted by Paul McKenzie

it could be characters, floating point, integers, etc.

---

It could be byte.

Quote:

---

Originally Posted by Paul McKenzie

That's why I'm stating to you that you are not converting anything. You're interpreting the data differently, but you are not converting. You're simply copying the same data from one memory block to another memory block, and then at the destination block, you're interpreting that data as pointing to a series of "bytes".

---

I clearly understood. Thanks Mr.Paul. How can I copy the data from one memory block to another. That is, I will copy the PVOID pointer data into byte *pointer like below.

Code:

---

PVOID pvData = (PVOID)data;
byte *bPoint = (byte*) pvData;

byte TempArr[1024];
byte *bTo = TempArr;

        memcpy(bTo, bPoint, 1024);

---

Then I will copy the byte pointer data into byte array.
Am I correct?

Quote:

---

Originally Posted by GCDEF

all you need to do is change your pointer type.

---

Yes. I agree . How is possible?

Quote:

---

Originally Posted by 2kaud

which as expected shows that using memcpy is the fastest and array indexing the slowest.

---

Ok I will use and come back.

Quote:

---

Originally Posted by OReubens

This is wrong. you're here doing an integer divide, throwing away the decimal part and storing only the integer result into dMircoSecond.

---

I will change my code.

Quote:

---

I have PVOID data only. But I like to draw real time image using byte array.

---

If all you need is a byte pointer to the data for drawing purposes, then replace

Code:

---

PVOID pvData = (PVOID)data;
byte *bPoint = (byte*) pvData;

byte TempArr[1024];
byte *bTo = TempArr;

memcpy(bTo, bPoint, 1024);

---

with

Code:

---

byte *TempArr = (byte*)data;

---

and then just draw from TempArr as currently.

Quote:

---

Originally Posted by saraswathisrinath

I clearly understood. Thanks Mr.Paul. How can I copy the data from one memory block to another. That is, I will copy the PVOID pointer data into byte *pointer like below.

---

First, it isn't clearly understood as to why you want to copy the data.

Quote:

---

My requirement is, I'm getting PVOID data continuously, using that i convert into 2D byte array & draw an image.Important note is these process are done by minimum 1 millisecond. If process time exceed then i miss some data in real time.

---

So you're getting this data "continuously", but you don't mention if that memory block is being continuously overwritten in an asynchronous manner. Real-time programming doesn't just mean you get the data fast -- where is the code to ensure that the block of data is not overwritten while you're copying or processing the data? Where is the queue or queue-like implementation to ensure that whatever data you get, it goes on your queue so that nothing is missed or overwritten?

Yes, processing data quickly is a requirement so that you don't overload the queue, but at the same time, you have to ensure that you don't miss any data coming in. You don't do that by "coding the fastest routine" and hoping that the routine processes all the data before some real-time function refreshes the buffer with new data. That is a naive and wrong approach. You have to use the proper data structures (probably a queue), that caches the data that comes in so you don't miss anything.

Regards,

Paul McKenzie

Quote:

---

I'm getting PVOID data continuously

---

Where is this data coming from and how does it get set into the memory pointed to by pvData? Is one thread producing this data and another thread processing it?

Hi Mr.2kaud & Mr.Paul,

Sample code like below,

Code:

---

byte TempArr[1024];

PVOID pvData = NULL; //store a line data into a pointer

pvData = GetCardData(); //I'm just using this function only. All functions are in the form of dll.
// Its getting from owner of card seller. using this Getting one line data = 1024 bytes @ every milliseconds

byte *bPoint = (byte*)pvData;

for(int i = 0; i < 1024; i++)                TempArr[i] = (byte)(*(bPoint + i)); //Started to copy the data for drawing purpose
//For this transaction only I'm getting delay.

---

But using this I'm missing a line data for every line like 1,3,5,7,.... etc.

Quote:

---

Originally Posted by Paul McKenzie

you have to ensure that you don't miss any data coming in. You don't do that by "coding the fastest routine" and hoping that the routine processes all the data before some real-time function refreshes the buffer with new data. That is a naive and wrong approach. You have to use the proper data structures (probably a queue), that caches the data that comes in so you don't miss anything.

---

after getting a line data, If I add any process means, the missing line increases like 1,5,11,etc... Sure Data are coming correctly. Myself only missing. Where is the problem?

Quote:

---

Originally Posted by 2kaud

Where is this data coming from and how does it get set into the memory pointed to by pvData? Is one thread producing this data and another thread processing it?

---

I'm using a card it gives the data at every millisec 1024 bytes. And the memory pointed to pvData. Using thread concept I'm pointing Data by pvData continuously and using paint method to draw an image.

Quote:

---

Originally Posted by saraswathisrinath

... after getting a line data, If I add any process means, the missing line increases like 1,5,11,etc... Sure Data are coming correctly. Myself only missing. Where is the problem?

---

It seems that your painting takes much more time than reading the data...

Quote:

---

Originally Posted by saraswathisrinath

I'm using a card it gives the data at every millisec 1024 bytes. And the memory pointed to pvData. Using thread concept I'm pointing Data by pvData continuously and using paint method to draw an image.

---

Which "thread concept" do you mean? I don't see any "thread concept" in the code snippet you have posted.

Besides, like Paul wrote:

Quote:

---

Originally Posted by Paul McKenzie

You have to use the proper data structures (probably a queue), that caches the data that comes in so you don't miss anything.

---

Do you use something like that?

Quote:

---

Originally Posted by saraswathisrinath

[/code]But using this I'm missing a line data for every line like 1,3,5,7,.... etc.
after getting a line data, If I add any process means, the missing line increases like 1,5,11,etc... Sure Data are coming correctly. Myself only missing. Where is the problem?

---

The problem is that you didn't put much thought in your design.

The solution is very simple in concept, and I explained briefly before, I will try again.

1) You are trying to write the fastest routine possible so that you can process the data quickly enough before the buffer gets overwritten with new data. This is the wrong approach.. There is no guarantee that your fast routine will be fast enough, so get rid of this idea of trying to write code that attempts to be speedier than the hardware that is sending you the data. What is speedy today may miss a byte or two of data tomorrow.

2) The solution or one such solution is to cache the data coming in, so that you don't lose any data. You save the incoming data on the back of the queue, while at the same time, you're processing the data that is sitting in front of the queue.

For the solution, you make the queue big enough so that it fits in with the processing time it takes to do one batch of data, plus accept any new data coming in. This is where experiments and trial/error comes in to see how big to make the queue. Once you have that set up, there is very little chance at all of missing data. If your drawing code is slow, then you make the maximum queue size large, if it's fast drawing code, then you can make the maximum queue size smaller. So basically, the speed of your graphics code won't matter.

Think of this like a cashier at a supermarket. The customers want to pay for the items they bought, so they must stand in line (a queue), waiting to be processed. Yes, the cashier needs to process each customer quickly, but there is no way a cashier can process a customer every single second or every two seconds.

Your code is trying to be the cashier that can process a customer every one or two seconds, therefore process each customer as they are about to check out of the store (which means no line is ever formed). You know that is practically impossible for a cashier to process a customer every second. So the solution is the same as what a supermarket does -- you create a line (queue) that where incoming checkout customers line up to be processed. If the cashier is not as fast as expected, then the queue will be longer, but every customer will be processed.

In your case, the customers are the same as the data the card is giving you, your graphics/drawing code is the cashier. So can you envision this? You make a queue for the data that the card is sending you. Any new data the card sends you gets placed at the back of the queue, while your graphics code processes the data that is in front of the queue.

For the queue, you will need proper synchronization so that reading and writing to the queue are done in a safe manner. So the only major thing you would need to do now is to implement the queue. There is std::queue from STL that can work for this, but again, you'll have to add the synchronization to ensure that you're not writing to the queue while reading from the queue, etc.

If your drawing code is very fast, e.g. the queue is empty most of the time, then that's ok -- just wait for data to be placed on the queue by the card.

Again, all of this should have been envisioned if you truly wanted to process real-time data. In no way should you have started trying to beat the card's speed. What if the processor(s) you were writing for are slow? What if the compiler's optimizer was bad (I know it's not, but what if it was slow)? What then? Get a faster computer? Or do you write the code so that no matter how slow or "bad" the drawing code is, it will process all the data that's coming in.

Regards,

Paul McKenzie

Other processes running would also affect the speed of your program.

I am just wondering if one line of data (1024 bytes) will be drawn to one line of the display that was created in Draw Image Using 2D Byte Array ?. Does this display scroll vertically after each line is drawn? Or does each successive line of data get drawn on a successive line of the display until the display is full and then the drawing starts at the top of the display again? What are the dimensions of the display? (Width = 1024?, Height = ?) With the rate of data you are getting (1,024,000 bytes per second), it seems that the entire display would be redrawn a few times a second. Is this data coming from a camera? The InvalidateRect() function only needs to be called at most 60 times per second. Calling it more often would be unnecessary since your monitor does not update that fast.

Is the DLL allocating the memory for the incoming data array? Is this memory location static? If not, where is it freed? If your code could allocate the memory and pass the DLL a pointer to that memory, you could pass the DLL a pointer to a line of your image data. This would do the drawing automatically.

Quote:

---

Originally Posted by VictorN

It seems that your painting takes much more time than reading the data.

---

I Already removed Paint and run my application. But no changes in the result.

Quote:

---

Originally Posted by VictorN

Which "thread concept" do you mean? I don't see any "thread concept" in the code snippet you have posted.

---

First I used Timer. called at every 1 milli seconds. This case I missed 15 to 20 lines. So I go to threads.
Your reference,

Code:

---

CWinThread *conThread;
PVOID pDaqData = NULL;
RETURN_CODE rc;       

byte *bPnt;

void CMyDlg::OnBnClickedRealTimeData()
{               
        if(m_bDaqContFlag)
        {
                UpdateData(TRUE);
                pDaqData = NULL;
                conThread = ::AfxBeginThread(RealDataThread, this);
        }
        else
        {       
                //End Data Capture
                conThread->SuspendThread();
                DataAaqEnd(m_nDevNo); //Dll function from Card Seller
        }
}
UINT CMyDlg::RealDataThread(LPVOID rd)
{       
        CMyDlg *obj = (CMyDlg *) rd;
        obj->realData();
        return true;
}

void CMyDlg::realData()
{       
               
    LARGE_INTEGER m_lnStart;
    LARGE_INTEGER m_lnEnd;
    LARGE_INTEGER m_lnFreq;

    QueryPerformanceFrequency(&m_lnFreq);//Retrieves the frequency of the high-resolution performance counter
    QueryPerformanceCounter( &m_lnStart);//Retrieves the current value of the high-resolution performance counter
 
  pDaqData = GetData(m_nDevNo, &rc);        //Dll function from Card Seller
               
    bPnt = (byte*)pDaqData;
        byte *bTo = TempRealDataArray;
        memcpy(bTo, bPnt,2048);

        for(int i = 0; i < 2048; i++)
        {
                TempRealDataArray[i] = (*(bTo + i)); //This array data passed to paint method to drawn an image.
                if(i == 1030) Temp[cnt] = TempRealDataArray[i];        //To find the missed lines.
                                                                                        // Temp[] stored the single data per line and print a file after stop the process
        }
        cnt++;
       
        QueryPerformanceCounter( &m_lnEnd );

        __int64 nDiff(0) ;
    double dMircoSecond( 0 ) ;
    if( m_lnStart.QuadPart !=0 && m_lnEnd.QuadPart != 0)
    {
        dMircoSecond = (double)(m_lnFreq.QuadPart)/1000000;
        nDiff = m_lnEnd.QuadPart-m_lnStart.QuadPart ;
        dMircoSecond = nDiff/dMircoSecond;
                CString str;
                str.Format("%d", nDiff);
                GetDlgItem(IDC_TIMEDIF)->SetWindowText(str);                //Takes 1200 to 1500 micro seconds
    }       

        InvalidateRect(&rect);        //Already defined the rect { 0, 0, 1024, 256}
        conThread = ::AfxBeginThread(RealDataThread, this);

}

---

I don't know, here Am I using the proper Data Structure ? Please guide me.

Quote:

---

Originally Posted by Paul McKenzie

The solution or one such solution is to cache the data coming in, so that you don't lose any data. You save the incoming data on the back of the queue, while at the same time, you're processing the data that is sitting in front of the queue.

---

Am I cache the Real Data correctly? Please refer the code above which I post.

Mistake is in my side. But sure I tell Painting & the compiler's optimizer not reason for this. B'cos I removed paint method and run my code and also run my application with higher configuration PC.

Quote:

---

Originally Posted by Coder Dave

What are the dimensions of the display? (Width = 1024?, Height = ?)

---

Width = 1024 & Height = 256.

Now, My problem is not drawing the image.

My first aim is getting the data correctly from camera with out any missing data.

If I achieve this then my problem solved 90%.

Pls take me at right path.....

Code:

---

for(int i = 0; i < 2048; i++)
{
        TempRealDataArray[i] = (*(bTo + i)); //This array data passed to paint method to drawn an image.
        if(i == 1030) Temp[cnt] = TempRealDataArray[i];        //To find the missed lines.
                                                                                        // Temp[] stored the single data per line and print a file after stop the process
}

---

The highlighted line in red just copies TempRealDataArray to itself! - as bTo is set to point to the beginning of TempRealDataArray.

The green line is just an attempt at a kludge to get the data because you are not using the proper structures for obtaining the data that have been suggested previously by Paul. The above code is not needed and should be removed.

You need to implement properly a queue with the producer being your realdata method and the consumer being the method/function to process or save the data as needed. The producer adds to the queue at the back and the consumer takes from the queue at the front. The queue consists of elements of 2048 bytes. Everytime your realdata method in its thread calls getdata() it copies the data to a new queue element at the back of the queue (add queue element - push in STL terms). Your consumer function takes elements from the front of the queue if there are any and processes the data (remove queue element - pop in STL terms). This can be done using the STL queue class and appropriate thread syncronisation.

See below for details of the STL queue
http://www.cplusplus.com/reference/queue/queue/

Quote:

---

Originally Posted by saraswathisrinath

I Already removed Paint and run my application. But no changes in the result.

---

The retrieval of the data from the card should do nothing except place the data onto the queue. Once the data is copied to the queue, that's it -- the reading of the card data is done. Instead, you're calling InvalidateRect and all sorts of other things during the reading of the data -- this is wrong.

The producer would do something like this.

Code:

---

while (forever)
{
    while (card_says_I_have_data)
      AddDataFromCardToBackOfQueue();
}

---

If it's anything more than that, you're not understanding the concepts that I've given you as well as 2kaud and Victor. You shouldn't be invalidating rectangles, putting together drawing buffers, etc.

On another thread (the consumer), you should then do this (this is happening "at the same time" as the code above):

Code:

---

while (forever)
{
    if ( queue_not_empty )
    {
      RemoveDataFromFrontOfQueue()
      DrawMyImageWithTheData();
    }
}

---

The while (forever) will quit at some time, either it is signalled that no more data will come in or by some other means. Of course, you have to use the proper synchronization between the consumer/producer threads so that the queue doesn't become corrupted. You could even "lock" the queue with the RemoveDataFromFromOfQueue (and even in this case, to make the read "fast", you only read at most, say 500 bytes at a time). Then when the Read is done, you, unlock the queue so that the producer (the card) places data into the queue.

If you don't understand the producer/consumer paradigm or do not understand what a queue is, just say so. It seems you're not quite getting why your method is flawed, and why I suggested that you should just scrap it and use proper techniques. Just copying data to an array in a "simple" loop from begin to end is not a queue. You can use an array as a queue (but not necessary as there is std::queue built for this), but unless you use that array correctly as a queue, you're not going to get anywhere.

A queue requires that you copy data to the back of the array, while the processing of the data uses the data in the front of the array.

As an example, let's say that the queue has room for 10,000 bytes. Then 2,000 bytes of data comes in from the card (the producer). You place this data in the queue, and at the same time, the drawing function (the consumer) is looking at the queue for any data -- it is not looking or even care what the card is doing.

So now let's say your drawing function has drawn 1,500 bytes of data, and then the card sends 2000 bytes of data. Your card reading thread takes this data and places the card data in the queue -- nothing is being overwritten, since the data is placed in the back of the line.

The queue had 9500 bytes of space since the drawing completed 1,500 bytes. At the same time, the drawing function finishes up with the remaining 500 bytes it had to do the first time, then it looks at the queue and sees that there are 2,000 more bytes of data, and then draws that data.

Let's say the drawing function now processes 1,999 bytes of data, and then the card sends another 2000 bytes of data right when the drawing program is going to draw the last byte. You place the new card data at the back of the queue (which can hold 9,999 bytes, since the drawing processed 1,999 bytes). The drawing function finishes drawing that last remaining byte, looks at the queue, and sees it needs to draw more data.

This producer/consumer interaction goes on forever, until the drawing ends. Do you now see that you never miss any data, provided that the queue is big enough? Do you see where your current method fails miserably? Do you see why having the fastest code in the world doesn't matter with using a queue? With your current method, all of the times when the drawing function didn't draw all the bytes, you got into trouble. With the queue, you will never get into trouble (and your function to draw doesn't even need to be fast, albeit it is good that it is fast).

Quote:

---

First I used Timer. called at every 1 milli seconds. This case I missed 15 to 20 lines. So I go to threads.

---

Threads are not going to help you -- you need to use the proper data structures, and you're not doing it. If you used the proper structure, it doesn't matter what the timings are.

Code:

---

TempRealDataArray[i] = (*(bTo + i)); //This array data passed to paint method to drawn an image.

---

There should be no drawing at all, or even mention anything about paint methods. This is the time where you place the data onto the queue and return.

Code:

---

if(i == 1030)[B] Temp[cnt] = TempRealDataArray[i];        //To find the missed lines.
                                                                              // Temp[] stored the single data per line and print a file after

---

Code:

---

GetDlgItem(IDC_TIMEDIF)->SetWindowText(str);                //Takes 1200 to 1500 micro seconds

---

And if it took 10,000 microseconds, then what do you do? What should be happening is that this painting routine doesn't care one bit about the timing, as long as the queue has room for more data coming in from the card. If the drawing routine takes a long time so that the queue is too small, then make the queue bigger -- that's how you fix the problem. You don't fix it by trying to write the fastest code in the world.

Quote:

---

Mistake is in my side. But sure I tell Painting & the compiler's optimizer not reason for this. B'cos I removed paint method and run my code and also run my application with higher configuration PC.

---

Again, forget about what the compiler's optimizer does, and forget about how fast your PC is, and honestly forget about QueryPerformanceCounter() -- pretend that function doesn't exist. None of those things would hardly matter if you coded using the proper data structures.

Quote:

---

My first aim is getting the data correctly from camera with out any missing data.
If I achieve this then my problem solved 90%.

---

If you took the advice of creating a queue, then 100% or close to it would be solved.

Quote:

---

Pls take me at right path.....

---

We are trying, but are you listening? So far you're still stuck on using the same broken design. Again, forget about what you wrote -- it won't work if you do not want to miss any data.

My feeling is that you're trying to code in a "simple" fashion instead of actually coming up with a design. The card has data, you draw the data to the screen, and if the data comes in too fast, you create the drawing code even faster. Sure, that's the easy way to think about it, and frankly any beginner would have done exactly what you've done. But it isn't correct and it is seriously flawed. You have to actually use certain design patterns to overcome the problem.

Regards,

Paul McKenzie

Quote:

---

Originally Posted by saraswathisrinath

Code:

---

        memcpy(bTo, bPnt,2048);

---

---

Why are you copying 2048 bytes when the input buffer is only 1024 bytes?

So your application is basically displaying the output of a black and white digital camera in a window. You are receiving 1,024,000 bytes/pixels per second from the camera. Your display has 1024 x 256 = 262,144 pixels, so it takes 256 milliseconds to receive one full image from the camera. That is, you are displaying 1,024,000 / 262,144 = 3.90625 full images per second. Even at this relatively slow speed for a video camera, if you miss a few lines of pixels here or there, they will be overwritten anyway after just a little more than a quarter second. Unless the camera data is being used for something other than just the display, does it really matter if some of the data is lost? I think your real problem is synchronizing the input signal with the display so that every pixel, that gets received, is drawn at the correct spot on the display.

Hi,
Thanks to support me. I'm beginner only.

I didn't use Queue before that. I learned the queue concept. By using the queue concept, feeling very difficulty.

I stored the data into the queue , now how can I retrieve the data from that?. I know the method pop(). But i want byte array format.
The sample code, i didn't try just i share my doubt before use the code.

Code:

---

#include <queue>
using namespace std;
queue<PVOID> mypvoidQ;

void CMyDlg::realData()
{       
  while (forever)
  {
      while (card_says_I_have_data)
              AddDataFromCardToBackOfQueue();

      if (!mypvoidQ.empty() )
      {
            RemoveDataFromFrontOfQueue(); //How is possible? how can i copy the data into byte array
            DrawMyImageWithTheData();
      }

  }
}
void CMyDlg::AddDataFromCardToBackOfQueue()
{
  pDaqData = GetData(m_nDevNo, &rc);       
  mypvoidQ.push(pDaqData);
  mypvoidQ.back();
}
void CMyDlg::RemoveDataFromFrontOfQueue()
{
  mypvoidQ.front();
  mypvoidQ.pop();//How is possible? how can i copy the data into byte array
}

---

Really I felt very bad about me. before use codegure, no one here to help me. But now, all of then help me but my knowledge, I'm felt very difficult :(
I will study the concept and use and come back with you. Please give any sample code for me about queue.

Quote:

---

queue<PVOID> mypvoidQ;

---

Er... No. queue needs a type of data that is to be stored. PVOID is not a suitable type as this is just a pointer to void.

I've knocked up some code to demonstrate the idea between a producer thread and a consumer thread for you. Its just win32 not MFC but you should get the idea about using a queue and sync between the threads. I don't have a camera to test so I don't know the timings for loss of data - but if this sort of program design still looses data from the camera then you really are in trouble as all the producer is doing is adding blocks of data to the queue.

Hope this helps

Code:

---

#include <iostream>
#include <queue>
#include <conio.h>
using namespace std;

#include <windows.h>
#include <process.h>

#define BUFSIZE        2048

//Structure with methods for camera data blocks
struct scam {
        char* pdata;

        scam()
        {
                pdata = (char*)new char[BUFSIZE];
                memset(pdata, 0, BUFSIZE);
        }

        scam (const char* dat)
        {
                size_t ll = strlen(dat);

                if (ll > BUFSIZE - 1) {
                        ll = BUFSIZE - 1;
                }

                pdata = (char*)new char[BUFSIZE];
                memset(pdata, 0, BUFSIZE);
                memcpy(pdata, dat, ll);
                pdata[ll + 1] = 0;
        }

        scam(const char *dat, int no)
        {
                pdata = (char*)new char[BUFSIZE];
                memcpy(pdata, dat, min(no, BUFSIZE));
        }

        ~scam()
        {
                delete [] pdata;
        }

        scam(const scam& scm)
        {
                pdata = (char*)new char[BUFSIZE];
                memcpy(pdata, scm.pdata, BUFSIZE);
        }

        scam& operator=(const scam& scm)
        {
                if (this != &scm) {
                        delete [] pdata;
                        pdata = (char*)new char[BUFSIZE];
                        memcpy(pdata, scm.pdata, BUFSIZE);
                }
                return *this;
        }

        void SetData(const char *dat, int no)
        {
                memset(pdata, 0, BUFSIZE);
                memcpy(pdata, dat, min(no, BUFSIZE));
        }

};

CRITICAL_SECTION        ccrit;

HANDLE        sync;

queue<scam> camdata;

//CONSUMER
unsigned __stdcall consume(void* pArguments)
{
scam        dataOut;

        while (true) {
                dataOut = "";
                WaitForSingleObject(sync, INFINITE);
                EnterCriticalSection(&ccrit);
                dataOut = camdata.front();
                camdata.pop();
                if (!camdata.empty()) {
                        SetEvent(sync);
                }
                LeaveCriticalSection(&ccrit);

                //PROCESS CAMERA DATA BLOCK HERE
                cout << dataOut.pdata << endl;
        }
        return 0;
}


int main()
{
unsigned        thid;

        sync = CreateEvent(NULL, FALSE, FALSE, "camev");

        InitializeCriticalSection(&ccrit);

        _beginthreadex(NULL, 0, &consume, NULL, 0, &thid);

        //PRODUCER
scam        dataIn1;

        //test loop - would be 'while there is data from camera'
        for (int i = 1; i < 50; i++) {
                EnterCriticalSection(&ccrit);

                //ADD DATA BLOCK HERE FROM CAMERA
                dataIn1.SetData("dfdgrehgfdhghghgdhgdhghngfegfrzz", 32);
                camdata.push(dataIn1);

                SetEvent(sync);
                LeaveCriticalSection(&ccrit);
        }

        _getch();
        DeleteCriticalSection(&ccrit);
        return (0);
}

---

Quote:

---

Originally Posted by saraswathisrinath

Hi,
Thanks to support me. I'm beginner only.

---

Well this is why in a real situation, it takes an experienced programmer to process "live" data correctly and efficiently. You can't really write a program with skills of a beginner to accomplish this.

First, get the idea of how a queue works by writing a simple program. Don't try and use your current program to learn how to use a queue.

For example, the queue::front() function returns a reference to the item that is first in the queue, but your sample program does nothing with the return value. All the queue::pop() does is remove the item from the queue -- it doesn't return to you a reference to the item. Second, why are you calling queue::back()? The push() automatically places the item in the back of the queue.

So even in a very simple situation, you're not using the queue correctly and you really are not sure what those member functions do.

Code:

---

#include <queue>
#include <cstring>

struct CardData
{
  char buffer[2048];
};

typedef std::queue<CardData> CardDataQueue;

using namespace std;

int main()
{
  CardData d1;
  CardDataQueue qData;
  strcpy(d1.buffer, "abc123");
  qData.push( d1 );
  strcpy(d1.buffer, "xyz");
  qData.push( d1 );
  //...
  CardData& mData = qData.front();  // this should get the abc123
  qData.pop();  // removes the top item
  CardData& mData2 = qData.front(); // this should get the xyz
}

---

Something like this is what you should have first attempted, so you get the idea of how to use push(), pop(), and front.

Regards,

Paul McKenzie

Thanks Mr.Paul. Really the above examples makes me to clear. If any doubt, i will come back.

Hi,
After long time, I came back and also my problem solved :).

I used the Buffer to store the data (1000 line of data) & used the memcpy function to copy the data in to the byte pointer. After that, I used the file method to print the data.Now, I received the data without missing. Really happy.

Single time(1000 lines data) real time data received successfully. here after trying to get the real time data continuously.

Thanks to all Guru's.