[RESOLVED] Processing Time For PVOID Into Byte Array Conversion

[saraswathisrinath]

Hi,

In my project I'm using PVOID pointer to store real time data.

After getting this data I will convert the data into byte array, like below

byte *bPoint = NULL;
PVOID pvData;
byte TempArr[1024];
bPoint = (byte*) pvData;
for(int i=0;i<1024;i++)
    TempArr[i] = (byte) (*bPoint + i);

Processing time for the above code takes 9500 to 9900 microseconds (Used QueryPerformanceCounter).

TempArr[0] = ((BYTE*) pvData) [0];

This code takes 1100 to 1200 microseconds.

My doubt is, The processing time of PVOID data into byte array conversion takes time like above?

Or any other easy way(PVOID data into byte array conversion) to reduce the processing time?

Please clear me.

[Paul McKenzie]

Hi,

In my project I'm using PVOID pointer to store real time data.

After getting this data I will convert the data into byte array, like below

Why aren't you using memcpy()?

Regards,

Paul McKenzie

[Paul McKenzie]

After getting this data I will convert the data into byte array, like below

Also, you are not converting anything. What you're doing is copying from one memory block to another. Mass copying is done efficiently using memcpy(), and not by writing a for-loop.

On the other hand, converting implies you're using some sort of algorithm or function to transform from one type to another, and you're not doing that here.

Regards,

Paul McKenzie

[OReubens]

Processing time for the above code takes[B] 9500 to 9900 microseconds (Used QueryPerformanceCounter).

9500 microseconds is 9.5 milliseconds, or 0.0095seconds... copying 1024 bytes does not take that long, not even on an original IBM XT would it take that long.

Note that QueryPerformanceCounter does NOT return microseconds.
What it returns is a computer dependant value that indicates a discrete time interval for that specific computer. You need to use QueryPerformanceFrequency() to convert the results from QueryPerformanceCounter() into actual second-based values.

[OReubens]

Mass copying is done efficiently using memcpy(), and not by writing a for-loop.

Not necessarily. A smart compiler optimizer will see the sequence and convert it to a memcpy on it's own, or potentially even tailor a specific optimal bit of copying code. Don't assume the compiler will simply make a loop and copy bytes one by one.

Then again, don't assume the compiler WILL make this optimisation either ;-)

[2kaud]

Or any other easy way(PVOID data into byte array conversion) to reduce the processing time?

First, your code has a problem

TempArr[i] = (byte) (*bPoint + i);

should be

TempArr[i] = (byte) *(bPoint + i);

to correctly index memory pointed to by bPoint

Using an array index within the loop is the most expensive way of performing the copy. On my computer using VS fully optimised for speed, this gives an average of 4140 (using QueryPerformanceCounter)

PVOID pvData = (PVOID)data;
byte *bPoint = (byte*) pvData;

byte TempArr[1024];
byte *bTo = TempArr;

	for(int i = 0; i < 1024; i++)
		TempArr[i] = (byte) *(bPoint + i);

Just changing the assignment to use pointers rather than index reduces the value to average 880

PVOID pvData = (PVOID)data;
byte *bPoint = (byte*) pvData;

byte TempArr[1024];
byte *bTo = TempArr;

	for(int i = 0; i < 1024; i++)
		*bTo++ = *bPoint++;

And using memcpy reduces this further to average 850

PVOID pvData = (PVOID)data;
byte *bPoint = (byte*) pvData;

byte TempArr[1024];
byte *bTo = TempArr;

	memcpy(bTo, bPoint, 1024);

[saraswathisrinath]

Why aren't you using memcpy()?

I have PVOID data only. But I like to draw real time image using byte array. So I convert this into byte array. I get that method from net and I used. After that only I noticed, my program getting delay for this data conversion.

On the other hand, converting implies you're using some sort of algorithm or function to transform from one type to another, and you're not doing that here.

I never used before any algorithm for data conversion. Please give any reference, after that I will try in my project.

You need to use QueryPerformanceFrequency() to convert the results from QueryPerformanceCounter() into actual second-based values.

I'm processing in the speed of Milliseconds. If I return second based means it must return 0 only.

LARGE_INTEGER m_lnStart;
    LARGE_INTEGER m_lnEnd;
    LARGE_INTEGER m_lnFreq;

    QueryPerformanceFrequency(&m_lnFreq);//Retrieves the frequency of the high-resolution performance counter
    QueryPerformanceCounter( &m_lnStart);//Retrieves the current value of the high-resolution performance counter
   
	byte *bPoint = (byte*)pvData;

	for(int i = 0; i < 1024; i++)
		TempArr[i] = (byte)(*(bPoint + i));

   	QueryPerformanceCounter( &m_lnEnd );

	__int64 nDiff(0) ;
    double dMircoSecond( 0 ) ;
    if( m_lnStart.QuadPart !=0 && m_lnEnd.QuadPart != 0)
    {
        dMircoSecond = (double)(m_lnFreq.QuadPart/1000000);
        nDiff = m_lnEnd.QuadPart-m_lnStart.QuadPart ;
        dMircoSecond = nDiff/dMircoSecond;
		CString str;
		str.Format("%d", nDiff);
		GetDlgItem(IDC_TIMEDIF)->SetWindowText(str);		
    }

First, your code has a problem

Sorry Mr.2kaud, that's my mistake. But I'm correctly used in my project.

And using memcpy reduces this further to average 850

850 microseconds or Milliseconds?

My requirement is, I'm getting PVOID data continuously, using that i convert into 2D byte array & draw an image.Important note is these process are done by minimum 1 millisecond. If process time exceed then i miss some data in real time.

[Paul McKenzie]

I have PVOID data only.

There is no such thing as "PVOID data".

You have a pointer, and starting at that pointer is data in one contiguous block. How you interpret that contiguous block could be anything -- it could be characters, floating point, integers, etc. But just because the data is interpreted differently doesn't change the data. The data at that memory location stays exactly the same.

That's why I'm stating to you that you are not converting anything. You're interpreting the data differently, but you are not converting. You're simply copying the same data from one memory block to another memory block, and then at the destination block, you're interpreting that data as pointing to a series of "bytes".

An example of a real conversion would be going from ANSI to Unicode, or from binary to hexadecimal, or going from little-endian to big-endian where you need to actually change the data by swapping bytes, etc.

Regards,

Paul McKenzie

[Paul McKenzie]

I never used before any algorithm for data conversion. Please give any reference, after that I will try in my project.

I'm not saying to use an algorithm. What I'm saying is that what you posted is not conversion. It is copying data from one memory location to another. I don't know how I can put it more simply.

So your goal is to copy contiguous data from one memory location to another -- that is what everyone responding to you is giving you advice on. The fastest way is to write the copying in assembly language using the fastest mass-copying assembly instructions for your CPU. You can't get any faster than that, regardless of your requirements.

Regards,

Paul McKenzie

[GCDEF]

Is the only reason you're copying it because you think you're converting it? If so, all you need to do is change your pointer type.

[2kaud]

850 microseconds or Milliseconds?

The numbers are the difference between two calls to QueryPerformanceCounter. For the sake of the comparisons I did it doesn't really matter what are the units of measurement - just the relative numbers - which as expected shows that using memcpy is the fastest and array indexing the slowest.

[OReubens]

        dMircoSecond = (double)(m_lnFreq.QuadPart/1000000);

This is wrong.
If you have to go this somewhat awkward way about it, you need to do a floating point divide. you're here doing an integer divide, throwing away the decimal part and storing only the integer result into dMircoSecond.

This will work and give you a better result.

        dMircoSecond = (double)m_lnFreq.QuadPart/1000000;

[saraswathisrinath]

it could be characters, floating point, integers, etc.

It could be byte.

That's why I'm stating to you that you are not converting anything. You're interpreting the data differently, but you are not converting. You're simply copying the same data from one memory block to another memory block, and then at the destination block, you're interpreting that data as pointing to a series of "bytes".

I clearly understood. Thanks Mr.Paul. How can I copy the data from one memory block to another. That is, I will copy the PVOID pointer data into byte *pointer like below.

PVOID pvData = (PVOID)data;
byte *bPoint = (byte*) pvData;

byte TempArr[1024];
byte *bTo = TempArr;

	memcpy(bTo, bPoint, 1024);

Then I will copy the byte pointer data into byte array.
Am I correct?

all you need to do is change your pointer type.

Yes. I agree . How is possible?

which as expected shows that using memcpy is the fastest and array indexing the slowest.

Ok I will use and come back.

This is wrong. you're here doing an integer divide, throwing away the decimal part and storing only the integer result into dMircoSecond.

I will change my code.

[2kaud]

I have PVOID data only. But I like to draw real time image using byte array.

If all you need is a byte pointer to the data for drawing purposes, then replace

PVOID pvData = (PVOID)data;
byte *bPoint = (byte*) pvData;

byte TempArr[1024];
byte *bTo = TempArr;

memcpy(bTo, bPoint, 1024);

with

byte *TempArr = (byte*)data;

and then just draw from TempArr as currently.

[Paul McKenzie]

I clearly understood. Thanks Mr.Paul. How can I copy the data from one memory block to another. That is, I will copy the PVOID pointer data into byte *pointer like below.

First, it isn't clearly understood as to why you want to copy the data.

My requirement is, I'm getting PVOID data continuously, using that i convert into 2D byte array & draw an image.Important note is these process are done by minimum 1 millisecond. If process time exceed then i miss some data in real time.

So you're getting this data "continuously", but you don't mention if that memory block is being continuously overwritten in an asynchronous manner. Real-time programming doesn't just mean you get the data fast -- where is the code to ensure that the block of data is not overwritten while you're copying or processing the data? Where is the queue or queue-like implementation to ensure that whatever data you get, it goes on your queue so that nothing is missed or overwritten?

Yes, processing data quickly is a requirement so that you don't overload the queue, but at the same time, you have to ensure that you don't miss any data coming in. You don't do that by "coding the fastest routine" and hoping that the routine processes all the data before some real-time function refreshes the buffer with new data. That is a naive and wrong approach. You have to use the proper data structures (probably a queue), that caches the data that comes in so you don't miss anything.

Regards,

Paul McKenzie