-
[RESOLVED] Why does this code leak memory?
Code:
// Check to see if the Found array needs to be expanded
if (Entry > Max)
{
// Give the array 30 more spaces to fill
Max += 30;
/* Create a temp array with the amount of space desired,
if the memory cannot be allocated return an error */
Temp = new POINT[Max];
if (!Temp)
{
delete [] Found;
LastERR = FAILED_MALLOC;
return NULL;
}
// Loop through the original and copy it to the new array
for (int iCountARY = 0; iCountARY < Entry; iCountARY++)
{
Temp[iCountARY] = Found[iCountARY];
}
// Release the old array
delete [] Found;
Found = Temp;
}
I'm a beginner to C++ and am trying to check if a dynamic array needs to be expanded, and if so, expand it. But my compiler (Dev-C++) is giving me this warning
Code:
warning: heap block at 003F3CF8 modified at 003F3DF0 past requested size of f0
Which I think, after a little research, means that I was trying to access memory outside of my dynamic array or something along those lines. If anyone could quickly go over what I'm doing wrong I'd really appreciate it, thanks for reading!
-
Re: Why does this code leak memory?
arrays are zero indexed, so your conditional should be
since your are allocating an array of size MAX.
I am not sure if that was clear so
POINT[5], contains 5 POINTS which you access with indices
0, 1, 2, 3, 4
using 5 for access will give the error you see
-
Re: Why does this code leak memory?
It doesn´t leak memory, you are accessing a memory location beyond array bonds. This will happen if (Entry +30) is larger than Max, because you resize your array by 30 instead of looking how many elements you´re gonna need. If you replace the line
Code:
// Give the array 30 more spaces to fill
Max += 30;
by
[code]
// adjust array size to hold Entry elements
Max = Entry;
[/b]
or
Code:
while( Entry > Max )
{
// resize array until it´s large enough to hold Entry elements
Max = Max * 2;
}
Maybe you can use std::vector instead?
-
Re: Why does this code leak memory?
Quote:
Originally Posted by Brownhead
I'm a beginner to C++ and am trying to check if a dynamic array needs to be expanded, and if so, expand it.
Use std::vector. Not only is it safer, it is much smarter and faster than using naked new[]/delete[] calls to do this.
Code:
#include <vector>
int main()
{
std::vector<int> IntArray(10); // an array of 10 integers.
IntArray.resize(20); // now it has 20 integers
IntArray[0] = 10; // use just like a regular array
}
There is your entire code in two lines.
Also, as mentioned before, what you're doing is slow compared to using vector and resize(). Believe me. I've timed code that looks like yours, and vector::resize() as opposed to using new[]/delete[] is many times faster because resize() has more intelligence than just doing straight new[] and delete[] calls each time the size changes.
Also, operator new[] does not return NULL if there is a failure. A conforming C++ compiler throws a std::bad_alloc exception if new fails.
Regards,
Paul McKenzie
-
Re: Why does this code leak memory?
Code:
Temp = new POINT[Max];
...
for (int iCountARY = 0; iCountARY < Entry; iCountARY++)
In addition to the above, you are only allocating "Max", but you are allowing the subscript to exceed "Max" since you compare against "Entry".
Change to
Code:
Temp = new POINT[Entry];
Mike
-
Re: Why does this code leak memory?
Thanks everyone! I'll just use vectors, it looks to be alot simpler because I have many dynamic arrays in my code and handling them has been a bit of a chore. Thanks again everyone!
-
Re: [RESOLVED] Why does this code leak memory?
It is better to properly learn to use new[]/delete[] properly than to live on the crutch of std::vector though.
std::vector requires the use of the STL new operator, which is defined to throw an exception on failure. It include <new> which defines it to throw a std::bad_alloc, STL expects this.
You cannot override it, if you try - you will get an error because it is not defined to be the same as the first declaration. (That of which is in <new>)
And exceptions are a lot of overhead, for the simple fact they are no good for library code based on the simple fact people can use that code 10 function calls deep, and have to throw it all the way to the GUI. This is not acceptable! (Or they think they have to)
Don't just tell someone to use std::vector, you can recommend it - however, if you are going to tell someone the advantages of something, if there are big performance and design issues (there are big ones with STL people do not admit) then they should be aware of that as well.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
It is better to properly learn to use new[]/delete[] properly than to live on the crutch of std::vector though.
That "crutch" you speak of is responsible for less bugs and faster development time.
Why do you call it a "crutch"? It sounds like you're calling programmers who use std::vector and other containers novices because we don't know how to use new[]/delete[]. Myself and others here that use vector already know how to use new[]/delete[]. Even the OP knew how to use new/delete, so it isn't a matter of not knowing how to use it.
Heck, wherever new[] is called, a corresponding delete[] on the pointer value returned must be done. Big deal. But the problem is that these operations are done in pairs, and each one of in that pair could be in who knows what part of the program, thereby causing a logical disconnect in the code between the new/delete calls. To try and make this connection requires coding that does all sorts of contortions, with no real confidence that the code is really sound for all conditions. I've seen it myself, and it isn't pretty.
The problem is that C++ programmers are tired of maintaining code that does the new[]/delete[] gymnastics that the OP posted (i.e. logical spaghetti), and many code reviewers would reject the code that was posted because of this. I personally had to debug code that looked like the OP's (where to delete, when to new, who's responsible for what, what if this happened, etc.). In short, a maintenance nightmare -- all of the code I had to debug could have been replaced with a vector and resize().
Quote:
std::vector requires the use of the STL new operator, which is defined to throw an exception on failure. It include <new> which defines it to throw a std::bad_alloc, STL expects this.
You cannot override it,
Yes you can, by creating your own allocator. That's what the second template parameter is there for. Anyway, take 1,000 experienced C++ programmers -- how many of them really look to this as a problem?
Quote:
Don't just tell someone to use std::vector,
Too late. The OP is now using vector, and seems to be making progress. Anyway, it's a C++ forum, and I gave a C++ answer.
Quote:
Also, you can recommend it - however, if you are going to tell someone the advantages of something, if there are big performance and design issues (there are big ones with STL people do not admit)
And what are these issues with vector, and who are these people that you speak of? I don't know of any issues, which is why I need not mention them. You mention some pathological case concerning some GUI, but anyone can come up with the odd case for anything to do with the language that >99% of programmers will not care about.
Also, I've already mentioned that manually using straight new[]/delete[], especially if the pointer variable is a member of a class (similar to the OP's code), is demonstrably slower than just using a vector as a member and using resize() when appropriate.
Given 1,000 programmers, will the vast majority choose more speed, quicker development time, and easy maintenance, or be concerned by operator new throwing an exception (which is rare anyway), thereby coding a nightmare that would scare off any programmer that may need to maintain the code and also risk slower execution (and more bugs)?
If the OP were using vector, more than likely he never would have posted anything on CodeGuru, and instead got his real work done.
http://www.parashift.com/c++-faq-lite/containers.html
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
Some source from a rolled class. You want to see why I do not use vector? Have fun. Let's see vector do something like this. Are you going to implement using a vector as the underlying container for a ListBox? Hah...
This is the real world buddy. Check out where it says RaiseContentChanged().
Code:
bool ListItemArray::SetCapacity(unsigned long dwCapacity,bool bKeepData)
{
if (bKeepData == false)
{
Clear();
}
if (dwCapacity > m_dwCapacity)
{
ListItem ** ppNewData = new ListItem*[dwCapacity];
if (ppNewData == 0)
{
return false;
}
if (m_dwCount != 0)
{
ListItem ** ppDest = ppNewData;
ListItem ** ppSource = m_ppListItems;
ListItem ** ppEnd = ppSource + m_dwCount;
while (ppSource != ppEnd)
{
*ppDest++ = *ppSource++;
}
}
if (m_ppListItems != 0)
{
delete [] m_ppListItems;
}
m_ppListItems = ppNewData;
m_dwCapacity = dwCapacity;
}
return true;
}
bool ListItemArray::SetLength(unsigned long dwLength)
{
if (dwLength > m_dwCount)
{
if (dwLength > m_dwCapacity)
{
const unsigned long dwAlloc = DetermineAlloc(dwLength);
if (!SetCapacity(dwAlloc,true))
{
return false;
}
}
ListItem ** ppStart = &m_ppListItems[m_dwCount];
ListItem ** ppEnd = &m_ppListItems[dwLength];
while (ppStart != ppEnd)
{
*ppStart = new ListItem;
++ppStart;
}
m_dwCount = dwLength;
if (m_bUpdating == false)
{
RaiseContentChanged();
}
}
else if (dwLength < m_dwCount)
{
for (; m_dwCount != dwLength; --m_dwCount)
{
delete m_ppListItems[m_dwCount - 1];
m_ppListItems[m_dwCount - 1] = 0;
}
if (m_bUpdating == false)
{
RaiseContentChanged();
}
}
return true;
}
void ListItemArray::Clear()
{
if (m_dwCount != 0)
{
ListItem ** ppStart = m_ppListItems;
ListItem ** ppEnd = ppStart + m_dwCount;
while (ppStart != ppEnd)
{
delete *ppStart;
*ppStart = 0;
++ppStart;
}
m_dwCount = 0;
if (m_bUpdating == false)
{
RaiseContentChanged();
}
}
}
bool ListItemArray::Add(ListItem * pItem)
{
if (pItem != 0)
{
const unsigned long dwNewLength = m_dwCount + 1;
if (dwNewLength > m_dwCapacity)
{
const unsigned long dwAlloc = DetermineAlloc(dwNewLength);
if (!SetCapacity(dwAlloc,true))
{
return false;
}
}
m_ppListItems[m_dwCount] = pItem;
++m_dwCount;
if (m_bUpdating == false)
{
RaiseContentChanged();
}
return true;
}
return false;
}
bool ListItemArray::AddString(const AnsiString & str)
{
ListItem * pItem = new ListItem;
if (pItem != 0)
{
pItem->SetText(str);
const bool bCheck = Add(pItem);
if (bCheck == false)
{
delete pItem;
}
return bCheck;
}
return false;
}
void ListItemArray::RaiseContentChanged()
{
ListItemArrayEventArgs * pArgs = new ListItemArrayEventArgs;
pArgs->Sender = this;
pArgs->Count = m_dwCount;
ContentChanged.Raise(pArgs);
delete pArgs;
}
unsigned long __stdcall ListItemArray::DetermineAlloc(unsigned long dwCount)
{
unsigned long dwBlockCount = dwCount / BlockSize;
if (dwBlockCount == 0 || (dwCount % BlockSize) != 0)
{
++dwBlockCount;
}
return dwBlockCount * BlockSize;
}
Do you understand WHY though your statement about std::vector is that's its faster? It's only because of iterator access (which for most types is just a pointer). (And it's method of not destroying objects under reallocation)
If someone uses pointer access like I do inside my newer classes, my classes run circles around vector.
Consider that fact most people use for loops, this type of loop guarantees atleast an implicit mulitplication from the integer iterator to get the correct offset. This is very slow compared to direct pointer dereference.
In the latter, you have one multiplication of this, and then all increment/decrements. That is the only thing that makes vector fast.
So you referring to a standard paper of sorts explains nothing. The actual binary code generated from a certain algorithm dictates how fast it is.
Pic Link
So you stand by the standard just because it's standard, (however, going nowhere) - get your heads out of your arse and wake up.
-
Re: [RESOLVED] Why does this code leak memory?
Out of curiosity, but wouldn't it have been simpler to write this:
Code:
void ListItemArray::RaiseContentChanged()
{
ListItemArrayEventArgs * pArgs = new ListItemArrayEventArgs;
pArgs->Sender = this;
pArgs->Count = m_dwCount;
ContentChanged.Raise(pArgs);
delete pArgs;
}
as this?
Code:
void ListItemArray::RaiseContentChanged()
{
ListItemArrayEventArgs pArgs;
pArgs.Sender = this;
pArgs.Count = m_dwCount;
ContentChanged.Raise(&pArgs);
}
Maybe I am missing something really obvious and serious, but it also seems to me that a vector<ListItem*> or vector<tr1::shared_ptr<ListItem> > boost ptr_vector<ListItem> could indeed be used as the underlying container. I do not understand: what makes RaiseContentChanged() so special?
-
Re: [RESOLVED] Why does this code leak memory?
As for the dynamic issue... Stack variables cannot be passed to another thread. If the handlers invoked in this thread, set an event handle (synchro event) and pass the args to the new thread, well... It has to be on the heap now doesn't it?
As for RaiseContentChanged(), you raise an event to the listbox so it can recalculate it's draw data and repaint if it's not 'updating'.
Or whatever wants notified when the contents of the container change. If you do this with a wrapper of std::vector, then you defeat the purpose that you are trying to portray about vector and it's performance.
So it is a fair comparison. Sure, RaiseContentChanged() adds overhead to the container itself, but it does so in a way that exposes FUNCTIONALITY... In which a wrapper of a vector does not to the vector itself.
Believe me my friends... STL touched on this base with function objects... It didn't go far enough. And it has it's downfalls (STL).
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
Some source from a rolled class.
Pathological case.
Quote:
You want to see why I do not use vector?
So just dismiss anything I stated to you about 99% of programmers not having any issues with using vector appropriately.
Quote:
Have fun. Let's see vector do something like this. Are you going to implement using a vector as the underlying container for a ListBox? Hah...
...
This is the real world buddy.
....
get your heads out of your arse and wake up
This isn't the World Wrestling Entertainment. Please be more mature in your responses. The last response could get you banned here. Also, it looks like you are trying to imply that vector was supposed to be some uber component that didn't work for you. It's purpose is as a generic dynamic array -- that's it.
Also, I deal with the real world, as all of my programs are used by thousands of persons. I don't know who uses your programs or code.
Also, the OP has no problems with using vector, as do many thousands of programmers.
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
As for the dynamic issue... Stack variables cannot be passed to another thread. If the handlers invoked in this thread, set an event handle (synchro event) and pass the args to the new thread, well... It has to be on the heap now doesn't it?
I have no real experience in threads (hobbyist programmer with specific hobbies, you understand), but that sounds like a good explanation.
Quote:
As for RaiseContentChanged(), you raise an event to the listbox so it can recalculate it's draw data and repaint if it's not 'updating'.
Or whatever wants notified when the contents of the container change. If you do this with a wrapper of std::vector, then you defeat the purpose that you are trying to portray about vector and it's performance.
Ah, so it is not that you are arguing that it is impossible, but that it is technically inferior performance-wise to wrap a vector in your case?
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by laserlight
Ah, so it is not that you are arguing that it is impossible, but that it is technically inferior performance-wise to wrap a vector in your case?
Which means we would have timing tests, comparisons, and actual code showing the vector implementation, all without the vitriol and bravado.
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by laserlight
I have no real experience in threads (hobbyist programmer with specific hobbies, you understand), but that sounds like a good explanation.
Ah, so it is not that you are arguing that it is impossible, but that it is technically inferior performance-wise to wrap a vector in your case?
Exactly. In C++, just about anything is possible. Even manual VTable implementation.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by Paul McKenzie
Which means we would have timing tests, comparisons, and actual code showing the vector implementation, all without the vitriol and bravado.
Regards,
Paul McKenzie
And it sounds like someone is getting too worked up on holding onto a bunch of projects that use STL, and is less willing to let go of archaic and to go with new technological ideas.
Remember, it's the CRT and STL and their caveats that have prevented standard C++ from going further than it has. The only way to circumvent them, is to let go of them. I am not trying to start a fuss, there are alot of projects that depend on STL, and because alot of people use them, they get 'attached' to the code they write that way.
Unfortunately for those people... Sticking with STL means that you will never have a full fledged library that you want, due to the very fact that the entire industry only agrees on one thing when it comes to the standard... and that's to disagree.
You cannot build a framework around STL and expect everyone to use it, because there are alot of programmers out there that hate STL, myself being one of them.
IMHO, I see it this way. There are 3 paths to take... You can stick with the 'standard' and open source movements, and lean towards Apple as well... Or you can embrace MS and .NET ...
The first two I view as stupid. And yes, I will take what Paul said about 99% of other people and discard it. (and it's not 99%, it's actually quite a bit lower than that) not having a problem with vector. Yeah they do, it's a template. Templates have overheads and things about them that every programmer hates. Paul, please use more rational thinking when you make a statement. This is the very proof you are not thinking clearly, and are running on emotion because of your attachment to vector, and how much you take it seriously trying to 'run with the standard'. I know that too, I see how much you refer to it. So calm down, and take a chill pill.
Why do I discard that? It doesn't apply to ME, that's why.
My situation is what I believe to be the smart one. Take the road of rolling my own framework that takes the ideas of old and new and goes down the right path. The standard people 'hate' those that reinvent (not really, generally though, it's a true statement)... And MS hates those that do something better than them (those that either go against their ideas, or those that implement them better than they do)...
I believe my path is the right one, due to the fact it's idealogy takes the route that it does not make a choice to go with either side.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
And it sounds like someone is getting too worked up on holding onto a bunch of projects that use STL, and is less willing to let go of archaic and to go with new technological ideas.
What are these "new technological ideas"?
Quote:
Remember, it's the CRT and STL and their caveats that have prevented standard C++ from going further than it has.
Could you elaborate? A comprehensive online essay would make for a good line, if feasible, especially if it is by a noted expert in C++.
Quote:
Unfortunately for those people... Sticking with STL means that you will never have a full fledged library that you want, due to the very fact that the entire industry only agrees on one thing when it comes to the standard... and that's to disagree.
I do not understand your reasoning. If people cannot agree when it comes to the standard, why would they agree to anything else?
Quote:
You cannot build a framework around STL and expect everyone to use it, because there are alot of programmers out there that hate STL, myself being one of them.
But you cannot build any framework and expect everyone to use it. Such an expectation is surely unrealistic. If you cater to one crowd, you will surely step on the toes of another crowd.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by laserlight
What are these "new technological ideas"?
Could you elaborate? A comprehensive online essay would make for a good line, if feasible, especially if it is by a noted expert in C++.
I do not understand your reasoning. If people cannot agree when it comes to the standard, why would they agree to anything else?
But you cannot build any framework and expect everyone to use it. Such an expectation is surely unrealistic. If you cater to one crowd, you will surely step on the toes of another crowd.
Okay, this would probably be a good idea to go to another thread on this issue. However, I would like to state take a look at the edit of my last post.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
You cannot build a framework around STL and expect everyone to use it,
Everyone means 100%. Where did I ever state anything is 100%? Not everyone will use anything in C++. Not everone will use facets. Not everone will use iostreams. Not everyone will use <complex>. Not everyone will use .. etc. .etc...
The common denominator when asked a question concerning C++ is to mention the standard components first and maybe boost, i.e. components that have been tested fully in the user community and are standard or as close to standard as possible. The reason is that the poster asking the question has plenty of resources available to them when it comes time to use the suggestions given to them.
Niche coding and pathological cases may be used when asked questions pertaining to them. If the OP mentioned "vector has these issues, so I decided to try this instead", then that is a different story. But just cluttering a thread with coding that no one really knows what it does, how it works, who coded it, who tested it, whether any claims are true, etc. is really non-productive when a straightforward answer is warranted.
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
Paul, please use more rational thinking when you make a statement. This is the very proof you are not thinking clearly, and are running on emotion because of your attachment to vector, and how much you take it seriously trying to 'run with the standard'. I know that too, I see how much you refer to it. So calm down, and take a chill pill.
All of this talk, and all I mentioned to the OP was to use std::vector?
It seems you are projecting your own emotions onto me and you are the one that needs calming down.
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
No, my original intent was actually myself getting tired of seeing what I see as inferior (as the fact, it would be possible through specialization as well - however, that's rolling your own at that point) - meaning std::vector, being ALWAYS the solution put out there.
In case you do not know Paul, I do like stringstreams. And any chance that STL is a possibility of use, I do encourage the use of stringstreams. However, NOT vector.
And actually Paul, this thread started about the possibility of a memory leak, and I brought up the point it would be better to learn the proper use of new/delete.
This isn't an issue of std::vector, I was the one who initially made the statement referring to what the user asked. The importance of new/delete is more important than knowing how to use std::vector.
Rolling your own class allows you to learn the caveats and generality of what it takes to build what std::vector does, that knowledge is more important that using std::vector. std::vector is just a class, the knowledge allows you to use what you learn in building others things than just a vector replacement.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
No, my original intent was actually myself getting tired of seeing what I see as inferior
That is your opinion. For most purposes, vector is far more preferable than the new/delete coupled with extreme logic to get the two working together.
Quote:
meaning std::vector, being ALWAYS the solution put out there.
For the OP's purposes, it was a solution. Read his last response. Anyway, there is nothing that is always a solution.
Quote:
In case you do not know Paul, I do like stringstreams. And any chance that STL is a possibility of use, I do encourage the use of stringstreams.
And others do not like streams. Also, iostreams is not STL. The streams have been part of C++ before STL was even devised.
Quote:
However, NOT vector.
That again is your opinion. I would rather value the opinion of noted C++ experts, coupled with my own experience than something out of the blue with no documentation, peer review, testing, etc.
Quote:
And actually Paul, this thread started about the possibility of a memory leak, and I brought up the point it would be better to learn the proper use of new/delete.
And I already stated that the OP knows how to use new and delete -- did you read the original code? The OP carefully tried to pair each new[] with delete[]. This is all that is required to know how to use it. The problem is that in the pairing of these two calls, the logic of the code gets in the way, making new and delete usage prohibitive. I am repeating myself -- did you not read my earlier post?
Quote:
This isn't an issue of std::vector, I was the one who initially made the statement referring to what the user asked. The importance of new/delete
The importance is when to use it, and when it is counter-productive to use it.
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
It's all in the eyes of the beholder as you say. I am saying that is not the case. Simply put, by factual data of the thread in hand.
My point is that he was worried about a memory leak, when actuality it was a different problem all together. So the problem was not that he ' wasn't using vector ', the problem was that he was misusing new/delete.
In that sense, proposing std::vector is NOT the solution Paul.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
In that sense, proposing std::vector is NOT the solution Paul.
I will judge by the OP's satisfaction as to what I proposed is a solution or not.
As to giving an answer based on a question: I would bet that a good percentage, maybe close to half of responses on CodeGuru to any poster's question does not look like the OP's original code or design.
If you saw someone use a screwdriver as a hammer, do you give him a hammer, or do you show him how to hit a nail with a screwdriver properly? I'll do the former.
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
James,
So are you going to post enough of your listbox data class so any real determination
can be made about relative performance if internally implemented with a vector?
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by souldog
James,
So are you going to post enough of your listbox data class so any real determination
can be made about relative performance if internally implemented with a vector?
Okay... I am not going to post the source code... However, have you seen a ListBox that can handle 9,000,000 strings or more? RAM seems to be the only limitation for mine.
Screen Capture Video Gotta love 7-zip on compressing MS-CRAM (MS Video 1) screen captures with no audio. :D
And technically, it's not holding 9,000,000 strings. It's holding 9,000,000 CListItem objects via pointers.
.NET? :thumbd: You're not going to see a .NET listbox do that.
Native? :thumb: ;)
Give me 16 gigabytes (this one has 1 gigabyte). 16 x 9 = 144.
I wonder what this code would be like on a 64-bit machine with 16 gigabytes. Could it handle 144,000,000 CListItem objects? :eek:
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
Okay... I am not going to post the source code...
So I guess the answer to souldog's question is "no, I have no such code to show you that compares an implementation using vector with what I've done".
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
As for the dynamic issue... Stack variables cannot be passed to another thread.
Of course they can. It is REFERENCES (or pointers) to stack variables that will cause issues in many cases (where the life of the thread is longer than the life of the stack frame).
ps: I completely agree with Paul. During a coding review, I will outright reject any custom memory management UNLESS it can be documented that a standard implementation (such as an STL container) can not do the job. Developers who submit such items more than once or twice, find themselves looking for new jobs.
-
Re: [RESOLVED] Why does this code leak memory?
It is perfectly alright to be proud of one's work, James, but your statements in this thread seem a bit over-the-top.
You need to concede on at least the following points:
1) You are not privy to any information that the C++ community is unaware of in their tendency to recommend std::vector.
2) A vector is not a ListBox. Stop comparing them.
I'd elaborate, but I think it's sufficient to say I agree with everything Paul's said so far.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by TheCPUWizard
Of course they can. It is REFERENCES (or pointers) to stack variables that will cause issues in many cases (where the life of the thread is longer than the life of the stack frame).
ps: I completely agree with Paul. During a coding review, I will outright reject any custom memory management UNLESS it can be documented that a standard implementation (such as an STL container) can not do the job. Developers who submit such items more than once or twice, find themselves looking for new jobs.
First of all... Each thread get's it's OWN STACK, therefore you cannot pass sometime that has anything in that thread's stack to another. Passing by value is different (a 32bit integer passed by value reinterpret_cast to the void pointer, that's different - Stop being a smart arse).
As far as comparing... Do you see a LISTBOX that is implemented using std::vector that is capable of that? Alright, case closed. I am not one to show source code out of my closed source projects that I am going to use in projects that I will make money on. Alright, with that said...
It will even handle 10,000,000 + after I let it swap out all the memory to file, and then back into memory after it defragmented it. (took forever to do that, but afterwards... It ran fine with 10,250,000 items in it)
The point is, the ListBox does nothing except use the CListItem container that holds the items. The container stores those 10,250,000 items. (And of course it draws them based on scroll position/etc...)
I do not believe that the default allocator for std::vector will allow you to allocate that many objects. 10,250,000 * sizeof(pointer) + 10,250,000 * sizeof(CListItem) + 10,250,000 * 16 (block size of CAnsiString member of CListItem)
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
As far as comparing... Do you see a LISTBOX that is implemented using std::vector that is capable of that?
That is your job -- tell us where to find a listbox that is implemented using a vector so that we can make a comparison. No one is going to go around searching source code for such an animal -- you provide it to us and we'll comment on it.
Quote:
Alright, case closed. I am not one to show source code out of my closed source projects
Do you have a vector class or not? If you do, then why not show us a simple program showing your vector implementation with std::vector? Why do you need to show us a list box program to compare timings between two components?
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
First of all... Each thread get's it's OWN STACK, therefore you cannot pass sometime that has anything in that thread's stack to another. Passing by value is different (a 32bit integer passed by value reinterpret_cast to the void pointer, that's different - Stop being a smart arse).
Not being a "smart args".
1) Passing be Value an item on one threads stack to another thread (fo storage on it's stack) is perfrectly reasonable. There is no need for using the heap.
2) Even passing be reference, can make sense. Consider the following psuedo code:
Code:
void Func()
{
SomeObject x[100]; // local on the heap
foreach (x)
StartThread(&x);
WaitForAllThreads();
}
Quote:
Originally Posted by JamesSchumacher
As far as comparing... Do you see a LISTBOX that is implemented using std::vector that is capable of that?
Nor should thre be any for two reasons:
1) a ListBox is meant to be VIEWABLE. The average UI Guidelines indicate that a ListBOx should never have more than 25-100 items (most on the lower end). Even wasing a few hundred bytes per item should not have an impact on modern (non-mobile/non-embedded) systems.
2) Any Linear access mechanism with 10M items is likely to be a poor design. Can you provide a real world sample where this would be the implementation of choice?
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by TheCPUWizard
Not being a "smart args".
1) Passing be Value an item on one threads stack to another thread (fo storage on it's stack) is perfrectly reasonable. There is no need for using the heap.
2) Even passing be reference, can make sense. Consider the following psuedo code:
Code:
void Func()
{
SomeObject x[100]; // local on the heap
foreach (x)
StartThread(&x);
WaitForAllThreads();
}
Nor should thre be any for two reasons:
1) a ListBox is meant to be VIEWABLE. The average UI Guidelines indicate that a ListBOx should never have more than 25-100 items (most on the lower end). Even wasing a few hundred bytes per item should not have an impact on modern (non-mobile/non-embedded) systems.
2) Any Linear access mechanism with 10M items is likely to be a poor design. Can you provide a real world sample where this would be the implementation of choice?
In response to #2, Linear access is actually the best design (this applies to #1 as well), simply for the fact that based on scroll position, one can determine the first item to be drawn simply by the division of the current scroll position (max defined as item count * item height - window height or zero in case of item count * item height < window height) by the item height.
Item height = 15
10000 items with window height of 200.
150000 - 200 is max scroll position.
149800 / 15 = 9986
I wonder how I determine where to begin? This is an example if scroll position is the maximum. Drawing is independent of how many items are in the listbox. If it was the 64-bit world, and it was using 64-bit scroll data, as far as the window is concerned, it wouldn't matter if it had 1,000,000,000 items in it. (The actual draw code that is)
However, for selected items, a linear buffer is NOT the best choice. In fact, I am optimizing the code to use a CListItemNode which takes a pointer to a CListItem in the constructor, in this manner, I do not need to know how many selected items there are in the case of multi-select.
And actually, a few hundred bytes per item does make a difference. 20 bytes (which I just dropped off, removing color data storage from the list items themselves) at 10,000,000 items saved 200,000,000 bytes. What is 200,000,000 / 1,048,576?
Anyways... If you want to test out my control in a test app (no source or development access, just the same test program I used. Check it out)
That can be found here in the self extractor along with my core *.DLL it uses.
And note that when you click the button, it adds 250,000 random 5 character strings into the list box. (That takes a bit due to generating the 250,000 strings)
So what point does this have? It goes back to the default allocator for std::vector as well. :cool:
If you use a specialized vector, that's not std::vector is it? That's your own rolled specialization. So why use vector at all?
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
In response to #2, Linear access is actually the best design (this applies to #1 as well), simply for the fact that based on scroll position, one can determine the first item to be drawn simply by the division of the current scroll position (max defined as item count * item height - window height or zero in case of item count * item height < window height) by the item height.
That wasn't the point. The point is that:
- No one wants to deal with a list that has substantially more than 25-100 items.
- If they do, it had BETTER have search capabilities
And here we are still talking about list boxes. What about the container it uses, that you claim is comparable but superior to vector (i.e. a simple, contiguous, random access container)? Or have you built memory management right into your list box?
Quote:
Originally Posted by JamesSchumacher
So what point does this have? It goes back to the default allocator for std::vector as well. :cool:
If you use a specialized vector, that's not std::vector is it? That's your own rolled specialization. So why use vector at all?
The default allocator in STL is essentially new([])/delete([]), and vectors are almost always mentioned in the context of replacing new[]/delete[]. What could possibly be your point, then?
Secondly, there is a huge difference between implementing a custom allocator and re-implementing vector. Not to mention that a custom allocator, if made properly, could be used by any type of container.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by Hermit
That wasn't the point. The point is that:
- No one wants to deal with a list that has substantially more than 25-100 items.
- If they do, it had BETTER have search capabilities
And here we are still talking about list boxes. What about the container it uses, that you claim is comparable but superior to vector (i.e. a simple, contiguous, random access container)? Or have you built memory management right into your list box?
The default allocator in STL is essentially new([])/delete([]), and vectors are almost always mentioned in the context of replacing new[]/delete[]. What could possibly be your point, then?
Secondly, there is a huge difference between implementing a custom allocator and re-implementing vector. Not to mention that a custom allocator, if made properly, could be used by any type of container.
Search capabilities can be added to my CListBox with ease. You can access the container from the listbox. How do you think I am adding 250,000 strings when the button is clicked? Which means the search could be implemented in an interface member of the CListBox, or as an outside function working on the collection itself. This is not something that would be hard to accomplish. In fact, it's ironic that I was saying this myself just last night.
With a search function in a list box (you are right, I said that last night myself) it is even viable to search a list of over 10,000,000 names for example... And it would be single selection mode then, in which the data of that person would be displayed in other controls.
And it's not like you (whoever client is) are limited to a CListItem... It stores pointers for a reason... And the container itself has SetCapacity() AND SetBlockSize() members for dealing with reallocations. I have thought this out very well.
The CListBox::AddString just creates a CListItem to add to it, and when it does calls CListBox::AddItem.
And now maybe you will realize you are talking to an IT professional. (Even though I have had no paying experience.)
Or maybe you just want to hear more about how important these things are and that you want to hear more technical details. Let's just put it this way, as far as technical issues goes...
This is my REALITY TEST, to wake up Microsoft and others out there, that .NET is a mistake... And bloated code just stacked on top of other bloated code (a lot of open source projects, and Apple's stuff as well - not open source) is not a viable solution either.
This is what I meant by 'real world' Paul. Not knowing the innards of what makes STL successful in what it is good at (speed in general, because of iterators), handicaps you (people in general) Paul.
Now understand why I threw my 2 cents in.
There, the ISSUE has been closed.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
Search capabilities can be added to my CListBox with ease. You can access the container from the listbox. How do you think I am adding 250,000 strings when the button is clicked? Which means the search could be implemented in an interface member of the CListBox, or as an outside function working on the collection itself. This is not something that would be hard to accomplish. In fact, it's ironic that I was saying this myself just last night.
That would be a linear (read: slow) search.
Quote:
Originally Posted by JamesSchumacher
And the container itself has a SetCapacity() AND SetBlockSize() members for dealing with reallocations. I have thought this out very well.
The fact that you refer to a method called SetBlockSize makes me suspect you are not talking about a strictly contigous container. Why, then, are you comparing to std::vector and not std::deque?
STL was also well thought out, you know.
Quote:
Originally Posted by JamesSchumacher
And now maybe you will realize you are talking to an IT professional.
A particularly proud one, too!
Quote:
Originally Posted by JamesSchumacher
There, the ISSUE has been closed.
Fine, I'm tired of asking questions that don't get answered.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by Hermit
That would be a linear (read: slow) search.
The fact that you refer to a method called SetBlockSize makes me suspect you are not talking about a strictly contigous container. Why, then, are you comparing to std::vector and not std::deque?
STL was also well thought out, you know.
A particularly proud one, too!
Fine, I'm tired of asking questions that don't get answered.
Yes, linear searches are slow. However, when dealing with 10,000,000 + items are you going to be able to handle that in some other type of container? The linear storage model is hard pressed at the time of 10,000,000 items. This is where you make a decision on tradeoff. And linear searches can be optimized to make sure you do not do things like this.
Code:
for (unsigned long i = 0; i < m_dwItemCount; ++i)
{
// Loading index from member variable that is not constant
// loading base pointer that is not constant itself
if (m_pDataArray[ i ].FindSubString("StringToMatch") != -1)
{
// ... Do whatever! Above is doing load from member variables
// plus implied multiplication by index and sizeof. This is SLOW!
}
}
// BETTER alternative. See below.
CListItem ** ppItems = m_pDataArray;
CListItem ** ppEnd = m_pDataArray + m_dwItemCount;
while (ppItems != ppEnd)
{
// Take this concept and apply it to the search function itself
// These concepts STACK
++ppItems;
}
This is the example of the idea of iterator approach. (Taken from what STL uses, iterators - which are in most cases pointers or pointer wrappers that use inlines)
You would be surprised by how much this makes a difference. Let me go get the link to my posted Gradient code and you can read what I commented in there.
This Thread, my last post in it Check out the main.cpp file.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
Yes, linear searches are slow. However, when dealing with 10,000,000 + items are you going to be able to handle that in some other type of container? The linear storage model is hard pressed at the time of 10,000,000 items. This is where you make a decision on tradeoff. And linear searches can be optimized to make sure you do not do things like this.
Code:
for (unsigned long i = 0; i < m_dwItemCount; ++i)
{
// Loading index from member variable that is not constant
// loading base pointer that is not constant itself
if (m_pDataArray[ i ].FindSubString("StringToMatch") != -1)
{
// ... Do whatever! Above is doing load from member variables
// plus implied multiplication by index and sizeof. This is SLOW!
}
}
// BETTER alternative. See below.
CListItem ** ppItems = m_pDataArray;
CListItem ** ppEnd = m_pDataArray + m_dwItemCount;
while (ppItems != ppEnd)
{
// Take this concept and apply it to the search function itself
// These concepts STACK
++ppItems;
}
This is the example of the idea of iterator approach. (Taken from what STL uses, iterators - which are in most cases pointers or pointer wrappers that use inlines)
You would be surprised by how much this makes a difference. Let me go get the link to my posted Gradient code and you can read what I commented in there.
This Thread, my last post in it Check out the main.cpp file.
A Hash table is not viable... There is no guarantee that with that many items that they will not collide. (And the size of the hash table! Imagine that >.< why not just use an array?) To make that work, we would have to store a collection of containers, in which you were looking for values that hash to the same value, and then you search those for an exact match. THAT is not a half bad idea, IF only used in CACHING previous results.
If you have that many items, that any set of them can be displayed at any point (random access), are you going to use a sorted tree? I can see how this would work, if it has references to load data from file and not keep everything in memory. However, why use a tree structure for a LIST?
A hash table COULD be used, based on previous searches. Although, any other data structure could be used as well. However, that is in ADDITION to the array style linear data. In any case, this is a case of CACHING the search matches. (I could write a search engine that would be as fast if not faster than google's. LMAO - First search requires looking for the file directly and see if it contains the data. This is done by having a 'crawl' to have a cache to search. This is where keywords come into play. Any search string hashes to point at a container of possible matches, and then returns them in the priority order.)
The point of the matter is this... If I can make it work in what seems the most extreme way to do it (keep it all in memory), and optimized so it works very well... I can take the same approach techniques and apply it to the other design models and make it even BETTER in that approach than someone else doing it that way. Why? I am aware of what CAN BE OPTIMIZED and how it can be, and why.
A listbox can be filled with any amount of data. Random not cached data even. (Hence the random 5 character strings to make a point)
To be able to have the extreme searching results of search engines (especially google) people do not realize that their machines do not 'SEARCH THE WEB' every time for all the possible matches. No... They check their cache, which is NOT a valid representation of the entire web, let alone of a site at any particular time.
The point here is that, to make a decision based off of all the variables, and decide what priorities must come first.
And actually, if you think about it. Linear searches are actually the fastest in some way, due to the fact you do not have to preprocess the data in order to find a match. (Exception: If an exact match was found earlier, in the case of compression, and caching those data matches (don't care where they are) in a hash table... Repetition finding) However... Linear is still considered fastest due to the fact that you cannot put together that data you are processing quicker in a way other than linear. The output is linear, therefore, any data structure dealing with it outputs the data linear. Therefore, there is no way to access it any faster than in linear form.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
This is what I meant by 'real world' Paul. Not knowing the innards of what makes STL successful in what it is good at (speed in general, because of iterators), handicaps you (people in general) Paul.
Yes, after 10 years of using it, and actually being taught by the person who created STL, I don't know STL. Shame on me.
So when are we going to get a real code to compare? That's all I care to see or be interested in.
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by Paul McKenzie
Yes, after 10 years of using it, and actually being taught by the person who created STL, I don't know STL. Shame on me.
So when are we going to get a real code to compare? That's all I care to see or be interested in.
Regards,
Paul McKenzie
Paul,
I would have thought by now you would have figured it out? :eek: ;)
Seriously James just seems to like to debate his point of view, despite the fact it contradicts just about every accepted premise in the industry.
Unfortunately too many developers focus on issues which are horribly outdated. Machines have grown tremendously (I remember working on a system that ran a School District with over 10,000 students...All on 16K of memory and the LARGEST disk was 128KW with a clock speed of 500Khz) is nearly every aspect.
The TRUE cost of software these days is both in time to market and post release support. Custom implementations (when a standard implementation will accomplish the job) can increase this cost significantly.
I hate to think what the impact of James' approach is to target companies who must support the software after he (and any staff trained directly by him) is gone. [Actually I book 3-5 contracts a year directly based on this type of situation].
He continualy bemoans modern (even 10+ year old) development platforms, and techniques. My professional experience has been that my current .Net applications actually perform faster [on a 1-2 year old machine], than many "Highly optimized" programs written in C++ did 5-7 years ago (again on machines that were 1-2 years old at the time the application was deployed). Additionally the current releases of these applications cost less to develop (even though hourly development rates have tripled), and technical support (including defect management) is less than 25% of what it was (based on ratio of new development hours to maintenance hours).
-
Re: [RESOLVED] Why does this code leak memory?
I thought that my descriptions of what I do was enough? I thought that I made myself perfectly clear in what I end up doing and how I write my code.
Here is the left button down handler code.
CListBox::OnLeftButtonDown
I will not post any more. The only reason that I made the decision to post it, is it may in fact help show somewhat how it is constructed, and for someone to better judge the quality of the control.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by TheCPUWizard
Seriously James just seems to like to debate his point of view, despite the fact it contradicts just about every accepted premise in the industry.
Now quotes from this thread are now being used on another forum.
James, if you're going to use quotes by Hermit, myself, or CPUWizard on another forum, the proper decorum is to let us know beforehand, or at the very least, "ping" the people you are refering to on the other forums.
I don't want to see a list box class. I want to see two sets of source code, one using your vector, and another using std::vector, with timing tests. I don't do list boxes (and many in the non-Visual C++ forum don't deal with this), as most of my work deals with imaging and back end issues. So a list box class means nothing to me, as a vector of system information won't mean much to a GUI designer.
This is what is asked for by any C++ programmers when confronted with the claims that "my x is faster than you're 'x'). No one is saying you didn't do a lot of work, but up to this point, there is no definitive data as to your claims.
You're lucky this is CodeGuru and not one of the established C++ newsgroups such as comp.lang.c++ The people there wouldn't take it this far without seeing an actual self-made vector class.
For example, these people have worked hard on providing test cases, reasons, actual code, timing tests, etc. when comparing their implementation of the standard containers for gaming as opposed to using the generic containers (and a lot of the 'C' library routines).
http://www.open-std.org/jtc1/sc22/wg...007/n2271.html
If you're going to make claims, what you see at that link, or even a subset of what you see there, is what is expected as far as evidence is concerned.
Regards,
Paul McKenzie
-
Re: [RESOLVED] Why does this code leak memory?
You know this seems to be a very complicated way of reinventing the wheel.
The Tree and list common controls have offered the dispinfo callbacks which allows a user to back the control to a container.
These allow the controls to not store string data in the control, just like James is touting his approach to the listbox. These controls have had the dispinfo support since NT3.51 and Win95 (both released in 1995).
For a nice little sample, see the LVSelState.zip file in the reply to the 'How to implement a virtual listview' post.
This sample uses a std::list that is used to back a virtual list control. The sample by default fills the list with 60,000 virtual items. The user can go up to a million by changing a spin control.
This sample uses multithreading and creates a secondary thread that fills the std::list. It's done for fun and the std::list is made thread-safe with the help of a couple of simple sync wrapper classes.
Since everyone is displaying code, I figured I'd add some too. Here's an example of attaching the std::list to the virtual list control and 'populating' the control.
Code:
// Method: Populate
// Purpose: Called within the Retrieve button to add items to
// the list control.
// NOTE: It can be argued that this type of data manager class shouldn't
// include any UI interaction. As such I could have cycled through the
// CLVItemDataList in the dialog itself and populated the list control.
// However I chose not to because that would have mean I would have had
// to publicly expose the CLVItemDataList member and force users to
// lock the list in the dialog class.
HRESULT Populate( CListCtrl& ctlList )
{
HRESULT hr = S_OK;
// Add a wait cursor
CWaitCursor wait;
long lIndex = 0;
LVITEM lvitem = { 0 };
int iItem = 0, iActualItem = 0;
lvitem.mask = LVIF_TEXT | LVIF_IMAGE | LVIF_PARAM;
// Lock the List Control window so it doesn't look
// like crap while filling
ctlList.LockWindowUpdate();
// Lock the list
CAutoLockT< CLVItemDataList > lock( &m_LVItemDataList );
// Cycle through the list and add each item into the list control
for(CLVItemDataList::iterator it = m_LVItemDataList.begin();
it != m_LVItemDataList.end();
it++)
{
CLVItemData* pLVItemData = (*it);
lvitem.iItem = iItem;
lvitem.iSubItem = 0;
lvitem.lParam = reinterpret_cast<LPARAM>(pLVItemData);
// Tell the list control to become 'Virtual' and request display
// data from the lvitem.lParam.
// See CLVSelStateDlg::OnLvnGetdispinfoList for where this happens
lvitem.iImage = I_IMAGECALLBACK;
lvitem.pszText = LPSTR_TEXTCALLBACK;
lvitem.cchTextMax = MAX_LVITEMLEN;
// Insert the item
ctlList.InsertItem(&lvitem);
// Set the text for each column (all virtual callbacks)
ctlList.SetItemText(iItem, 0, LPSTR_TEXTCALLBACK);
ctlList.SetItemText(iItem, 1, LPSTR_TEXTCALLBACK);
ctlList.SetItemText(iItem, 2, LPSTR_TEXTCALLBACK);
iItem++;
// Check if user cancelled operation
if( WAIT_OBJECT_0 == WaitForSingleObject( GetShutdownEvent(), 0 ) )
{
return hr;
}
// Pump messages to allow cancel button click
// during control filling. Populating the control
// with thousands of records takes time, so we need
// to process messages to handle the cancel button
// and close messages
if(!ProcessMessages( ))
{
return hr;
}
// Restore the wait cursor
// (if you don't do this, the wait cursor may go
// away after processing messages)
wait.Restore();
}
// Unlock the List control window
ctlList.UnlockWindowUpdate();
return hr;
}
I could post the code that shows how the control displays its data when a dispinfo message is received, but I'm sure you've seen enough of my code for the day. If you are that interested, just follow the link.
Lastly, I make no claims about performance. Certainly the virtual list control is faster than it's non-virtual counter part. And it's fast enough to be used within windows (like the right pane of explorer). That's good enough for me.
-
Re: [RESOLVED] Why does this code leak memory?
I was aware of this. Just as I am aware that there is a mechanism for an edit control that allows you to allocate the memory it uses.
Anyways... My point here is this. If we need something minor, and that fits better than something has loads of overhead for the job at hand (vector does no matter how you slice it), and a basic collection class is not that hard to 'roll your own' if you know what you are doing.
We could all be doing this instead
Instead of arguing over little things.
-
Re: [RESOLVED] Why does this code leak memory?
Quote:
Originally Posted by JamesSchumacher
and a basic collection class is not that hard to 'roll your own' if you know what you are doing.
I can imagine handing off code to clients where I would have to justify hand written linked lists and other container classes. That isn't going to fly.