libpng error: Read Error

[Red Squirrel]

I'm using pngwriter (which I noticed has not been updated in ages... anything better?) and at random, when reading a png file, I get this, which causes the application to crash:

libpng error: Read Error

This is very random and due to lack of detail I think this will be a hard one. Is there any way I can troubleshoot this, or is there maybe a better png class I could use?

[ninja9578]

libpng is still the standard, you won't find anything better. How about you show us the code that's causing the error. My guess is that you are placing your read pointer in the wrong place.

[Red Squirrel]

Read pointer? I did not see anything in the docs about it. anyway, this is the code in question, it's quite long:

void ShardListEntry::UpdateInfo()
{
	string sname=UnFilterSQL(string(m_row["sname"]));
	
	Debug("Updating info...");

	//add stat entry:	
	
	string query="INSERT INTO stats(stdate,stisup,stshardid,stonline,stresponse,stavgresponse) VALUES(";	
	
	query+="'" + m_curdatestr + "',";
	query+="'" + string(m_sisup ? "10000" : "0") + "',";
	query+="'" + m_sid + "',";
	query+="'" + m_susers + "',";
	query+="'" + m_sresponse + "',";
	query+="'" + m_savgresponse + "'";
	
	query+=");";
	
	if(!m_con->DoQuery(query))
	{
		Debug("Stat add query error:" + query + "\n" + m_con->GetError(true));	
	}
	
	
	
	//calculate averages based on last 30 days:
	
	query="SELECT ";	
	query+="avg(stisup) as suptime, ";
	query+="avg(stonline) as savgusers, ";
	query+="max(stonline) as smaxusers, ";
	query+="avg(stresponse) as savgresponse, ";
	query+="count(stisup) as sdatasamples ";
	
	query+=" FROM stats WHERE stshardid='" + m_sid + "' AND stdate>(" + m_curdatestr + "-" +BitStream::Int2Str(MAX_STAT_KEEP_SECS)+") LIMIT 1;";
	
	if(!m_con->DoQuery(query))
	{
		Debug("AVG query error:" + query + "\n" + m_con->GetError(true));	
	}
	else
	{
		UseQueryResult res = m_con->GetResult();
		
		Row tmprow = res.fetch_row();		
		
		m_savgresponse=string(tmprow["savgresponse"]);
		m_savgusers=string(tmprow["savgusers"]);
		m_smaxusers=string(tmprow["smaxusers"]);
		m_suptime=string(tmprow["suptime"]);
		m_sdatasamples=string(tmprow["sdatasamples"]);
		
		res.fetch_row();//flusher
	}
	
	
	
	
	//check if this shard has been down too long.  If yes we set it as innactive:
	
	if((BitStream::Str2Int(m_slastonline)+604800)<BitStream::Str2Int(m_curdatestr))
	{
		Debug("Shard has been offline for too long, setting as innactive");		
		m_sofflineinactive="1";
		
		//decided not to send out emails anymore as the debug version might send out as well
		/*
		string emailbody;
		emailbody+="Dear " + m_susername + ",\n\n";
		emailbody+="We have detected that your shard has been offline for at least 7 days. ";
		emailbody+="In order to keep our listing as clean of innactive shards, it has been set as innactive. ";
		emailbody+="You may login at any time to reactivate it once the shard is back up, by going to this link:\n\n";
		emailbody+="http://www.uogateway.com/shard.php?act=edit\n\n";
		emailbody+="Thank you for your interest in UOGateway, we hope to see you active again!\n\n";
		emailbody+="Red Squirrel, UOGateway Owner";		
		
		EmailClient client;		
		client.SetFrom("noreply@uogateway.com");	
		client.SendTo(m_semail,"UOGateway Shared Listing Deactivated",emailbody);*/
	}
	
	
	//resolve IP so we can update it...
	
	BitStream bsip;
	ResolveHost(string(m_row["slogonserver"]),bsip);	
	string ipstr="";
	
	ipstr += rslib::Uint2Str(bsip.ReadInt8(0)) + ".";
	ipstr += rslib::Uint2Str(bsip.ReadInt8(8)) + ".";
	ipstr += rslib::Uint2Str(bsip.ReadInt8(16)) + ".";
	ipstr += rslib::Uint2Str(bsip.ReadInt8(24));	
	

	//update shard info:
	
	
	
	query = "UPDATE shards set ";
	query += "sofflineinactive='" + m_sofflineinactive + "', ";
	query += "slastonline='" + m_slastonline + "', ";
	query += "spolling=0,";
	query += "slaststatupdate='" + m_curdatestr + "', ";
	query += "slogonserverip='" + ipstr + "', ";
	query += "sresponse='" + m_sresponse + "', ";
	query += "savgresponse='" + m_savgresponse + "', ";
	query += "susers='" + m_susers + "', ";
	query += "savgusers='" + m_savgusers + "', ";
	query += "smaxusers='" + m_smaxusers + "', ";
	query += "suptime='" + m_suptime + "', ";
	query += "sisup='" + string(m_sisup ? "1" : "0") + "', ";
	query += "sdatasamples='" + m_sdatasamples + "' ";

	query +=" WHERE sid='" + m_sid + "' LIMIT 1;";
	
	if(!m_con->DoQuery(query))
	{
		Debug("Shard update query error:" + query + "\n" + m_con->GetError(true));	
	}	
	
	
	
	
	
	
	Debug("Updating graphs...");
	
	string shardidstr = m_sid;
	
	//init graph images:
	
	pngwriter networkgraph(367,238,1.0,string("../../img/graphs/network_" + shardidstr + ".png").c_str());
	pngwriter usersgraph(367,238,1.0,string("../../img/graphs/users_" + shardidstr + ".png").c_str());
	pngwriter statusimg(150,97,1.0,string("../../img/graphs/status_" + shardidstr + ".png").c_str());

	networkgraph.readfromfile("../img/graph.png");
	usersgraph.readfromfile("../img/graph.png");  //**** Crashes here ****
	statusimg.readfromfile("../img/status.png");
		
	
	//prepare for query:

	int x=62,y=68; //initial graph start point
	
	const unsigned int MAXROWS = 288; //max rows to fetch (288 = 1 full day)
	
	
	//--get Maxes:	
	if(!m_con->DoQuery("SELECT max(stonline) as *********, avg(stresponse) as avgresponse, max(stresponse) as maxresponse FROM (SELECT stonline,stresponse FROM stats WHERE stshardid='" + shardidstr + "' ORDER BY stdate DESC LIMIT " + BitStream::Int2Str(MAXROWS) + ") sub;"))
	{
		Debug("Graph update get max data query error\n" + m_con->GetError(true));	
	}
		
	UseQueryResult res = m_con->GetResult();	
	Row maxrow = res.fetch_row();
	
	unsigned int ********* = BitStream::Str2Int(string(maxrow["*********"]));
	unsigned int cmaxresponse = BitStream::Str2Int(string(maxrow["maxresponse"]));	
	unsigned int avgresponse = BitStream::Str2Int(string(maxrow["avgresponse"]));	
	
	unsigned int maxresponse=MathHelper::GetMin(avgresponse + (avgresponse/2),cmaxresponse);
	
	
	while(res.fetch_row()); //flush
	//---

	
	
		
	//get samples:
	if(!m_con->DoQuery("SELECT stisup,stshardid,stonline,stresponse FROM stats WHERE stshardid='" + shardidstr + "' ORDER BY stdate DESC LIMIT " + BitStream::Int2Str(MAXROWS) + ";"))
	{
		Debug("Graph update fetch data query error\n" + m_con->GetError(true));	
	}

		
	//loop through results and start plotting:
	
	int rownum=MAXROWS;
	
	res = m_con->GetResult();
	while(Row row = res.fetch_row())
	{
		bool stisup = string(row["stisup"])!="0" ? true : false;
		unsigned int stonline = (unsigned int)row["stonline"];
		unsigned int stresponse = (unsigned int)row["stresponse"];	

		if(stisup)
		{	
			//convert to a % so it fits in 150px of graph height
			stonline = ((float)stonline / (float)MathHelper::GetMax(1,*********))*150;
			stresponse = ((float)stresponse / (float)MathHelper::GetMax(1,maxresponse))*150;
			
			usersgraph.line_blend(x+rownum,y,x+rownum,y+stonline,0.85,		0.1,0.1,1.0);
			if(stresponse<=150)	networkgraph.line_blend(x+rownum,y,x+rownum,y+stresponse,0.85,	0.1,0.1,1.0);			
			else
			{
				networkgraph.line_blend(x+rownum,y,x+rownum,y+150,0.85,	0.1,0.1,1.0);
				networkgraph.line_blend(x+rownum,y+150,x+rownum,y+153,0.85,	1.0,0.1,0.1);
			}
		}
		else
		{
			//add downtime color:
			usersgraph.line_blend(x+rownum,y,x+rownum,y+150,0.3,	0.0,0.0,0.0);
			networkgraph.line_blend(x+rownum,y,x+rownum,y+150,0.3,	0.0,0.0,0.0);				
		}
		
		
		rownum--;
	}
	
	while(res.fetch_row()); //flush
	
	string writeval,writeval2;
	
	//shard name:
	usersgraph.plot_text("../img/UO.ttf",10,62,224,0,(char *)sname.c_str(),0,0,0);	
	networkgraph.plot_text("../img/UO.ttf",10,62,224,0,(char *)sname.c_str(),0,0,0);
	
	//plot type:
	usersgraph.plot_text("../img/UO.ttf",10,175,50,0,"Users",0,0,0);
	networkgraph.plot_text("../img/UO.ttf",10,120,50,0,"Network response time (ms)",0,0,0);
	
	
	//max num:
	writeval=BitStream::Int2Str(*********);
	writeval2=BitStream::Int2Str(maxresponse);
	usersgraph.plot_text("../img/UO.ttf",8,50-(writeval.size()*5),210,0,(char *)writeval.c_str(),0,0,0);
	networkgraph.plot_text("../img/UO.ttf",8,50-(writeval2.size()*5),210,0,(char *)writeval2.c_str(),0,0,0);
	
	//zero:
	usersgraph.plot_text("../img/UO.ttf",8,50,60,0,"0",0,0,0);
	networkgraph.plot_text("../img/UO.ttf",8,50,60,0,"0",0,0,0);
	
	
	
	
	//set status:

	
	if(!m_sisup)
	{
		statusimg.plot_text("../img/arialblack.ttf",16,24,40,0,"OFFLINE",0xDDDD,0,0);
	}
	else
	{
		statusimg.plot_text("../img/arialblack.ttf",16,26,59,0,"ONLINE",0,0xDDDD,0);
		
		writeval="Users: " + m_susers;	
		statusimg.plot_text("../img/berlinsans.ttf",12,32,40,0,(char *)writeval.c_str(),0,0,0);
		
		writeval="Ping: " + m_sresponse + "ms";	
		statusimg.plot_text("../img/berlinsans.ttf",12,32,20,0,(char *)writeval.c_str(),0,0,0);		
	}
	

	
	
	

	//close images:

	networkgraph.close();
	usersgraph.close();				
	statusimg.close();    //memory leak here
	
	Debug("Done updating graphs");
}

I can post the entire app, but that's a lot of code to sift through. That is the function that uses the pngwriter library and crashes. It crashes at the usersgraph.readfromfile("../img/graph.png"); function (about half way)

[ninja9578]

Okay... that's not libpng. libpng is not an object oriented library.

[Red Squirrel]

It's pngwriter, but I think it uses libpng in the background, so that's probably why I got that error. It's exactly:

libpng error: Read Error

On the screen, followed by a segfault. It does it when I read a png image into the app but it's very random. Might do it once every 100 times so this is very hard to figure out.

I'm thinking of just generating the image in code, rather then read from an existing png and just add my stuff over it.

Managed to get it to crash again, this is the full stack trace:

libpng error: Read Error

Program received signal SIGSEGV, Segmentation fault.
0x0000003f3dc128c9 in png_destroy_read_struct () from /usr/lib64/libpng12.so.0
#0  0x0000003f3dc128c9 in png_destroy_read_struct () from /usr/lib64/libpng12.so.0
#1  0x000000000043a838 in pngwriter::read_png_info (this=<value optimized out>, fp=0x7fffd16005f0, png_ptr=0x7fffd16005ec, info_ptr=0x7fffd16005c0)
    at pngwriter.cc:1535
#2  0x000000000043f319 in pngwriter::readfromfile (this=0x7fffd1600880, name=0x44bf8b "../img/graph.png") at pngwriter.cc:1350
#3  0x0000000000426f5c in ShardListEntry::UpdateInfo (this=0x7f3ecd33cb80) at includes/ShardListEntry.cpp:347
#4  0x000000000042897d in ShardList::UpdateShards (this=0x7fffd1601640, waittillalldone=false) at includes/ShardList.cpp:114
#5  0x0000000000428af8 in ShardList::QueryShards (this=0x7fffd1601640, amt=5) at includes/ShardList.cpp:53
#6  0x0000000000429a8d in main (argc=2, argv=0x7fffd16019e8) at poller.cpp:100
The program is running.  Exit anyway? (y or n) [answered Y; input not from terminal]

ShardListEntry.cpp:347 is the line of code I highlighted that it crashes at. This app will run this line of code many times and not fail, then at random, it will fail.

[Red Squirrel]

Bah this is only getting worse. Commented out the readfromfile but now it still crashes, but no info on the backtrace.

0x0000003a74c32215 in raise () from /lib64/libc.so.6
#0  0x0000003a74c32215 in raise () from /lib64/libc.so.6
#1  0x0000003a74c33d83 in abort () from /lib64/libc.so.6
#2  0x0000003f3dc13e95 in png_create_write_struct_2 () from /usr/lib64/libpng12.so.0
#3  0x00007fff03a348d0 in ?? ()
#4  0x0000000000000000 in ?? ()

I seem to really suck at C++... wonder if I should learn java instead at this point. This is really getting depressing.

[ninja9578]

My guess by looking at the stack track is that pngwriter::read_png_info isn't creating a large enough read structure. I'm betting the pngwriter library is making incorrect assumptions about the internal workings of libpng and libpng changed. Try using libpng14 instead of libpng12. I think it's pngwriter's fault, not yours. I would use libpng directly.

Also, turn the optimizations down to nothing in your IDE. You shouldn't try to debug with any optimizations turned on and I can tell by this line:
#1 0x000000000043a838 in pngwriter::read_png_info (this=<value optimized out>, fp=0x7fffd16005f0, png_ptr=0x7fffd16005ec, info_ptr=0x7fffd16005c0)
at pngwriter.cc:1535

that you have a few turned on.

[Red Squirrel]

I don't have any optimizations on as far as I know, this is my compile string:

-w -lpthread -lmysqlpp -L/usr/lib64/mysql -lmysqlclient -I/usr/local/include/mysql++/ -I/usr/include/mysql `freetype-config --cflags` -I/usr/local/include -L/usr/local/lib -lpng -lpngwriter -lz -lfreetype

I completely skipped over the libpng stuff and it's still crashing, somewhere else now, with no backtrace at all. So I can rule out libpng. At this point I'm at a loss.

If anyone actually feels like going through my code it would be appreciated, but there's quite a lot:

www.uogateway.com/misc/uogpoller.zip

You should most likely only have to look at the uogpoller code but the dependencies are provided in case.

Thanks in advance.

[Lindley]

Run the program through valgrind, it might give you some fresh clues where to check for problems.

Also, is this program multithreaded?

[Red Squirrel]

That's another issue, valgrind says there is a memory leak at the last close() function of my 3 png graphic objects. If I swap those lines of code around, it's always the last one that is marked as having a leak. It seems if I remove all the png related code, there is no leak.

This does not really tell me much though since I still have no idea what would even be causing this leak as I'm hardly using pointers and the ones I am using are being deleted properly. The library is very easy to use and I'm using it as per the documentation so I don't know where I'm going wrong.

I'm running a test now with the png library included, but not actually used. It's been running for about 10 minutes, but I'll have to leave it run for a good half hour to know for sure, and even then... This is the problem with these type of crashes, they are so randomized its hard to tell if it's really fixed/ruled out or not. If I can rule the png library as being the cause, I can probably look at using libpng directly as suggested, and see if that solves the issue.

Oh and yeah it's multithreaded. There's only a small portion of the work that is done in threads though. The rest of the code happens in the main program's thread with information previously gathered by the threads.

.. And it just crashed now. ARG.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f3eb04d3950 (LWP 8100)]
0x0000003a74c83b2e in memcpy () from /lib64/libc.so.6
#0  0x0000003a74c83b2e in memcpy () from /lib64/libc.so.6
#1  0x0000003c1d6a3005 in std::string::append () from /usr/lib64/libstdc++.so.6
#2  0x000000000042ee46 in std::operator+<char, std::char_traits<char>, std::allocator<char> > (__lhs=0x435d84 "{", __rhs=@0x198aa78)
    at /usr/lib/gcc/x86_64-redhat-linux/4.3.0/../../../../include/c++/4.3.0/bits/basic_string.tcc:677
#3  0x000000000041e016 in ShardListEntry::Debug (this=0x198a9a0, str=
        {static npos = 18446744073709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x7f3eb04d3020 "x\ufffd\001"}}) at includes/ShardListEntry.cpp:52
#4  0x0000000000428d52 in PollThread (data=0x198a9a0) at poller.cpp:49
#5  0x0000003a7580729a in start_thread () from /lib64/libpthread.so.0
#6  0x0000003a74ce439d in clone () from /lib64/libc.so.6
The program is running.  Exit anyway? (y or n) [answered Y; input not from terminal]
==7042== 
==7042== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 1)
==7042== malloc/free: in use at exit: 40 bytes in 1 blocks.
==7042== malloc/free: 107 allocs, 106 frees, 24,459 bytes allocated.
==7042== For counts of detected errors, rerun with: -v
==7042== searching for pointers to 1 not-freed blocks.
==7042== checked 1,301,168 bytes.
==7042== 
==7042== LEAK SUMMARY:
==7042==    definitely lost: 0 bytes in 0 blocks.
==7042==      possibly lost: 0 bytes in 0 blocks.
==7042==    still reachable: 40 bytes in 1 blocks.
==7042==         suppressed: 0 bytes in 0 blocks.
==7042== Rerun with --leak-check=full to see details of leaked memory.

[Red Squirrel]

Im getting all sorts of weird errors, it's always different. Got "broken pipe" a while ago in my debug function.

I'm starting to suspect it's something with my multithreading, but I don't get it, I'm using mutexes for any objects that are shared. Anything else I could be doing wrong? This is such as simple app I can't believe I'm having this much trouble.

[ninja9578]

I would protect libpng too, libpng might use globals too.

[Lindley]

It's always a good idea to mutex calls into libraries which don't explicitly document that they have MT support.

[Red Squirrel]

The png stuff is running only in the main thread, basically the only thing that is running in a seperate thread is a polling to get data and change values in the class object, and only when this thread is done, does this object call the libpng and other code.

I also ruled out png as my app still craps out without it but it just takes longer.

My logging function seems to be problematic, but I enclosed the whole thing in a mutex to ensure the log file does not get read/write by multiple threads, but it's still crashing.

[ninja9578]

Do this:

std::string temp("hello world");
std::string other(temp);

std::cout << (void*)temp.c_str() << std::endl;
std::cout << (void*)other.c_str() << std::endl;

If this prints out the same value, then strings are doing reference counting, which is not thread-safe. You can change that using different compiler settings.