Worst Professional Blunders...

[TheCPUWizard]

SIZE=4]Wondering how many people will be willing to post some of their worst professional technology blunders...[/SIZE]
#1) While designing a targeting system for the main gun of a Frigate....

We were on sea trials, a videeo camera was loadede into the main gun, and drone aircraft were sent flying around. If the drone stayed in the center of the cross hairs, all was good.

Afrter about 2 hours, the ship suddenly reverberated with a heave metal on metal BANG, that rattles the entire ship. The was mild panic, since this was the first time the ship was out at sea, and anything upto and including a hull fracture could have been the cause. Nothing was found.

About an hour later, the same thing, less panic, but more concern. Again nothing was found.

Another 1.5 hours passed, and it happened again. This time a sailor saw the main gun move from the targed possition, slam down against the bottom stop, slam up agains the top stop, then return to tracking the target.

The cause was found...but why???

Looking at some of the machine logs the reason was determined. A gun must correct for "actual" wind. But the ships sensors (anonemeters) measure relative wind. You need to subtract out the ships speed and heading (the induced wind). Now when dealing with angular measurements, you typically use either 0-360 or +/-180. I had chosen the latter.

There was a filter algorythm that smoothed the data. If the wind was coming in over the stern (very rare), then sometimes you would get numbers in the 170-179.99 range other times in the -179.99 to -170 range.

Now what happens if you average these two numbers. YOU GET A NUMBER NEAR ZERO.

The computer interpreted this as an instant hange from a tail wind to a head wind!. The gun went berserk......

[June 1991 abord an RNLN M-Class Frigate]

[Ejaz]

Well, the technical blunders that I did, were int most cases because of the limitations imposed by client, as in one of my major project, I've to integrate different 3rd party SDKs, and some of them were not complient with each other (for example, integration of GameSpy SDK with DirectPlay, which isn't supported by GameSpy), but I was told to "Just Do It" , therefore, making layers over layers, so that everything remains intact totally shattered the design and degraded performance.

One of the biggest blunder that I did, was, I used atoi(), where I should use atof(), infact I shouldn't be using any of them at all, I should go with Automatic Type Conversion,
the expected value truncated and luckily, the result returned, it ceased the system, otherwise, it could be a real big news at media.

Another big blunder that I remember (it wasn't because of me, I just joined the company and the very first day that happened), was because of a design flaw. It was like, clients connect to a proxy and send an order request, which further send it to the main server. I don't know how, but somebody, executed 2 proxies at same LAN, and when sending the orders to the main server, they also send it to the other proxy, which take it as another order and again send it to the main server but also send it to the other proxy again (I guess it was sort of broadcast type of thing). This way, one single order got executed, I guess 13K times Imagine, if you are a stock market broker and this happens with you, how would you feel? The bug couldn't identified for about 2 months, when one day, it got replicated by me while I was giving demonstration to a client, coz I didn't knew the sequence of executions and by chance I executed two proxies at my LAN.

[sreehari]

#1 ) Not sure if this can actually be termed as a blunder, i would term this as in-experience, as it was my first project, and about in total 2 months of experience (barring the fact that my educational backgroud is of Computer science ).

The application was a real time tracking system, on a hardware, embedding a processor with a processing speed of only about 25 MHz. I proceeded with implementing it as an multi-threaded application, with 4 threads. Now all these threads were transfering data among themself's with a means of shared memory. And i totally forgot the concepts of OS. *cry* thread synchronization. and to top it all, it goes undetected through all the testing phase. finally during the customer demo, Every time the application was started on the hardware.... it caused the hardware to freeze completely :GOSH:

[Krishnaa]

I remember my biggest mistake till date was a typo, I used 2 same named variables (with a little difference in spelling) and used them all over assuming I had only one..... Then the thing started showing up bugs in variety and I wasted 2 days looking for problem, went mad as there was nothing wrong.

Luckily, I was debugging with some other debugger and I browsed debug symbols for the module for some reason and saw 2 names for that variable in the list. I was so ashamed to tell that to my boss, I had to make up something else.

[dglienna]

My first real job, out of school. Working in a huge office, doing WordPerfect on the Novell Lan. One day, a worker's machine crashed, and I told her to check for her backup copy. She couldn't get to the BACKUP folder.

I looked on my machine, and saw that each user had THE SAME FOLDER for all the document backups! I mentioned it to my boss, and he almost had a fit! They called in the big (blue) guns to come in and rectify the situation over the weekend.


They had to determine who owned the files! I made sure that I didn't try to open any of them

[CBasicNet]

2 weeks ago, I happen to saw a comment and source code I wrote 2 years ago, because I was reusing the class.

Something along these lines.

// have to allocate on the heap, as we do not know how big the vector 
// will be, might overflow the stack if allocate on the stack.
vector<string> *pvecStr = new vector<string>;

*LOL* I must be having a brain fart that day. Vector's array is always allocated on the heap! Later I removed the comments and changed the code to below,

vector<string> vecStr;

[Siddhartha]

I think bugs and software coding errors should not be classified as "professional blunders".

A professional blunder is (IMO) something on lines of... Taking up a wrong job, or firing the wrong email to the wrong people... Losing an opportunity, etc... Basically, actions that can hinder professional progress.

Bugs - well, software is one of the few professions where bugs are taken for granted and developers even get paid for fixing them.

(Think of Win 98 and the number of blue screens it came with - can you think of buying a car that came anywhere that close in quality?)

[TheCPUWizard]

I think bugs and software coding errors should not be classified as "professional blunders".

A professional blunder is (IMO) something on lines of... Taking up a wrong job, or firing the wrong email to the wrong people... Losing an opportunity, etc... Basically, actions that can hinder professional progress.

Bugs - well, software is one of the few professions where bugs are taken for granted and developers even get paid for fixing them.

(Think of Win 98 and the number of blue screens it came with - can you think of buying a car that came anywhere that close in quality?)

Well, lets think about this... A "Bug" is a programmer writing code that does not do what is intended...

A surgeon cuts off a persons arm because of knee pain... If that a "bug" or a MAJOR Professional Blunder???

IMHO, one of the biggest problems with the software industry is the lack of responsibility that is placed on developers. If you hire an accountant and that accountant makes a mistake, they are typically libel for any criminal charges/penalties/fines. However if you write an accounting package, and distribute it, it is generally held that the programmer is NOT accountable.

I seriously wonder how much actuality would go up if people were held legally responsible for any and all effects that arose from bugs within their programs.....

[Note: I am niot actually suggesting it, just making an observation on the mindset such as you describe].

[Siddhartha]

A surgeon cuts off a persons arm because of knee pain... If that a "bug" or a MAJOR Professional Blunder???

It definitely is only a 'bug'. At best a high-priority one that ought to have been detected and resolved either in the reviews or in the pre-delivery testing stage.

Compiled output of a programmer should never be a direct deliverable to a surgeon's computer. If ever a surgeon cut the wrong part of a body thanks to software, then it is a flaw in the process that deployed evidently un-tested software for such a critical operation. The tragedy is definitely not the programmer's fault. It is probable that the bug lies in a generic software module that interacts with the database, and does not deal with body-parts at all!

This is precisely why mission-critical software products often have testing teams that are as big or bigger than the development teams!
(You can include Cardio-Pulmonary Machines, etc in that category.)

I seriously wonder how much actuality would go up if people were held legally responsible for any and all effects that arose from bugs within their programs.....

Software is therefore usually well-protected by the EULA.

In a worst case scenario, the software-making company is held responsible and has to pay-up, but individual developers are rarely at fault. You can take the anti-trust rulings against Microsoft as an example, or other issues raised by its competitors against software developed by Microsoft.

[TheCPUWizard]

I my analogy to a surgeon, I was not referring to him using any technology. Just doing the wrong thing. Maybe he thought it was a different patient. Maybe he was just distracted.

The point is he has to take responsibility! Most developers don't. Not even employment responsibility. Imagine simply being fired if you checked 5 bugs into the repository over a 5 month period.

Regarding the programming/testing ration. It sdould be heavily weighted on testing.

I budget my "Hand-On Technical" time (not including administrative task)

1. 10% Analysis Architecture
2. 20% Programming/Coding
3. 40% Testing (and Test Development)
4. 10% Debugging
5. 10% Documentation

That means out of a 40 hour week, 8 hours coding, and the remaining 32 performing other development tasks. I very rarely go over-budget, or miss ship dates...

In a larger team (say 20 people)

1. 2 Architects
2. 4 Product Developers
3. 8 Test/QA Personal
4. 2 Troubleshooters
5. 2 Technical Documentation Writers.

[Siddhartha]

Software is inherently different when compared to other professions and in ways a more privileged profession.

When comparing software to surgery...

Software is rarely life-critical and when it is, it probably is tested by more eyes (and other software) than a surgeon's task.

Secondly, a surgeon who does a bad job is usually easily replaced by another that can do the job well. Software ironically is a profession where the person that codes the bug is usually the best candidate also for fixing it - for he knows 'his' module.

Thirdly, all software has bugs. No software developer can promise bug free code.
Good surgeons however can promise and deliver great results, consistently.

Fourthly, some surgeons make mistakes and some do get away without people knowing it. Bugs in software are easier to detect, to reproduce and to measure.

Last but not the least, big software is a lot more complicated than big surgery. There was a parallel I read somewhere that compared developing an Operating System to the complexities of building a Space Shuttle.

Imagine simply being fired if you checked 5 bugs into the repository over a 5 month period.

Yes, imagine the only guy who is capable of fixing the bug is the one getting fired.

[TheCPUWizard]

Any company (except a one man shop) that allows the situation where only one person is experienced in a peice of code DESERVES to immediately go bankrupt.

At a minimum the person who tests (white box) the code needs to know how every line of the code functions to ensure code coverage and boundary condition checking.

Ideally every peice of code should be written by a team of at least two programmers working in tandem. Some shops take this to the extreme that they share a single computer workstation (I dont go quite that far).

Also, I engourage a company with multiple developers to establish a process where the person who wrote the code (
and checked it into tthe repository) is not allowed to perform any of the maintenance work on that module. It MUST be done by a different person.

This encourages people to write the cleanest possible code. Otherwise their coworkers will rapidly come to hate them.

The teams that have established these simple guidelines do find it works!!!

(And egotistical developers who think they can not be easily replaced, soon find themselves on the street)).

[Siddhartha]

Any company (except a one man shop) that allows the situation where only one person is experienced in a peice of code DESERVES to immediately go bankrupt.

Actually, multiple people knowing one piece of code is not possible in any big organization producing a colossal bit of software.

What is proposed above may work only in systems that you so correctly called - "shop".

Most (all that I know of) successful companies build themselves on the capabilities and talents of their employees - and derive from factors that makes the talent unique. Any model that proposes that developers be easily replaceable resources and liable to be fired when they write bugs would ensure employee un-happiness, low productivity and a high attrition rate.

I agree with the point you make about more people knowing and being responsible for one software. But, this is not with the perspective that the resource be easily fired when he delivers bugs, rather that sharing knowledge increases interaction between people, raises team competencies and reduces pain when the employee quits.

Redundancy costs a lot of money by the way, and few companies can afford to let two people know every line of code on an organization-wide basis.

[GremlinSA]

I figured this as the worst blunder i've done...

I also do some network instalations, and when istalling the switch-hubs, when making the mini flyleads, I plug both ends into the switch to check them.. (1 or 2 often needs a little extra press with the crimpler) after doing a few dozen of them, linking all the sites, and switching a few leads over, The system ran to speck..

needless to say with initial testig the system ran fine.. Although half an hour later i was called back, because the network had bogged down, and connectivity was imposible..

Looking at the hubs.. Activity was at 100%... after a little time, i found the problem..

I accidently put two flyleads to link two switchs... with smart packet switch's this did not cause immidiate problems but as soon as one of the systems sent a broadcast, the two hubs kept on bouncing this packet back and forth, also seding it out too all nodes... thus effectivly killing the network...

[TheCPUWizard]

Redundancy costs a lot of money by the way, and few companies can afford to let two people know every line of code on an organization-wide basis.

In my experience [30 year primarily in the USA], the cost of "redundancy" is actually very low. Considering that nearly every developer knows multiple pieces of code. There is bound to be a time when he/she needs to work on both at the same time, while another developer may sit idle. Having significant cross-training addresses this issue, as well as issues such as vacations, illness and other reasons why one person may be unavailable.

My "emergency service" contract costs clients approximately 1.5 times my normal rate, since I have to step into an environment where they do not have a knowedge expert available.