multithreading vs multi-machines?

[hwright]

I'm an mechanical engineering student so I'm not extremely well informed when it comes to multithreading but I understand it can be a difficult process. I have however, read a few blogs that seem to indicate that programming across multiple machines is easier than on multiple cores. Is this true? Wouldn't the simple geographic overheads associated with multiple machine programming (distance, network problems, etc) cause huge problems?

Thanks for the info guys

[zerver]

Hi!

No, multi-machine is even worse than multithreading.

The separation of hardware means you must use multiple processes rather than multiple threads.
Ordinary synchronization primitives cannot be used so you need to use other mechanisms to coordinate access to shared data. You also need to decide where the shared data should reside. It gets complicated.

[DreamShore]

It depends on what you do.

For multi-threading, tasks have to wait each other for resources. And switching between threads in on execution core would slow things down.

And multi-machines, resources are seperated, so tasks can run independently. However they have to have some relationship, you'll find that resources are seperated too much.

If communication between tasks is relatively very little, and they depend on same resource very much if run on one machine, and you can afford the price of other machines, multi-machine would be nice.

But one thing is sure, mulit-machine programming won't make thing easier. You have to plan very carefully. The real problem for multi-core is almost only about the cache.

[Arjay]

In addition to what the other guys have said.

Multithreaded programming doesn't have to be difficult if you take a RAII approach to synchronization. Most developer run into trouble when sharing data between threads, and find the synchronization of the data difficult.

Structuring the program property and using RAII makes things relatively simple.

[TheCPUWizard]

Multi-Machine is "easier" because you are very well aware of what is happening. When you think a problem through it is ALWAYS easier.... :D

One other technique for dealing with many of the issues is "immutable objects". In this scenario (heavily used in many of the newest MP libraries), an object NEVER changes state.

Operations which would have changes an objects state, instead create a new object with the new state. This even goes as far as "containers".

If you have a list with 10 items in it, and want to remove one, you take the 10 item list as an input, and output a 9 item list as well as a "loose" instance....

This sounds downright wierd.....until you have used it and realized the power....