multicore support in C#

[Mutant_Fruit]

This statement is completely false:

"You won't increase performance and you run the risk that some other developer has had the same (bad) idea and then both of your programs will suffer a huge performance penalty as they fight for the same core despite having several completely free cores."

Let me make my position clearer:

On a standard desktop (or standard server), if you app1thread1 is forced to execute on core0 and app2thread1 is forced to execute on core0, then both of those threads will execute on core0. If you have a 10000 core box, those two threads will still execute on core0.

Therefore both applications will have 1/2 the performance they should have as compared to letting the OS schedule the two threads onto core0 and core1 respectively.

So yes, my statement is correct. Sure, you can employ third party software which will change the processor for active threads/processes, but then you're just substituting the OS scheduler for a third party scheduler. Sure, this can work. The third party scheduler could easily be better. But unless you employ this software, you *should not* force threads to execute on a specific core.

EDIT:
If you have third party software which manages which thread operates on which core, then trying to set affinity in code is a waste of time as the third party scheduler will just override that. So you've just weakened the argument for trying to set the affinity through code.

Having properly weighted factors used to determine process/thread placement in cores for the enviornment you are in, gives you much better odds of increasing your performance, if even for a short period of time, than blind-luck does.

It also would make it impossible to "fight for the same core despite having several completely free cores" in all cases, again, worse case being better off in the short term than long term.

Unfortunately these properly weighted factors will not be available in usercode.

[TheCPUWizard]

I think the two points of view have finally converged (I have been watching for a while).

IMHO:

1) There are definately viable alternatives to the "built-in" Widnwos scheduling/allocation of resources. These take into accound the entire environment on the system, and are often very sophesticated. For many system these may be much better.

2) Attempting to "coerce" things from inside an application without having a view of thge total system is likely to have a (potentially severe) negatvie impact when used on "normal" desktops or servers.

Both of these mimic my own personal/profession experiences.

[cblind]

Thanks for valuable insights !

To further clarify our particular predicament: we have PC box running Win Xp OS and our custom application. In our case we don't have an issue of multiple developers "overbooking" one core while others are "idle" - it's less then handful of people working on the app, code is reviewed, and overall development is tightly managed...What we want, is to free a core (or two) to just process video in real time (video processing is a module inside of our app) - 'hope' is that video thread will not have to compete for resources but rather have dedicated space to it's thing.

[TheCPUWizard]

Thanks for valuable insights !

To further clarify our particular predicament: we have PC box running Win Xp OS and our custom application. In our case we don't have an issue of multiple developers "overbooking" one core while others are "idle" - it's less then handful of people working on the app, code is reviewed, and overall development is tightly managed...What we want, is to free a core (or two) to just process video in real time (video processing is a module inside of our app) - 'hope' is that video thread will not have to compete for resources but rather have dedicated space to it's thing.

You are missing the poiint. To be effective,the code MUST be aware of every process runing on the system. One my current system there are 53 processes running WHEN THE SYSTEM IS IDLE( as counted by TaskMgr)., and many of these are multi-threaded

To dedicate a processor to YOUR work, you would have to impact the processing of all of these processes/threads. Unless you coordinate ALL of the threads of ALL of the processes, you are NOT going to achieve your goal.

[Mutant_Fruit]

In our case we don't have an issue of multiple developers "overbooking" one core while others are "idle"

Just so long as you're sure that no *other* developer working on a completely separate project, (maybe from a completely separate company) decides to set thread affinity on his application so he can do complex processing on a specific core... which happens to be the one you choose too.

Go benchmark the difference, the Process class has a method for setting thread affinity. Benchmark your app with no affinity set on the video thread and with affinity set. The difference should be negligible.

[ZOverLord]

Let me make my position clearer:

On a standard desktop (or standard server), if you app1thread1 is forced to execute on core0 and app2thread1 is forced to execute on core0, then both of those threads will execute on core0. If you have a 10000 core box, those two threads will still execute on core0.

Therefore both applications will have 1/2 the performance they should have as compared to letting the OS schedule the two threads onto core0 and core1 respectively.

So yes, my statement is correct. Sure, you can employ third party software which will change the processor for active threads/processes, but then you're just substituting the OS scheduler for a third party scheduler. Sure, this can work. The third party scheduler could easily be better. But unless you employ this software, you *should not* force threads to execute on a specific core.

EDIT:
If you have third party software which manages which thread operates on which core, then trying to set affinity in code is a waste of time as the third party scheduler will just override that. So you've just weakened the argument for trying to set the affinity through code.


Unfortunately these properly weighted factors will not be available in usercode.

Again, you keep making the auto-balancing code appear stupid and brain dead.

If you add balancing code to a process that is going to launch many other processes, you don't force things to go to the worst core so that you run 50 percent worse than without your balancing code.

Again, the best method would be to have a process do ALL core management, but as a secondary solution adding balancing logic to a program that is responsible to launch many other processes over its runtime life is a good idea, especially when you are trying to exclude some cores so that they won't have the workload of the processes you are launching added to them.

It's common sense, you can't "Dumb That Up!" You also can't make claims like "Let Windows manage it, because you will screw it up somehow" Windows can't auto-magically manage this, because there is core exclusion required, get it?

The goal is to use factors to choose the "Best" core at that time, that's the ONLY goal, ("The program in question here is NOT a system resource management application") which could and will become worse overtime ("Because the program in question is not globally managing all processes"), but at that moment is the "Best" which may include the exclusion of some cores as well.

Also any and all information required to make create these properly weighted factors is avalable via user code.

What system information are you claiming is not available via user code to create these weighted factors, exactly?

[ZOverLord]

You are missing the poiint. To be effective,the code MUST be aware of every process runing on the system. One my current system there are 53 processes running WHEN THE SYSTEM IS IDLE( as counted by TaskMgr)., and many of these are multi-threaded

To dedicate a processor to YOUR work, you would have to impact the processing of all of these processes/threads. Unless you coordinate ALL of the threads of ALL of the processes, you are NOT going to achieve your goal.

The code in question does "Not" need to be aware of "every process running on the system" it could be, but it is "Not" required to select the best core and exlcude others when it does this.

If the goal is to limit impact on specific cores from a process that will launch many other processes during its lifetime, then you CAN achieve your goal, because those specific cores will now have NONE of your processes you launched running in them. They may have other processes running in them, but none would be yours. So you would in fact acheive your goal. The goal for this application is not system wide resource management, but....it does include leaving some cores NOT running any of its created processes.

Here is a perfect example:

Say you have 8 cores and wish to dedicate 2 to development and leave the other 6 to your user base. You have some development applications that create many processes over time and you want to add balancing code to them.

You have no need to have knowledge of every process running in every core, or even the 2 cores you wish to use for development applications. All you need are;

1. What cores are these processes allowed to run in.

2. Using factors created from "System Level" information not down to the level of process information select the best core of the 2 allowed.

One would be hard pressed to make claims that the user base, now does not have better resonse time. Of course the development team could be screaming bloody murder, but even in a worse case, you could make the available cores be 3 not 2 then.

The ability to isolate and exclude cores here, is also being missed here. "built-in" Windows resource management does not deal with this issue, auto-magically.

There also is no need to "Micro-Manage" this balancing down to a process level, that kind of detail is in most cases Over-Kill.

Factors and weighting them properly are important as well, simple things which most programmers don't know can hurt you.

One example is cores/CPU's can be using "Mutual Exclusion" for things like memory management, as one example, more than some others, so they will never obtain a high CPU busy rate for example, yet they are the worst cores to choose because they are stalled so much. You can assign a lower weight to this factor so that when using factors to select cores, these cores will also be more likely to be exlcuded.

Factors allow you to tune the balancing for your enviornment and the workload it is exposed too.

Not to be insulting, but....so far I have seen one person here, who has used "libraries" to provide some level of this "Once" in their entire lifetime. This is the only "Real-Life" experience I have seen in this thread so far of using your own balancing methods for process launch.

I have designed and coded a commercial product that does this today for many comapnies wordwide and has been doing this for over 20 years now, with systems that have 2-16 cores per node and can have over 2,000 nodes per network. That's my real-life experience.

I have seen NO supporting evidence of any kind, that shows that in this case, adding this balancing code would not be productive and a benefit to this system in question, nor have I seen anyone explain how to use "built-in" Windows resource management to exclude cores, auto-magically, without code for this case. What I have seen is dire warnings on how you will "hurt" yourself in some way, yet no detail is provided other than statements like "You will be 50 percent slower".

[TheCPUWizard]

ZOverLord

Your last reply directly contradicts my experiences over the past decade (including using high-end machines such as IBM's 64 (Xeon) processor servers).

I have personally (professionally) experienced many parasitic conditions where only a single "application" (implemented as multiples processes that were each multi-threaded) was running on the system, but SYSTEM Processes wer left to the Operating Systems "whim" (this has been true on Windows, Solaris and Unix based systems).

While the types of management you are promoting can appear to yield "better" performance on a statistical basis, the real test comes down to what is the actual WORST case performance that can be induced.

For example, on a completely "unmanaged" (no affinity control) basis, the timing for an operation may have a normal distribution curve over the 3-5 second range (centered at 4 seconds).

When affinity is applied, 99.9% of the sampled meansurements may be between 1-3 seconds (centered around 2 seconds), but with a 0.1% mearuement rate of 10 seconds.

For a true real-time system, this means that the "managed" system did NOT double the performance (from 4 seconds, to 2 seconds), it actually slowed it down by a factor of 2 (from a worst case of 5 seconds to a worst case of 10 seconds).

Of course these numbers are simply for illustrative purposes (and easy calculations ) but the situation is quite real.

FWIW: In the case of the IBM Xeon system, we catually removed 48 of the processors from the operating system ENTIRELY and implementing "bare-metal" single threaded code for the real-time requirements (effectively making them dedicated microcontrollers.....This achieved a significant (4x) improvement over the best that could be achieved with processes running on top of an OS.

[ZOverLord]

ZOverLord

Your last reply directly contradicts my experiences over the past decade (including using high-end machines such as IBM's 64 (Xeon) processor servers).

Please show us respected white papers that support this theory, links please. There should be many for statements like these:

"For a true real-time system, this means that the "managed" system did NOT double the performance (from 4 seconds, to 2 seconds), it actually slowed it down by a factor of 2 (from a worst case of 5 seconds to a worst case of 10 seconds)."

"While the types of management you are promoting can appear to yield "better" performance on a statistical basis, the real test comes down to what is the actual WORST case performance that can be induced."

These statement make no commone sense, statement 1 is saying that you will have a 200 percent performance impact by attempting to manage a system, show me respected white papers please that say this. Statement 2 says, don't auto-balance, simply define the worse case and use that for everyday core selection. Again, show me respected white papers that say this please.

Are you reading what you are writing?

Secondly, please explain how you would exclude cores as well in this case, without code.

Also, I never made a claim that adding balancing logic to a system will double the performance of a system. Not sure where you got that from?

Thanks.

[TheCPUWizard]

The quickest reference is factually right in the current issue of MSDN, where the difference betweej "Fairness" and "Total Performance" is discussed. Most of the detailed material I have is either subject to NDA or otherwise difficultto post, but I will look for some when I am back in the office.

It is acutally very simple to demonstrate with even simple applications that keep track of the WORST CASE time to perfom a given task. Simply run an number of instances equal to the number of cores. Let them run without doing any "affinity". Then start "locking" various instances to specific cores.


As far as excluding the cores, it is a fairly simple matter to adjust the HAL so that it simply does not report their existance to the operating system. As far as the OS is concerented a Q660 can easily become a 1,2,3 cor system, of an i7 a 2,4,6 "core" systems (due to some of the windows internals, we have NOT found a viable method of blocking a single hyper threaded pathway on a specific core, and that scenario is very unlikely to provide benefit in any case.

[ZOverLord]

The quickest reference is factually right in the current issue of MSDN, where the difference betweej "Fairness" and "Total Performance" is discussed. Most of the detailed material I have is either subject to NDA or otherwise difficultto post, but I will look for some when I am back in the office.

It is acutally very simple to demonstrate with even simple applications that keep track of the WORST CASE time to perfom a given task. Simply run an number of instances equal to the number of cores. Let them run without doing any "affinity". Then start "locking" various instances to specific cores.


As far as excluding the cores, it is a fairly simple matter to adjust the HAL so that it simply does not report their existance to the operating system. As far as the OS is concerented a Q660 can easily become a 1,2,3 cor system, of an i7 a 2,4,6 "core" systems (due to some of the windows internals, we have NOT found a viable method of blocking a single hyper threaded pathway on a specific core, and that scenario is very unlikely to provide benefit in any case.

The article in MSDN does not support your statements that I reference in my previous post and the goal here is not to hide cores from all processes by hiding them from the operating system, it is to limit launches of other processes to them from this process in question.

MSDN Article: http://msdn.microsoft.com/en-us/magazine/dd252943.aspx

[ZOverLord]

Hi,
I tried searching but am coming up short.. I have a quadcore PC running Win XP and am looking for a solution where I can selectively assign jobs to each processor - i.e I want to pick one of my processors and tell it to exclusively work on some specific data crunching. I tried to find a library that will allow me to do so but with no luck.
Anyone know of a library that will let me assign jobs to specific processor? Any other approaches and solutions are greatly appreciated

I would avoid most libraries if possible and roll your own. While this is a thread balancing example, the point still stands:

"Initially, the thread control was implemented using Microsoft's Parallel Extensions Library. This made life a little easier, but unfortunately, performance was terrible."

From: http://www.codeproject.com/KB/threads/PetriDish.aspx

Just one of many complaints you will find about this library.

Instead create your own factors, as in this example, because you will minimize the overhead of using a library that maybe doing far more than what you need, and the overhead of that library may exceed the benefit it could provide you for this case in question:

http://www.codeproject.com/KB/mcpp/loadbalancing.aspx

Sometimes when people create libraries, they try to be all things to all people, which can defeat the purpose they are created for. Using the techinques above, you should be able to create important factors to use for weighting and you will have a much faster system response time, if done right ("Which is not complicated or hard to do"), which will also create the isolation of cores as you wish as well.

[Mutant_Fruit]

I think you're confusing the issue.


A standard desktop only has the windows scheduler. It has no other way to automatically set thread affinity. In this scenario, trying to micromanage threads is 'wrong' for the reasons I have very clearly laid out already. Therefore under these circumstances trying to micromanage which core your process works on is both 'stupid and braindead' as you pointed out.

Adding balancing code to automatically adjust processor affinity to 'choose the best core' on a standard desktop is both 'stupid and braindead' too. Why would you replicate a complex feature which the operating system already provides for free? Especially when that code is already running under the windows scheduler and so its performance is already constrained by the windows scheduler.

Again, the best method would be to have a process do ALL core management, but as a secondary solution adding balancing logic to a program that is responsible to launch many other processes over its runtime life is a good idea, especially when you are trying to exclude some cores so that they won't have the workload of the processes you are launching added to them.

No. How can your process tell which core is the best? Implement complex algorithms which will have a 50/50 chance of failing? How can you tell how busy a core really is when the windows sceduler can pre-empt any of your code for an unknown (and potentially lengthy) amount of time? You can't do this from usercode. Core exclusion is exactly the same as setting affinity to one (or more) cores. You can still very easily 'exclude' yourself from the only free core.

Say you have 8 cores and wish to dedicate 2 to development and leave the other 6 to your user base. You have some development applications that create many processes over time and you want to add balancing code to them.

That's fine for development. It's your machine, you know what's running on it, you can have a reasonable idea which core is 'best'. But for release you don't have any idea what the end-user is running on their desktop and so you shouldn't do this. Otherwise you run the risk of crippling performance artificially.

So my point is that if you are designing an application for the standard desktop or standard server, you are shooting yourself in the foot by trying to micomanage the cores your threads operate on.

I have designed and coded a commercial product that does this today for many comapnies wordwide and has been doing this for over 20 years now, with systems that have 2-16 cores per node and can have over 2,000 nodes per network. That's my real-life experience.

This is not a standard server or desktop.

[Mutant_Fruit]

http://www.codeproject.com/KB/threads/PetriDish.aspx

This is an exercise in learning how to multi-thread an applications workflow. It deals with splitting a task up into decent sized chunks which can then be executed in parallel. It isn't really relevant to the discussion unless you want to use it as an example of how you can achieve great performance by letting windows automatically manage which core your thread executes on...

http://www.codeproject.com/KB/mcpp/l....aspx#inaction

I'm not sure what relevance this has at all.

[ZOverLord]

I think you're confusing the issue.

A standard desktop only has the windows scheduler. It has no other way to automatically set thread affinity. In this scenario, trying to micromanage threads is 'wrong' for the reasons I have very clearly laid out already. Therefore under these circumstances trying to micromanage which core your process works on is both 'stupid and braindead' as you pointed out.

First, we are NOT talking about threads here, we are talking about launching processes. Secondly, A standard desktop has more than the windows scheduler. Here is just one example of many. But one example trumps "no other way ", so my point has been made.

http://www.pctipsbox.com/start-appli...-and-priority/

http://www.howtogeek.com/howto/windo...windows-vista/

Adding balancing code to automatically adjust processor affinity to 'choose the best core' on a standard desktop is both 'stupid and braindead' too. Why would you replicate a complex feature which the operating system already provides for free? Especially when that code is already running under the windows scheduler and so its performance is already constrained by the windows scheduler.

False. It is easy to do, and is smart when you wish to exlude cores as well.

If it was 'stupid and braindead' as you say then why is load-balancing part of the next version of Visual Studio 2010? Is Microsoft 'stupid and braindead' as well?

from: http://channel9.msdn.com/pdc2008/TL26/

"The manycore shift presents an unprecedented business opportunity for developers to design new software experiences that take advantage of the performance power of manycore architectures. At the same time, parallel programming is complex, difficult and labor-intensive, for even the most skilled developers."

from: http://msdn.microsoft.com/en-us/conc...y/default.aspx

No. How can your process tell which core is the best? Implement complex algorithms which will have a 50/50 chance of failing? How can you tell how busy a core really is when the windows sceduler can pre-empt any of your code for an unknown (and potentially lengthy) amount of time? You can't do this from usercode. Core exclusion is exactly the same as setting affinity to one (or more) cores. You can still very easily 'exclude' yourself from the only free core.

You can do this from usercode. There is nothing complex about it for most people.

"Performance Counter Walkthroughs

The walkthroughs in this section introduce you to the use of the PerformanceCounter component. The walkthroughs show you how to use both system performance counters and custom performance counters.

from: http://msdn.microsoft.com/en-us/library/d8xb98ke.aspx

That's fine for development. It's your machine, you know what's running on it, you can have a reasonable idea which core is 'best'. But for release you don't have any idea what the end-user is running on their desktop and so you shouldn't do this. Otherwise you run the risk of crippling performance artificially.

Microsoft is NOT adding this to Visual Studio 2010 for development machines, it's doing it to help you load-balance on your users system.

"Come learn how the next version of Visual Studio and the Microsoft .NET Framework can help you write better performing and more scalable applications."

from: http://channel9.msdn.com/pdc2008/TL26/

So my point is that if you are designing an application for the standard desktop or standard server, you are shooting yourself in the foot by trying to micomanage the cores your threads operate on.

My point is, every, not some, statements you have made in this post, have no factual basis at all, as in none, nada, zip.

Unlike you, I provided links to prove that. Now you can call Microsoft crazy too. My intent is to provide facts to the original poster, not conjecture, or unsupported statements.

I think I have done that, with your help, of course.

This is not a standard server or desktop.

There is no standard server or desktop. What would they be? What do they look like?

Please list the software restrictions and limitation on "Standard" servers and desktops.