CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 15 of 15
  1. #1
    John E is offline Elite Member Power Poster
    Join Date
    Apr 2001
    Location
    Manchester, England
    Posts
    4,835

    Multiple core processors

    Suppose I build a multi-threaded app and someone runs it on a multiple core processor. I suppose ideally, the OS would allocate threads between the different cores but from what I've observed that rarely (if ever) seems to happen. What happens is that the first core runs at 90% while the others sit there doing nothing. It isn't necessarily a big problem but it might be nice occasionally to make use of the extra processing power that's available. Where can I find some useful programming tips for sharing tasks between the available cores?
    "A problem well stated is a problem half solved.” - Charles F. Kettering

  2. #2
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,620

    Re: Multiple core processors

    Windows takes care of designating threads to free cores unless you instruct it specifically that a certain process thread gets affine to a certain core. In your case one thread seems seized the core once, never lets it go, and Windows can do nothing with that.
    Best regards,
    Igor

  3. #3
    John E is offline Elite Member Power Poster
    Join Date
    Apr 2001
    Location
    Manchester, England
    Posts
    4,835

    Re: Multiple core processors

    No, that's not the problem exactly. Our product (an audio workstation) uses buffers for media streaming. Obviously for the media to get streamed seamlessly and continuously, each buffer needs to get filled in less time than it takes to read the data back out. But (like most digital audio workstations) ours can use 3rd party plugins. These use up extra processing power and can lengthen the time it takes for Windows to fill the buffers. Naturally, Windows doesn't know that filling the buffers is a time-critical task.

    So I'm wondering if we could benefit by moving all this 'buffer filling' stuff onto a dedicated core.
    "A problem well stated is a problem half solved.” - Charles F. Kettering

  4. #4
    Join Date
    Jul 2005
    Location
    Netherlands
    Posts
    2,042

    Re: Multiple core processors

    Quote Originally Posted by John E View Post
    Suppose I build a multi-threaded app and someone runs it on a multiple core processor. I suppose ideally, the OS would allocate threads between the different cores but from what I've observed that rarely (if ever) seems to happen. What happens is that the first core runs at 90% while the others sit there doing nothing.
    Perhaps the extra threads are waiting or are not doing CPU-bound work.
    If you create a simple test application that does some CPU-bound work in multiple threads without any synchronization, you'll see multiple cores being used.
    Cheers, D Drmmr

    Please put [code][/code] tags around your code to preserve indentation and make it more readable.

    As long as man ascribes to himself what is merely a posibility, he will not work for the attainment of it. - P. D. Ouspensky

  5. #5
    Join Date
    Jul 2005
    Location
    Netherlands
    Posts
    2,042

    Re: Multiple core processors

    Quote Originally Posted by John E View Post
    So I'm wondering if we could benefit by moving all this 'buffer filling' stuff onto a dedicated core.
    What does the 'buffer filling' do? Does is read from disk? Does it allocate memory?
    Those are typical tasks that don't scale well when using more threads. Reading from disk is I/O bound and it's best to do it in a single thread (e.g. using asynchronous I/O). If multiple threads need to quickly allocate memory, you can use separate heaps or your own allocator to make the allocation lock-free.
    Cheers, D Drmmr

    Please put [code][/code] tags around your code to preserve indentation and make it more readable.

    As long as man ascribes to himself what is merely a posibility, he will not work for the attainment of it. - P. D. Ouspensky

  6. #6
    John E is offline Elite Member Power Poster
    Join Date
    Apr 2001
    Location
    Manchester, England
    Posts
    4,835

    Re: Multiple core processors

    Quote Originally Posted by D_Drmmr View Post
    What does the 'buffer filling' do? Does is read from disk? Does it allocate memory?
    Those are typical tasks that don't scale well when using more threads.
    Understood... but presumably, moving it to another core would help? Otherwise, what's the point of having multiple cores?
    "A problem well stated is a problem half solved.” - Charles F. Kettering

  7. #7
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,620

    Re: Multiple core processors

    Quote Originally Posted by John E View Post
    Naturally, Windows doesn't know that filling the buffers is a time-critical task.
    Not exactly. When you use DirectShow, it does know that. But when you circumvent DS, you're on your own.

    Quote Originally Posted by John E View Post
    Otherwise, what's the point of having multiple cores?
    John, when your new thread gets created, it really uses multiple cores. But when the thread spends 99% of time being blocked in I/O operations (waiting for data from DMA, whatever NIC or HDD controller), how having multiple CPUs could improve the situation? The cores might help later and only if your post-processing plugins allow the buffers be processed in parallel and not locking on each other's results.
    Last edited by Igor Vartanov; January 2nd, 2013 at 01:09 PM.
    Best regards,
    Igor

  8. #8
    John E is offline Elite Member Power Poster
    Join Date
    Apr 2001
    Location
    Manchester, England
    Posts
    4,835

    Re: Multiple core processors

    Maybe I'm misunderstanding the situation.

    On a single core CPU, threads are always running serially aren't they? Even when they're not blocked waiting for each other, they only give the impression of running in parallel. They're actually running in serial time slices. So the more threads you create (on a single CPU) the less efficiently each of them runs.

    So if you can run some threads on an underused core (assuming they're not waiting for threads on the first core) they'll run more efficiently won't they? Or have I misunderstood it?
    "A problem well stated is a problem half solved.” - Charles F. Kettering

  9. #9
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,620

    Re: Multiple core processors

    Again, when your thread waits for I/O completion, the CPU just sleeps. Does it matter for you how many cores will sleep simultaneously? From your OP I can guess that this is what really might happen in your case.

    Now, to judge on how effective multiple cores will be in your case you must be sure your problem is lack of calculation power. This is where CPUs good at. Otherwise, when CPUs have nothing to calculate, you are going to have no gain, to your sincere surprise maybe.
    Last edited by Igor Vartanov; January 2nd, 2013 at 01:46 PM.
    Best regards,
    Igor

  10. #10
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,620

    Re: Multiple core processors

    Quote Originally Posted by John E View Post
    Maybe I'm misunderstanding the situation.

    On a single core CPU, threads are always running serially aren't they? Even when they're not blocked waiting for each other, they only give the impression of running in parallel. They're actually running in serial time slices. So the more threads you create (on a single CPU) the less efficiently each of them runs.

    So if you can run some threads on an underused core (assuming they're not waiting for threads on the first core) they'll run more efficiently won't they? Or have I misunderstood it?
    One more comment to add. Threads always run in serial time slices, due to the nature of Windows multithreading. And the rule "So the more threads you create the less efficiently each of them runs" is always true, disregarding single or multiple cores the system has. There are hundreds to thousands threads running simultaneously in OS, so they must be properly dispatched and (re)prioritized, which overhead is inevitable even with multicore architectures.

    Absolutely different thing is gaining some performance rates from using multicore environment. The gain is not automatically achieved just by using several threads, or some more threads compared to what it was before. It's not a mechanical thing, it's a matter of skills and art. A matter of comprehension and design.
    Best regards,
    Igor

  11. #11
    John E is offline Elite Member Power Poster
    Join Date
    Apr 2001
    Location
    Manchester, England
    Posts
    4,835

    Re: Multiple core processors

    Thanks to you both. After your explanations I now have a much better understanding of this subject - but there's still one thing puzzling me... what part does DMA play in all this? I always thought that DMA was a way for peripheral devices (graphics card / disk controller etc) to transfer data to or from RAM without needing to hold up the CPU.
    "A problem well stated is a problem half solved.” - Charles F. Kettering

  12. #12
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,620

    Re: Multiple core processors

    And? What contradiction you can see with the said before? Say some thread needs data to be read from file. Reading from file actually needs the data be retrieved from HDD. File System Driver encapsulates all the details of dealing with hardware, so all the details are hidden from user space thread. But the data anyway must appear on RAM to be accessible for the thread, and OS just marks the thread as 'in I/O' until underlaying hardware drivers report the I/O completed. The thread remains blocked and eats no CPU. The data retrieval is done mostly by DMA mechanisms (which interface between HW buffers and RAM) that again eat no CPU. Nobody eats CPU, but data retrieval never becomes instant because of using no CPU. The freed CPU allows OS to do some other thread processing, but your thread still remains blocked in I/O operation with near to zero CPU percentage.

    More than that, the CPU counters just reflect CPU cycles spent in user mode only, but clock cycles spent in kernel mode remain not reported. You typically see that effect when doing some bulky copying over network, especially from some network resource to other one. Windows seems doing nothing heavy according to CPU counters, but the GUI response is noticeably low. You are never reported of the CPU spent in MPR, TCP/IP stack, etc.
    Last edited by Igor Vartanov; January 6th, 2013 at 08:54 AM.
    Best regards,
    Igor

  13. #13
    John E is offline Elite Member Power Poster
    Join Date
    Apr 2001
    Location
    Manchester, England
    Posts
    4,835

    Re: Multiple core processors

    Quote Originally Posted by Igor Vartanov View Post
    And? What contradiction you can see with the said before?

    [...]

    The data retrieval is done mostly by DMA mechanisms (which interface between HW buffers and RAM) that again eat no CPU. Nobody eats CPU, but data retrieval never becomes instant because of using no CPU. The freed CPU allows OS to do some other thread processing, but your thread still remains blocked
    This is what I found confusing... eariler, you said:-

    Quote Originally Posted by Igor Vartanov View Post
    when your thread waits for I/O completion, the CPU just sleeps. Does it matter for you how many cores will sleep simultaneously?
    I thought you meant that whenever an I/O thread (running on any core) got blocked, waiting for hardware, all the other cores would need to stop processing for some reason. Whereas what you really meant (I assume) is that any threads that are dependent on the blocked thread will also get blocked, no matter which core they're running on.
    "A problem well stated is a problem half solved.” - Charles F. Kettering

  14. #14
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,620

    Re: Multiple core processors

    What I really meant is the following. Imagine a situation when you have 10 threads that OS dispatcher looks after. When your thread is blocked in I/O, the dispatcher walks along the other 9 and finds them the same blocked in I/O, or having no window messages in queue to process. In this case the CPU gets halted and sleeps consuming no power until next dispatching time slice begins.

    In case of multi core system all the CPU cores are halted the same way as they just have nothing to do. Having more CPUs never makes your thread to wake up earlier than I/O operation completion allows that.

    This is the picture from OS dispatcher's standpoint. But from the thread's point of view, when thread is blocked no CPU resource is delivered by OS to the thread, so CPU "sleeps" from the blocked thread's standpoint as well.
    Best regards,
    Igor

  15. #15
    Arjay's Avatar
    Arjay is offline Moderator / EX MS MVP Power Poster
    Join Date
    Aug 2004
    Posts
    13,490

    Re: Multiple core processors

    Quote Originally Posted by John E View Post
    So the more threads you create (on a single CPU) the less efficiently each of them runs.
    Not necessarily. It depends on what the threads are doing. For example, if you have a long running operation where several portions of the operation have to wait on external events and can be run in parallel, splitting these portions into multiple threads may improve performance even on a single proc/single core machine.

    As far as thread scheduling, in general, let the OS thread scheduler do it's job. It will spread threads over multiple processors as it sees fit.

    In your specific case, it may not spread it over because the scheduler decides not to (doesn't want to peg all cpu's at 100%) or it could be that the third party plug-in are pinning the threads.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  





Click Here to Expand Forum to Full Width

Featured