How to raise priority of MIDI input callbacks? Thread pools etc.
Hi,
I'm the author of a realtime MIDI software called ChordEase which makes use of the MIDI aspects of the multimedia API, specifically MIDI input callbacks. In XP and before, these callbacks originated in the kernel and therefore had realtime priority by definition, but from Vista on, they originate in thread pool threads, and have a priority of zero. This is a problem because at priority zero they can be blocked by the GUI thread, causing serious latency, and I have proved that such blocking occurs.
I have experimented with raising the callback thread priority, using either of the following methods: 1) calling SetThreadPriority within the MIDI input callback function, and then setting a flag so that it isn't done repeatedly, or 2) creating a DLL that catches thread creation via DLL_ATTACH_THREAD in DllMain, and calling SetThreadPriority there. The first method is slightly wasteful since the flag has to be tested for every MIDI input event, but it also has the advantage of only affecting the MIDI input threads, whereas the second method affects all threads in the pool regardless of what they're used for. Neither method appears to cause any harmful effects but they make me nervous*. Other possible methods would include 3) using the thread pool API to raise the priority of the pool (assuming I could gain access to the pool handle somehow), or 4) permanently lowering the priority of the GUI thread, which I'm very reluctant to do because of the risk of unintended consequences.
I'm assuming the MIDI input callbacks are using threads in the default thread pool though I haven't actually proved this. Assuming that's so, are these threads private to my application, or is my application sharing them with other applications? Is there a safer way to achieve the result of increasing the priority of MIDI input callbacks? It's incredibly frustrating that MS would change the behavior of MIDI input callbacks so drastically without even telling anyone, but that's how it goes!
Best regards,
Chris Korda
http://chordease.sourceforge.net/
*See for example theses warnings about changing thread pool priorities on Hari's blog:
http://blogs.msdn.com/b/harip/archiv...-priority.aspx
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
Quote:
Originally Posted by
ckorda
In XP and before, these callbacks originated in the kernel and therefore had realtime priority by definition
Origin of events has no effect on priority.
None of the windows versions has actual 'real time' priority. At best you can set your process and thread priority so high that it prevents any other apps from taking precedence (and cause all kinds of havoc while doing so if done incorreclty), but that just gives you 'top level application priority'. Yes, MS calls that "real time priority", but it gives no guarantees as to the time that can pass before it actually gets serviced.
Even at that priority it it still possible to find your app doesn't get any time slot allocated for several seconds. (extreme/exceptional, yes, but it can happen).
Quote:
but from Vista on, they originate in thread pool threads, and have a priority of zero.
I don't recall any MIDI bahaving as such, but granted, it's been a while since I did any work on that area.
In any case, hard to give any kind of advice if you aren't telling which precise part of the windows API you're referring to.
My old MIDI apps still seem to work exactly the same, so I'm not sure what you're alluding to.
Quote:
This is a problem because at priority zero they can be blocked by the GUI thread, causing serious latency, and I have proved that such blocking occurs.
Might wanna get your terminology correct... if they are blocking, then that is entirely caused by your own code.
if they aren't getting timeslices from the scheduler, then that may still be a problem of your own code. Or it may simply be a case of inadequate hardware. In any case, it is "normal" to create worker threads and thread pools with a lower priority than your main UI thread, this is the preferred mode of operation.
There are cases where you need to raise priorities of individual threads.
Needing to raise priority of the entire process, typically points towards underlying design issues. There's a few cases where it's necessary, but more often than not, it's being abused.
Quote:
2) creating a DLL that catches thread creation via DLL_ATTACH_THREAD in DllMain, and calling SetThreadPriority there.
Bad idea, because threads may not be created for the callbacks so you'll be raising prio for threads that don't need it and make the problem worse. Just because you don't notice it on your PC doesn't mean the problem doesn't exist.
Also doing this in DLLmain is a bad idea anyway.
Quote:
3) using the thread pool API to raise the priority of the pool (assuming I could gain access to the pool handle somehow)
without knowing which API's your using, hard to give any real help.
TYpically speaking, any API that operates with threads should either offer means to adjust priority fairly obviously, or it it fully intended that you don't mess around with it entirely.
Quote:
4) permanently lowering the priority of the GUI thread, which I'm very reluctant to do because of the risk of unintended consequences.
Again... bad idea... :)
Quote:
are these threads private to my application, or is my application sharing them with other applications?
threads are Always part of a process.
there is a chance the threads resice in a host application and you are given an API to inteface with them (this is how out of process COM would do it). But then that excludes direct access to your app's variables and functions from the callback.
Something tells me this isn't going to be the case, but again, "hard to tell without more info".
Quote:
Is there a safer way to achieve the result of increasing the priority of MIDI input callbacks? It's incredibly frustrating that MS would change the behavior of MIDI input callbacks so drastically without even telling anyone, but that's how it goes!
AFAIK, then haven't.
but hard to say :)
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
Thanks for your reply.
When I said "realtime priority" what I meant was THREAD_PRIORITY_TIME_CRITICAL, AKA priority 15. Sorry for the confusion. The simple fact is that if you display the priority of a MIDI input callback (MidiInProc) in XP, via GetThreadPriority(GetCurrentThread()), you will get 15. If you do this in Windows 7 (and likely Vista and 8 too), you will get zero. This is a major change and it isn't documented anywhere AFAIK.
As a result of of this change, the MIDI event callback threads (one for each input device) and the GUI thread all have the same priority. For simplicity let's give a scenario on a dual-core machine, with no hyperthreading. Suppose both cores are busy doing priority zero work. In XP, if a MIDI input event is received, the MIDI input event will be handled immediately (or close enough), because the MIDI callback runs at time critical (15) which certainly preempts priority zero. In Windows 7, the MIDI event will not necessarily be handled immediately, because the callback is only running at priority zero and therefore doesn't preempt until one of the running threads uses up its time slice.
Callbacks are different from worker threads. The whole point of MIDI event callbacks is that they're supposed to occur at high priority, so that the app can respond to MIDI events quickly. Servicing them with normal priority worker threads (from a thread pool) defeats this purpose. My guess is that MS went on a rampage and decided that everything had to be implemented with the latest thing (thread pools, added in Vista) but didn't think through the consequences in this case because they don't care about MIDI anymore (e.g. deprecation of DirectMusic).
Quote:
because threads may not be created for the callbacks so you'll be raising prio for threads that don't need it
Yes that's a risk but I have control over the important worker threads that my app launches. I can arrange things so that all the threads I care about are not affected, and only the threads created automatically by the system (i.e. the thread pool threads) are affected. I have monitored this using a thread viewer and yes there are occasional transient threads but the main use of the thread pool in my app is for MIDI input callbacks. So it might be a risk worth taking.
Quote:
Also doing this in DLLmain is a bad idea anyway.
Why exactly is it a bad idea? It works for me and has an advantage over setting the priority from the MIDI input callback: it only gets done when the thread launches, so there's no need to waste time testing a bHaveWeRaisedPriorityYet flag for every single MIDI input event.
Quote:
API that operates with threads should either offer means to adjust priority fairly obviously
When MS made this change, they could have thoughtfully added a SetMIDIInputCallbackPriority to the multimedia API, but they didn't.
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
Quote:
Originally Posted by
ckorda
When I said "realtime priority" what I meant was THREAD_PRIORITY_TIME_CRITICAL, AKA priority 15. Sorry for the confusion. The simple fact is that if you display the priority of a MIDI input callback (MidiInProc) in XP, via GetThreadPriority(GetCurrentThread()), you will get 15. If you do this in Windows 7 (and likely Vista and 8 too), you will get zero. This is a major change and it isn't documented anywhere AFAIK.
As a result of of this change, the MIDI event callback threads (one for each input device) and the GUI thread all have the same priority. For simplicity let's give a scenario on a dual-core machine, with no hyperthreading. Suppose both cores are busy doing priority zero work. In XP, if a MIDI input event is received, the MIDI input event will be handled immediately (or close enough), because the MIDI callback runs at time critical (15) which certainly preempts priority zero. In Windows 7, the MIDI event will not necessarily be handled immediately, because the callback is only running at priority zero and therefore doesn't preempt until one of the running threads uses up its time slice.
Callbacks are different from worker threads. The whole point of MIDI event callbacks is that they're supposed to occur at high priority, so that the app can respond to MIDI events quickly. Servicing them with normal priority worker threads (from a thread pool) defeats this purpose. My guess is that MS went on a rampage and decided that everything had to be implemented with the latest thing (thread pools, added in Vista) but didn't think through the consequences in this case because they don't care about MIDI anymore (e.g. deprecation of DirectMusic).
You may have to seriously read up on how the scheduler works and what SetThreadPriority() does and how the whole priority affects running code, because you're pretty much dead wrong on all accounts in the above.
SetThreadPriority() sets a priority level of individual threads INSIDE YOUR APP ONLY, and RELATIVE to priority of the process.
The OS combines process priority with thread priority along with some other bits and bobs (honestly this alone takes a several chapters in an in-depth book so don't expect me to go into the details here) to calculate a system wide priority level for the thread.
those systemwide priorities get adjusted from time to time to prevent apps grom hogging and preventing other threads from never getting any time slice (within some parameters, both hogging and starvation are both intentional in some cases).
Higher prio threads DO NOT PREEMPT LOWER PRIORITY THREADS !!! (read that again just to be sure, make it into a mantra and chant it every morning).
Threads do not preempt other threads. EVER.
THe scheduler decides to give a thread a timeslice based on priority, available threads, and basically however it feels like (again... 'chapters' :p), the thread gets to complete it's timeslice regardless of whether higher priority threads are available or have become available meanwhile, a thread can relinquish the remainder of it's timeslice (by calling one of the WaitFor...() family of functions, by creating a sync object with initial locking or by Sleep() or one of several other ways).
Time slices are quite short, so yes, it does SEEM like the low prio thread gets aborted in favour of a higher prio one, but that's not what actually happens. In fact, there are a few ways by which a thread can "buy more time" in it's timeslice and extend it, and even a lowest possible thread can do this when a highest priority thread is waiting to get serviced).
Quite a lot has changed in the scheduler from XP to Vista, In vista it uses a scheduler much more akin to how it is/was in the Windows Server build, than how it was in the low end 95/98/me/xp.
THe only way a thread can get preempted is to service a hardware interrupt, which typically gets serviced, it's job queued, and the thread that was preempted gets to continue for it's remaining time. Pending kernel queued jobs get priority over other threads.
You're actually falling into a common pitfall.
Yes, on the old mechanism your callback was called directly from the kernel, which ALWAYS runs at highest possible priority. But it wasn't because another thread was preempted, it's because in XP the pending kernel tasks got serviced at kernel level priority and that's a problem, because at that priority you can basically "buy more time" for your timeslice indefinately and prevent anything else from running. It's also a serious problem if that callback then ends up waiting for other threads that won't be at that priority and basically causing a deadlock.
On the new Vista approach, pending kernel jobs get serviced at application level priority which is much safer. But it still runs basically at the same time/moment as it did on XP... just at lower priority.
If said code has waits/sleeps (a bad idea on a callback), then yes, you may be relinquising your own timeslice yourself in favour of another higher priority thread, but that really is a design issue, not a fault of the OS.
Quote:
Why exactly is it a bad idea? It works for me and has an advantage over setting the priority from the MIDI input callback: it only gets done when the thread launches, so there's no need to waste time testing a bHaveWeRaisedPriorityYet flag for every single MIDI input event.
because DLLMain is pretty notorious for being a cause for all sorts of mayhem (even systemwide at times). You are technically restricted by what you can safely do in DllMain (read the MSDN docs on DllMain carefully). There's unfortunately not even a proper list of what you safely CAN do. I've seen enough of such issues to put question marks on anything in DllMain.
Stuff in Kernel32.dll (like SetThreadPriority) is usually safe, but even Kernel32 functions have a couple exceptions here and there.
Changing priorities liberally might end up being a serious issue though, again, it may work on your machine, but I see "threads of unknown origin" in my own apps all the time. THe kernel makes some, the anti virus does, the video driver does etc. You could make things worse or unworkable on someone's PC... if your code only needs to run on your PC, then "go ahead".
Quote:
When MS made this change, they could have thoughtfully added a SetMIDIInputCallbackPriority to the multimedia API, but they didn't.
it wasn't needed, because there really isn't any actual change for normal case scenario's. If you are having actual problems with the new approach, then you likely have a deeper design issue that only now becomes aparent. Trying to mess around with thread priorities, is a band aid solution, it's unreliable if you need this to run on a wide variety of computers and it doesn't cure the actual ailment.
LIke I said before, an app should not normally have to muck around with thread priorities. THere are exceptions, but in that case you need to be able to give a clear reasoning behind it. "because it does not work without this" is not a good reason, and indicates an underlying problem.
--
Anyways...
If this is for MidiInOpen(), then CALLBACK_FUNCTION has Always been an application level callback.
If you need this to be higher level priority (with reason), then you need to create the higher priority thread yourself and use CALLBACK_THREAD instead (suspend the thread, it'll get continued as part of the 'callback', do your work, then get back to suspended state for the next loop).
This has the advantage of being system exclusive. THe disadvantage is that it doesn't work for error notification and you'll have to fetch the event yourself).
I'm slightly confused though, the midi apps I made still run perfectly fine in unchanged, unrecompiled form as I built them several years ago, and they use CALLBACK_FUNCTION without any hickups.
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
In what source might I find support for your claim that "the thread gets to complete it's timeslice regardless of whether higher priority threads are available"? This is contradicted by Microsoft's documentation:
"If a higher-priority thread becomes available to run, the system ceases to execute the lower-priority thread (without allowing it to finish using its time slice), and assigns a full time slice to the higher-priority thread."
Scheduling Priorities
http://msdn.microsoft.com/en-us/libr...=vs.85%29.aspx
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
Hmmm strange, since that contradicts observed behaviour.
Will have to dig into this.
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
According to my observations, the OS allocates a minimum of 1 time slice per second to a thread even if it is a low priority thread and other high prio threads want to fully load all cores.
I guess MS decided this was a good idea because it allows for some graceful behavior even with crazy thread priorities.
Hyperthreading makes things more complicated because the logical cores allow low prio threads to steal CPU time even when the physical core is loaded by a higher prio thread.
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
I quote from "Windows Internals 6th Edition Part 1 by Russinovich, Solomon and Ionescu" chapter 5 page 409
"After a thread is selected to run, it runs for an amount of time called a quantum. ... A thread might not get to complete its quantum, however, because Windows implements a preemptive scheduler. If another thread with a higher priority becomes ready to run, the currently running thread might be preeemptied before finishing its time slice."
Further technical details are provided in the book.
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
Quote:
Originally Posted by
2kaud
A thread might not get to complete its quantum, however, because Windows implements a preemptive scheduler. If another thread with a higher priority becomes ready to run, the currently running thread might be preeemptied before finishing its time slice."
This is certainly consistent with my experience.
The following test demonstrates disruption of MIDI input callbacks by normal priority threads. After the hogging threads are launched, obvious and erratic MIDI latency is observed, consistent with the MIDI input callbacks being blocked and/or preempted. If the priority of MIDI input callbacks is raised to above normal priority, the test doesn't disrupt the callbacks at all.
Code:
#include <math.h>
UINT BurnThread(LPVOID param)
{
double t = 0;
for (int i = 1; i < INT_MAX; i++)
t += log(i);
printf("%f\n", t);
return 0;
}
void CMainFrame::OnTest()
{
SYSTEM_INFO si;
GetSystemInfo(&si);
for (UINT i = 0; i < si.dwNumberOfProcessors; i++)
AfxBeginThread(BurnThread, 0);
}
Based on the results of this test, I have decided to implement solution #2 which was mentioned in the original post: loading a DLL that catches thread creation via DLL_ATTACH_THREAD in DllMain, and raises the priority of any threads with priority 0 to priority 1. SetThreadPriority is in kernel32.dll and is not otherwise prohibited by either DllMain entry point or Dynamic-Link Library Best Practices. From the former: "Therefore, the entry-point function can call functions in Kernel32.dll that do not load other DLLs." Apparently (as one might expect) SetThreadPriority is a core capability of the kernel that doesn't require loading other DLLs. Yes it's not without risk, but risk/benefit trade-offs are unavoidable in programming as in life. "There is no security on this earth, there is only opportunity." -General Douglas MacArthur.
Here's the code of the ThreadBoost DLL. Note that it only affects normal threads.
Code:
BOOL APIENTRY DllMain( HANDLE hModule,
DWORD ul_reason_for_call,
LPVOID lpReserved
)
{
if (ul_reason_for_call == DLL_THREAD_ATTACH) {
HANDLE hThread = GetCurrentThread();
if (GetThreadPriority(hThread) == THREAD_PRIORITY_NORMAL)
SetThreadPriority(hThread, THREAD_PRIORITY_ABOVE_NORMAL);
}
return TRUE;
}
And, here's the corresponding code in the ChordEase application's MIDI engine:
Code:
// In Vista and later, MIDI input callbacks run at priority 0 instead of 15.
// Consequently input callbacks no longer preempt normal threads, including
// the GUI thread, and this can increase the latency of our response to MIDI
// input, particularly if all CPUs are saturated with normal priority work.
// We solve this by loading a DLL that catches thread launching, and boosts
// any threads having priority 0 (normal) to priority 1 (above normal).
OSVERSIONINFO osvi;
osvi.dwOSVersionInfoSize = sizeof(osvi); // init struct size before call
if (GetVersionEx(&osvi) && osvi.dwMajorVersion >= 6) // if Vista or later
m_ThreadBoost.LoadLibrary(_T("ThreadBoost.dll"));
I have tested the above solution and it definitely solves my problem, i.e. it prevents MIDI input callback latency in the case where all cores are busy doing priority zero work. I will be monitoring ChordEase closely over the coming weeks and months to ensure that no undesirable side effects result. Thank you all for your time.
Chris
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
Quote:
Originally Posted by OReubens
Hmmm strange, since that contradicts observed behaviour.
I think it's a terminology problem; correct me if I'm wrong, by "preempting" you seem meaning the act of interrupting a thread *during* its timeslice to pass execution *immediately* to some other thread without thread intervention ( this is probably consistent with your realtime background, ain't it ? ), this is in contrast to what it seems the more common usage of the term by meaning the *general* act of passing execution to some other thread by virtue of the scheduler authority. In your meaning of the term, I think you're right when you say that threads never preempt other threads.
Quote:
Originally Posted by 2kaud
"After a thread is selected to run, it runs for an amount of time called a quantum. ... A thread might not get to complete its quantum, however, because Windows implements a preemptive scheduler. If another thread with a higher priority becomes ready to run, the currently running thread might be preeemptied before finishing its time slice."
actually, this does not say "why" preemption occurs, it just says that it occurs when some other thread is "ready", that could simply mean that the former thread has simply given away his timeslice ( a wait, etc.. ) prematurely, then the scheduler decides to switch to a high priority thread instead of round robin ( possibly resulting in starvation of the former ). This is how I interpret that quote ( but I can be wrong :) ).
Quote:
Originally Posted by ckorda
I have tested the above solution and it definitely solves my problem,
anyway, instead of messing with priorities you could let the OS decide for you, by using the Multimedia Class Scheduler Service. Under the hood it may do more or less the same thing but your code will be more robust in the end ( what if the OS changes his scheduling policy again in the future ? your code will have higher proabability of retaining a consistent behavior in this latter case ).
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
Quote:
Quote:
Originally Posted by 2kaud
"After a thread is selected to run, it runs for an amount of time called a quantum. ... A thread might not get to complete its quantum, however, because Windows implements a preemptive scheduler. If another thread with a higher priority becomes ready to run, the currently running thread might be preeemptied before finishing its time slice."
actually, this does not say "why" preemption occurs, it just says that it occurs when some other thread is "ready", that could simply mean that the former thread has simply given away his timeslice ( a wait, etc.. ) prematurely, then the scheduler decides to switch to a high priority thread instead of round robin ( possibly resulting in starvation of the former ). This is how I interpret that quote ( but I can be wrong ).
As I indicated in my post #8, my quote came from the book "Windows Internals 6th Edition Part 1 by Russinovich, Solomon and Ionescu" chapter 5 page 409. There are many, many pages devoted to the topic of thread scheduling but my understanding of this (which could of course be wrong) is that if a priority change has occurred that makes the current standby or running thread no longer the highest priority ready thread on its selected logical processor the current thread is suspended during its quantum and the higher priority ready thread is then run - preemptive multitasking.
There are also scheduling issues due to multiprocessor systems as Windows attempts to schedule threads on the most optimal processor for the thread, taking into account the thread's preferred and previous processors as well as the configuration of the multiprocessor system. Although Windows attempts to schedule the highest-priority runnable threads on all available CPUs, it guarantees only to be running one of the highest priority threads somewhere.
Also see http://msdn.microsoft.com/en-us/libr...=vs.85%29.aspx which states that a common reason for a context switch is that a thread with a higher priority has become ready to run.
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
[QUOTE=2kaud;2163477]There are many, many pages devoted to the topic of thread scheduling but my understanding of this (which could of course be wrong) is that if a priority change has occurred that makes the current standby or running thread no longer the highest priority ready thread on its selected logical processor the current thread is suspended during its quantum and the higher priority ready thread is then run - preemptive multitasking.[quote]
Well that is what is confusing me, because in all the tests I've conducted. I've never actually observed this behaviour. Where a low prio thread is being preempted/stopped in favour of a higher priority thread before the lower priority thread's quantum ran to completion.
The only 2 exception to that rule is when a hardware interrupt occurs and when the kernel memory organizer thread needs to run (you're into deep trouble already if this is an issue for you).
Yes, I know it's there, but I never observed this to actually happen. It made me (maybe inaccurately) assume that it's just one of the other many minor mistakes/incosistencies in MSDN and that it should really be "when a thread with the highest priority" (being either an IRQ or the System kernel thread).
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
Quote:
Originally Posted by
ckorda
This is certainly consistent with my experience.
The following test...
Any thread that does I/O like this is just a terrible way to conduct a proper test.
For starters, I/O is going to cause waits at some point when the system tries to catch up.
console access on anything other than your main thread is just a downright bad idea anyway.
Quote:
Based on the results of this test, I have decided to implement solution #2 which was mentioned in the original post: loading a DLL that catches thread creation via DLL_ATTACH_THREAD in DllMain, and raises the priority of any threads with priority 0 to priority 1.
If you only need that to run on YOUR PC. Then by all means, a cludge that works might be more feasible than trying to find the actual cause and solve it properly.
If you need this to run on "every" pc out there. Then it's going to fail somewhere. most likely on a PC with a single core, or with just 2 cores (intel I3's or celerons come to mind). Or on any windows that makes more threads like this than it has actual cores, and you'll be raising priority of more threads than cores and basically totally nullify the effect.
Re: How to raise priority of MIDI input callbacks? Thread pools etc.
Quote:
Any thread that does I/O like this is just a terrible way to conduct a proper test.
You're mistaken yet again. The purpose of that thread function is merely to burn CPU time for a few minutes, which it does admirably. The printf ensures that the loop (which repeatedly calculates the log function) doesn't get optimized away due to non-usage of the result.
Quote:
a cludge that works might be more feasible
I'll take my own experience over ill-informed conjecture any day.
