How can I get two threads to run simultaneously in a single process?
Experimenting with MS Text to Speech (TTS) I built a simple dialog that speaks a greeting when a button is pressed. I wish to incorporate an avatar that mouths the spoken words. Without getting into all the details of how to match the phenomes with the facial movements, I just included an avatar that moves it's mouth.
I cannot figure out how to coordinate the speech and the avatar movement. My first attempt was to try to run them both together, but that doesnt work -- whichever one runs first must complete before the other runs. My second attempt was to interleave a spoken word with a change in bitmap image -- that sort of works but is very slow and jerky. (see attached app). Perhaps a better approach would be to use multithreading and run speech and avatar moves each as separate worker threads. Sounds good, but I do not know how to do that. And after spending many hours reading books and scouring the web for information on Win API and MFC multithreading, I am more confused than ever.
I would appreciate your thoughts on the problem and how you might go about trying to solve it. Thanks.
But the problem remains of how to coordinate the bitmap viseme images with the speech visemes, assuming that we have a collection of viseme bitmap images that correspond to the 21 or so possible viseme events. Any idea how this might be accomplished ?
Last edited by Mike Pliam; July 18th, 2012 at 02:51 PM.
Re: How can I get two threads to run simultaneously in a single process?
It's my understanding that the TTS SDK includes a sample program that does exactly what you are attempting, using a "talking microphone" with different mouth shapes for each different viseme. Look in the SDK for files like mic_eyes_narrow.bmp and mic_mouth_2.bmp
If you can't find it, then it seems that this article explains the SDK sample code: "Converting Text-To-Speech and and using mouth motion animation" by Agus Kurniawan at http://www.codeproject.com/Articles/...ing-mouth-moti
Re: How can I get two threads to run simultaneously in a single process?
Originally Posted by Mike Pliam
Any idea how this might be accomplished ?
See your modified sample attached.
Code:
case SPEI_VISEME :
// Viseme was determined by synthesis engine. The high word of wParam is the duration, in milliseconds,
// of the current viseme. The low word is for the next viseme of type SPVISEMES. The high word of lParam
// is the viseme feature defined in SPVFEATURE. This value will be zero if the current viseme is not primary
// stress or emphasis. The low word of lParam is the current viseme being synthesized.
// wmId = LOWORD(wParam);
// wmEvent = HIWORD(wParam);
//_RPT1(0, "wmId = %d\n", wmId);
//_RPT1(0, "wmEvent = %d\n", wmEvent);
pVoice->GetStatus( &eventStatus, NULL );
ULONG viseme;
viseme = eventStatus.VisemeId;
_RPT1(0, "viseme = %d\n", viseme);
// now you know what viseme to render
// animation goes here below
// decide what bitmap to draw
bmpIdx = (viseme > 11); // simple animation:
// consonants are above 11,
// vowels are up to, therefore,
// mouth4.bmp will go for vowels,
// and test.bmp for consonants
// make your window to re-paint the sprite region
InvalidateRect(hWnd, NULL /* in fact you need your sprite rect here*/, FALSE);
break;
default:
break;
}
SpClearEvent( &eventItem );
}
break;
case WM_CREATE:
// pre-load sprite images
hBitmap[0] = (HBITMAP)LoadImage(hInst, L"mouth4.bmp", IMAGE_BITMAP, 0, 0, LR_LOADFROMFILE);
hBitmap[1] = (HBITMAP)LoadImage(hInst, L"test.bmp", IMAGE_BITMAP, 0, 0, LR_LOADFROMFILE);
break;
case WM_PAINT:
PAINTSTRUCT ps;
HDC hdc;
BITMAP bitmap;
HDC hdcMem;
HGDIOBJ oldBitmap;
hdc = BeginPaint(hWnd, &ps);
hdcMem = CreateCompatibleDC(hdc);
oldBitmap = SelectObject(hdcMem, hBitmap[bmpIdx]);
GetObject(hBitmap[bmpIdx], sizeof(bitmap), &bitmap);
BitBlt(hdc, 0, 0, bitmap.bmWidth, bitmap.bmHeight, hdcMem, 0, 0, SRCCOPY);
SelectObject(hdcMem, oldBitmap);
DeleteDC(hdcMem);
EndPaint(hWnd, &ps);
break;
Last edited by Igor Vartanov; July 19th, 2012 at 05:48 AM.
* The Best Reasons to Target Windows 8
Learn some of the best reasons why you should seriously consider bringing your Android mobile development expertise to bear on the Windows 8 platform.