Need guidance - how to build an embedded device

I was thinking, which is always dangerous...

I was thinking about small embedded device. There area million little computerized devices out there, so I was wondering what it takes to build one. Where would a person start?

I know there are raspberry and other computer boards but what if I wanted to build something smaller? If I wanted to build a little device size of a quarter with LEDs and a Accelerometer, how or where would I go about that? If I wanted to code, would I be able to use C# or would licensing be costly versus using C on Linux?

Any one have thoughts or background on this?

Thanks!