Researching File Formats

[Xatrix]

Hello, gentlemen,

I'm not sure if I'm posting it in the right section, but at least it's better to post it here than in some specific forum because of the nature of the question.

I'm not even sure if the word "researching" describes the whole problem well. But I definately know what I want.

For a long time I've been trying to figure out how people make programs (utils, tools, etc.) for already released applications. Let's take games as an example, because games got me interested to study such subject.

Assume we have some game which can read some texture files (apart from far more complicated files such as 3D geometry). Usually, the files which can be simply modified or "stolen" are somehow encryped, basically, companies write their own file formats based on their own or some published algorythm. For example the original file was a normal TGA 32-bit file. Then they made their own util which converts the TGA file into another file of their own format, and the output is something like a TEX file. Now, the game itself knows this file format, and can decipher it back to a normal form if needed.

The question is. How come there are always some really tallanted people who can write some sort of reverse converters, i.e. they can generate an original TGA from a TEX file, or even far more complicated stuff, which involves 3D. Just by looking at a HEX view of the TEX file will lead you nowhere. I've spent a lot of time surfing the net for some info, I've found tons of stuff on everything but what I was searching for. Not a single tutorial on how to decode the "unknown" file format. When I do a google search, all there is, is already published file formats, such as JPEG, PCX, whatever, but no info on how to "research" the file format which has not been officially or unofficialy posted. The only info I could find on the net was about how to crack programs,,, and, well, naturally, I've spent a lot of time learning this stuff hoping that it will help me in deciphering file formats. Heck, I now even understand assembly language more than I did before. Such software as disassemblers, hex-editors, debuggers, registry and file monitors are now always present on my HDD. Having started all this about 2 years ago, I still can't figure this out. I'm just lost. I've read so much and that still doesn't help me achieve my goals. I've always wanted to write utilities and tools which would give users more flexibility... like for example to such people as modders if you know what I mean, I myself am a modder too, i.e. if I can't mod a game which is of interest to me, I'm not interested in it anymore... which means if I do write some tools eventually - it will become modable, and I will be happy! ... Once I know the format, I can write utils easily, I know several high-level programming languages, but I've never been able to find out the file format on my own

If any of you could direct me in the right direction it would be very much appreciated.

Best regards and happy holidays!

Sincerely,
Oleg