how can i save sth in FAT12 file system

[Master.]

hello all , im trying to save a file in a fat12 structure , but cant find any info on how to do it .
to achieve this , i need to find free cluster , and hence i should search the fat ,
in know how to find free cluster but for saving a cluster in fat12, i have no idea how i should proceed .
you know , i dont know exactly how to go about it . !
i mean when we want to load a value from a cluster , we check to see if its even or odd ( the cluster i mean), if its even we and it with 0000111111111111b and if its odd we shift the value 4 times to right ( meaning we want the high twelve bits) .
so do i have to consider this while saving two ? i mean should i do the same thing while saving or i should do it reversely ! i mean for even clusters instead of shr , i actually do shl, 4 times ( so that when loading when we shr 4 times , we get the actual value )
and for 'and'ing i dont know !
can anyone help me on this?

[VictorN]

Are you going to develop your own drivers for FAT12?
Or your own OS?

[Eri523]

Hi Master., nice to hear from you again!

So the problem you have here is with the endian-ness of the FAT12 entries. (And again I had to look that up myself too.) FAT12 is consequently little-endian, just like the Intel CPU is. That means you have to do the and when you want the even cluster (2n) and do the shift for the odd one (2n+1).

On first sight I wondered how you wanted to get through with shifting by only four bits. But of course that's ok when you only consider the 16 most significant bits of the 24-bit value containing two FAT entries.

You may want to lock all that nitty-bitty stuff up into some C++ class and try not to think about that any more once you got that working. You can simplify your boot process by enforcing that your stage two boot code is not fragmented on the boot disk. (And I doubt that it would be possible to handle a potentialy fragmented file within the 512 bytes of the first stage boot code at all.) You can further simplify by enforcing that the stage two boot code always starts at cluster #2. (And as far as I remember, DOS used at least the first of these two simplifications.)

And while we're at it, have a look at the way DOS used to find free clusters for allocation: Upon boot time, DOS sets up an index into the FAT in memory, pointing to the first available cluster. When that cluster gets allocated, the index is advanced to the next free cluster. And this is the only operation done on that index until shutdown (more likely) or reaching the end of the volume (less likely in most cases). Maybe the intention in doing it that way was to minimize head movement, but in fact it is one of the major reasons for rapid fragmentation of FAT volumes. Especially when doing concurrent writes on more than a single file on the same volume, they are guaranteed to get fragmented.

And yes, Victor, he is in fact writing his own OS. More on that topic can be found here.

[Master.]

Are you going to develop your own drivers for FAT12?
Or your own OS?

yes a tiny experimental os which serves educational purposes this os uses fat12 as its file system .

Hi Master., nice to hear from you again!

So the problem you have here is with the endian-ness of the FAT12 entries. (And again I had to look that up myself too.) FAT12 is consequently little-endian, just like the Intel CPU is. That means you have to do the and when you want the even cluster (2n) and do the shift for the odd one (2n+1).

On first sight I wondered how you wanted to get through with shifting by only four bits. But of course that's ok when you only consider the 16 most significant bits of the 24-bit value containing two FAT entries.

You may want to lock all that nitty-bitty stuff up into some C++ class and try not to think about that any more once you got that working. You can simplify your boot process by enforcing that your stage two boot code is not fragmented on the boot disk. (And I doubt that it would be possible to handle a potentialy fragmented file within the 512 bytes of the first stage boot code at all.) You can further simplify by enforcing that the stage two boot code always starts at cluster #2. (And as far as I remember, DOS used at least the first of these two simplifications.)

And why we're at it, have a look at the way DOS used to find free clusters for allocation: Upon boot time, DOS sets up an index into the FAT in memory, pointing to the first available cluster. When that cluster gets allocated, the index is advanced to the next free cluster. And this is the only operation done on that index until shutdown (more likely) or reaching the end of the volume (less likely in most cases). Maybe the intention in doing it that way was to minimize head movement, but in fact it is one of the major reasons for rapid fragmentation of FAT volumes. Especially when doing concurrent writes on more than a single file on the same volume, they are guaranteed to get fragmented.

And yes, Victor, he is in fact writing his own OS. More on that topic can be found here.

hi there Eri523
same here
how do you know all of those information concerning dos ? :-/
aside from that i dont have any problem with finding free clusters , as i only work in simulated environments ( , im using Virtual floppy disk plus bochs to test my stuff , if need to test it on real system i use cool disk( flash drives)

and thanks for the tip, i actually wrote the reverse ( shifting and 'and'ing stuff) ! and i now understand why my kernel never loaded!

and about that C++ class , yeah , i dont think i get there that fast , there are alot of stuff to cover before going for coding in c++.
thanks again and i'm waiting for your answers .

[VictorN]

...
And yes, Victor, he is in fact writing his own OS. More on that topic can be found here.

The good luck to OP!
And I'll give up...

[Eri523]

how do you know all of those information concerning dos ? :-/

I used it myself for many years (and not only in an experimental manner).

and thanks for the tip, i actually wrote the reverse ( shifting and 'and'ing stuff) ! and i now understand why my kernel never loaded!

So you used some kind of 3rd party file system driver to set up your virtual boot disk? Because when you reverse the byte order in both writing and reading, it doesn't matter as long as your disk image doesn't come into contact with any software that does it the right way 'round.

[Master.]

I used it myself for many years (and not only in an experimental manner).



So you used some kind of 3rd party file system driver to set up your virtual boot disk? Because when you reverse the byte order in both writing and reading, it doesn't matter as long as your disk image doesn't come into contact with any software that does it the right way 'round.

there is no 3rd party file system driver .
i have not yet implemented the write method ! there was(is) only a read method ( which just now i figured i wrote it backward !oO )
im stuck in writting and dotknow how i should go about it , i mean before saving the value , if the cluster is even and or if it is odd what should i do with the value /?
if i get this im done here .
( for odd cluster , i shl this time 4 times right?, but what should i do to the even clusters? )

[Eri523]

im stuck in writting and dotknow how i should go about it , i mean before saving the value , if the cluster is even and or if it is odd what should i do with the value /?
if i get this im done here .
( for odd cluster , i shl this time 4 times right?, but what should i do to the even clusters? )

IMO you should always start and finish working on a complete 24-bit pair of cluster entries, in a 32-bit register. This is for convenience and in turn reduces the probability of bugs. Don't worry about shifting by 4 or 12 bits: At least form the Pentium-class processors upward, the number of clock ticks taken by a shift instruction is constant, in particular it's not proportional to the shift distance. (I think this is because the processor uses a cirquitry called barrel shifter for that purpose.)

Then, when you want to access the even-numbered cluster, you would and the value by 0xFFF. For the odd-numbered cluster you would do a right shift by 12 bits. When you're done, you would do a left shift (only for odd-numbered clusters) and finally or the two cluster entries of the pair together again, of course after masking out the old value.

If memory is no concern, you may consider holding the entire FAT in memory for even more convenience. (AFAIK, DOS did not do that, simply because the FAT of a hard disk could be quite large, compared to the small amount of base memory that DOS had to come along with.) But if you do so, don't forget to flush dirty FAT sectors back to disk from time to time. The ideal strategy with respect to file system integrity would simply be flushing the FAT whenever you write the actual cluster data to disk, but this might give you a significant performance penalty, in particular on a real physical magnetic drive. You most certainly will have to find some balance between performance and safety.

So far for now... HTH

[Master.]

IMO you should always start and finish working on a complete 24-bit pair of cluster entries, in a 32-bit register. This is for convenience and in turn reduces the probability of bugs. Don't worry about shifting by 4 or 12 bits: At least form the Pentium-class processors upward, the number of clock ticks taken by a shift instruction is constant, in particular it's not proportional to the shift distance. (I think this is because the processor uses a cirquitry called barrel shifter for that purpose.)

Then, when you want to access the even-numbered cluster, you would and the value by 0xFFF. For the odd-numbered cluster you would do a right shift by 12 bits. When you're done, you would do a left shift (only for odd-numbered clusters) and finally or the two cluster entries of the pair together again, of course after masking out the old value.

If memory is no concern, you may consider holding the entire FAT in memory for even more convenience. (AFAIK, DOS did not do that, simply because the FAT of a hard disk could be quite large, compared to the small amount of base memory that DOS had to come along with.) But if you do so, don't forget to flush dirty FAT sectors back to disk from time to time. The ideal strategy with respect to file system integrity would simply be flushing the FAT whenever you write the actual cluster data to disk, but this might give you a significant performance penalty, in particular on a real physical magnetic drive. You most certainly will have to find some balance between performance and safety.

So far for now... HTH

thank you very much dear Eri523 , i really appreciate that .
but i guess in real mode we only have access to 16 bit registers , and thus working with 16 bit is practical ,do we have access to the full 32 bit registers in real mode?
and about FAT , i'm currently doing what you have just suggested, actually memory is of no concern to me .
i wonder how current file systems operate , i mean what do they do to keep FAT synced !

[Eri523]

thank you very much dear Eri523 , i really appreciate that .

You're still welcome. You always are.

but i guess in real mode we only have access to 16 bit registers , and thus working with 16 bit is practical ,do we have access to the full 32 bit registers in real mode?

According to Intel's processor manual, operand size overrides are available in real mode, and so you can use 32-bit registers. Of course this will only work if your assmbler supports it (don't have too much experience with post-16-bit assemblers) and you can rely on that your OS will never be run on something before a 386.

i wonder how current file systems operate , i mean what do they do to keep FAT synced !

I don't know much about recent file systems either. I simply never had the need to know that. But IMO the strategy for updating the FAT (or whatever it is called in another file system) usualy is not part of the definition of the file system itself, but instead is left to the people that write the file system drivers.

BTW: A few months ago, I read an article about an amazing new file system. (Don't remember the name by now, but I could look it up.) I think it was for Linux, that IMO already has the broadest spectrum of file system support anyway. This file system was entirely constructed as a tree structure. It had some nice features like transaction safety (well, not that uncommon at all) and the ability to do roll-backs to any state since the creation of the file system (now that is uncommon IMO). Just a little peek over the fence...

CU

[Master.]

You're still welcome. You always are.



According to Intel's processor manual, operand size overrides are available in real mode, and so you can use 32-bit registers. Of course this will only work if your assmbler supports it (don't have too much experience with post-16-bit assemblers) and you can rely on that your OS will never be run on something before a 386.



I don't know much about recent file systems either. I simply never had the need to know that. But IMO the strategy for updating the FAT (or whatever it is called in another file system) usualy is not part of the definition of the file system itself, but instead is left to the people that write the file system drivers.

BTW: A few months ago, I read an article about an amazing new file system. (Don't remember the name by now, but I could look it up.) I think it was for Linux, that IMO already has the broadest spectrum of file system support anyway. This file system was entirely constructed as a tree structure. It had some nice features like transaction safety (well, not that uncommon at all) and the ability to do roll-backs to any state since the creation of the file system (now that is uncommon IMO). Just a little peek over the fence...

CU

Thank you .
thats great . i've been coding in Masm 5.10! ( ancient i know )
and also i've been coding in Nasm recently (nasm 0.98.35) , but im kinda more familiar with our ancient compiler , thats why i started using masm 5.10 in the first place.but i think i can easily port what i wrote in masm 5.10 to nasm and use what you have suggested .
it seems Nasm is superior to masm in so many ways , open source , cross platform,cuter syntax,...
and about FAT , how much difference is there between Fat12 and Fat16 in terms of reading/saving in them(reading/saving values from/in FAT) ?
i mean do we have those restrictions in fat16 too? shifting n times to right and stuff ? cause in Fat16 we are utilizing the whole 2 Bytes ! whereas in fat12 we are using only 12 bits .
implementing a Fat16 driver seems more convenient compared to fat12. right?
and one more question , is there any problem if i format a floppy disk with fat16?
fat 16 can address those whole 2880+ sectors of a floppy disk right? if so why didnt people prefer fat16 over fat12 after dos time and still floppy disk are formatted in fat12?


and that new file system you just mentioned , i really like to see and use it .
by the way , making a new fie system is trivial right? only the file system driver programming section is important and hard to achieve am i right? i think making a new file system is more of achieving an optimized implementation of the driver rather than making it itself in the first place, am i right?.

thanks

[Eri523]

You really know Masm 5.10? I think this one was really ancient 16-bit stuff. It might not even support these operand size overrides at all, so you might consider swithcing to a more recent one.

I really am not familiar with Nasm, but it looks to be some kind of relevant. I'm gonna teach a fellow developer assembly, and Nasm might be a good point to start that. (Of course this is somehow mutual, just like the entire forum here works: I will teach him assembly and he lends me a hand on VBA.)

In fact, what you can see here and here is mainly a preparatiion for that assembly lesson: The core loop of that VBA code is meant to be replaced by an assembly DLL in the end. And both of us can learn something while doing that... (Of course it could be done in C++ pretty fast as well, but no one of us needs or wants to learn that... )

And about FAT12 vs. FAT16, I don't think they can be considered really similar from the implementation point of view. FAT16 can be handled quite much easier, and the only clash you might encounter is when you try to support all of the FAT file systems at once.

And no, designing a file system is far away from trivial. Although much of the stuff about that involves the development of the file system drivers, defining a good file system is quite complicated. Implementing a file system driver is quite complicated, but designing The file system itself (as long as it's really more advanced compared to FAT) deserves at leat as much merits as inventing the file system drivers themselves.

Ok, now I really have to go to bed...

[Master.]

You really know Masm 5.10? I think this one was really ancient 16-bit stuff. It might not even support these operand size overrides at all, so you might consider swithcing to a more recent one.

I really am not familiar with Nasm, but it looks to be some kind of relevant. I'm gonna teach a fellow developer assembly, and Nasm might be a good point to start that. (Of course this is somehow mutual, just like the entire forum here works: I will teach him assembly and he lends me a hand on VBA.)

In fact, what you can see here and here is mainly a preparatiion for that assembly lesson: The core loop of that VBA code is meant to be replaced by an assembly DLL in the end. And both of us can learn something while doing that... (Of course it could be done in C++ pretty fast as well, but no one of us needs or wants to learn that... )

And about FAT12 vs. FAT16, I don't think they can be considered really similar from the implementation point of view. FAT16 can be handled quite much easier, and the only clash you might encounter is when you try to support all of the FAT file systems at once.

And no, designing a file system if far away from trivial. Although much of the stuff abut that involves the development of the file system drivers, defining a good file system is quite complicated. Implementing a file system driver is quite complicated, but designing The file system itself (as long as it's really more advanced compared to FAT) deserves at leat as much merits as inventing the file system drivers themselves.

Ok, now I really have to go to bed...

tanx.
but talking about supporting different FATs, how am i supposed to know which kind of FAT im dealing with ? i've read that the Fatname located in BPb should not be used as a sign for the current file system ,because it might have garbage value , Microsoft guys never used it in their FAT drivers as far as i know , instead they used the total number of sectors as an indicator of FAT.
im my case what should i do , suppose there is a new file system , should i look at that bpb ?
Good night

[Eri523]

but talking about supporting different FATs, how am i supposed to know which kind of FAT im dealing with ? i've read that the Fatname located in BPb should not be used as a sign for the current file system ,because it might have garbage value , Microsoft guys never used it in their FAT drivers as far as i know , instead they used the total number of sectors as an indicator of FAT.

I agree that relying on that ASCII signature of the FAT type in the BPB might be dangerous. But I think you can check whether it contains a valid FAT name and only resort to other mechanisms if it doesn't.

I don't really know how MS did it, but as long as nothing beyond FAT16 is involved, the media descriptor could be used a an indicator too. And as long as it can be assumed that the MS formatter makes its decision about whether to format a hard disk in FAT16 or FAT32 depends solely on the total number of sectors, your software can do it the same way. I don't remember that the MS formatter gave the user the option to chose between the two FAT types, so I think this assumption can be made. But then I don't know where exactly the disk size limit between FAT16 and FAT32 is.

im my case what should i do , suppose there is a new file system , should i look at that bpb ?

You mean some non-FAT file system? I don't think you could use the BPB as a safe indicator for this, when it is not even reliable for FAT-type systems.

I think in one of your earlier posts you raised the question whether it might be possible to format floppies with FAT16. Of course this might be done theoretically, but it would be dramatically incompatible as it is unlikely that any standard software would expect this.

[Master.]

I agree that relying on that ASCII signature of the FAT type in the BPB might be dangerous. But I think you can check whether it contains a valid FAT name and only resort to other mechanisms if it doesn't.

I don't really know how MS did it, but as long as nothing beyond FAT16 is involved, the media descriptor could be used a an indicator too. And as long as it can be assumed that the MS formatter makes its decision about whether to format a hard disk in FAT16 or FAT32 depends solely on the total number of sectors, your software can do it the same way. I don't remember that the MS formatter gave the user the option to chose between the two FAT types, so I think this assumption can be made. But then I don't know where exactly the disk size limit between FAT16 and FAT32 is.



You mean some non-FAT file system? I don't think you could use the BPB as a safe indicator for this, when it is not even reliable for FAT-type systems.

I think in one of your earlier posts you raised the question whether it might be possible to format floppies with FAT16. Of course this might be done theoretically, but it would be dramatically incompatible as it is unlikely that any standard software would expect this.

thank you very much again.
whats the use of bpb then ? when its not even considered to be storing valid stuff ?
anyway , if happened to format a floppy disk , i shouldnt have any problem theoretically ? because any software which is to work with a storage device like floppy disk should look at the bpb , and then find out about the structure there, as far as i know windows xp for example has drivers for both Fat12 and Fat16 ( fat32 aside ) and then it knows how to work with them , right? then whats avoiding us to actually format floppies with fat16 ! ?

and another question , if i want to build a fat12 filesystem , is it enough to only format the bootsector properly ? i mean i write the fat12 specific data into the bpb ? will it magically make the whole FATs Root directory and data area sections for me ? or i have to make those sections myself ? or making that bpb will make those sections but they should be initialized properly ? ( i mean Fats and Root directory entries )