Secure Memset
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 15 of 15

Thread: Secure Memset

  1. #1
    Join Date
    Aug 2005
    Location
    San Diego, CA
    Posts
    1,054

    Question Secure Memset

    I read some code recently where someone wrote their own function called secureMemset where they used a temporary volatile pointer to the memory block and executed the write operation an extra time to ensure that the data was actually assigned to the new sanitization value. The reasoning for that was that supposedly the library function memset could be completely optimized away by the compiler, depending on how you configure a compiler and whether the compiler decides that the operation was necessary. This was news to me, as I had never heard of that before. Is this also true for the std::fill and std::fill_n algorithms? I tend to prefer the C++ algorithms anyway when I write code, but I wonder if they are also vulnerable to this security problem. Would any C++ security experts out there care to answer this question and provide some more insight. If so, please advise. I read some other forums but I did not see any place where std::fill was discussed as an alternative to memset.
    http://stackoverflow.com/questions/2...memory-when-se
    https://www.securecoding.cert.org/co...sensitive+data

  2. #2
    Join Date
    Nov 2003
    Posts
    1,815

    Re: Secure Memset

    >> Is this also true for the std::fill and std::fill_n algorithms?
    Probably, unless documented otherwise. The problem is still the same, whether you use memset() or std::fill() - the compiler see's an unused stack buffer being filled with 0's at the end of it's lifetime. If the compiler can prove to itself that removing it doesn't alter program behavior, then it's legal to remove it as an optimization.

    gg

  3. #3
    Join Date
    Aug 2005
    Location
    San Diego, CA
    Posts
    1,054

    Lightbulb Re: Secure Memset

    Surely there must be something more elegant than using the volatile keyword and a manually written loop. Honestly, the aforementioned links don't do a very good job of explaining why the volatile keyword prevents optimization. Moreover, how can the compiler possibly know what is going to happen at run-time? If I try to reset pointers to zero in a destructor after destroying an object, will that be optimized away? That makes no sense to me, but the compiler could determine that operation is pointless since the object is being destroyed just as easily as it could assume that the memset isn't really necessary. Unfortunately, I have not seen a realistic example that proves the theory one way or the other.

  4. #4
    Join Date
    Jan 2006
    Location
    Singapore
    Posts
    6,316

    Re: Secure Memset

    Quote Originally Posted by kempofighter
    Honestly, the aforementioned links don't do a very good job of explaining why the volatile keyword prevents optimization.
    I believe the idea is that due to the volatile keyword, the compiler can no longer assume that there are no further accesses to the variable, just because it does not find any in the code it is given.

    Quote Originally Posted by kempofighter
    If I try to reset pointers to zero in a destructor after destroying an object, will that be optimized away?
    It could be optimised away, though of course we cannot say if it will be.
    C + C++ Compiler: MinGW port of GCC
    Build + Version Control System: SCons + Bazaar

    Look up a C/C++ Reference and learn How To Ask Questions The Smart Way
    Kindly rate my posts if you found them useful

  5. #5
    Join Date
    Oct 2008
    Posts
    1,151

    Re: Secure Memset

    I think you cannot expect a standard guaranteed solution to this problem, as it relates to the side effects of code on "real" memory whose relation with the "memory" of the abstract machine ruled by the standard is unspecified.

    So, IMHO this kind of things should be implemented at the compiler/OS support level, like using SecureZeroMemory on Windows ...
    otherwise, if not avaliable on all targets, I'd write a test system used to systematically check for required memory behavior in the final release binary ...

  6. #6
    Join Date
    Jun 2009
    Location
    France
    Posts
    2,319

    Re: Secure Memset

    AFAIK, the point of using alorithms like std::fill is that when used with POD data, the meta programming will cause the call to degenerate all the way into a very efficient memset/memcopy.

    This means that no matter what type you are using, std::fill is pretty much optimal. In theory, it could even be implemented using micro multi-threading!

    ----
    If I may ask a question though, why would you want to prevent optimization? If the data you are writing to MUST be memset, yet you never actually read that data, then wouldn't the correct solution be to rightfully declare that data volatile.

    If your data is not volatile, and you never read it, then the compiler can and will optimize way whatever you do, memset, loops, std::fill etc. For the compiler, it's all the same.

    The solution of "secure" memset is merely putting the burden of volatile declaration from the developer to the algorithm. Not a good idea, IMO.
    Is your question related to IO?
    Read this C++ FAQ LITE article at parashift by Marshall Cline. In particular points 1-6.
    It will explain how to correctly deal with IO, how to validate input, and why you shouldn't count on "while(!in.eof())". And it always makes for excellent reading.

  7. #7
    Join Date
    Aug 2005
    Location
    San Diego, CA
    Posts
    1,054

    Lightbulb Re: Secure Memset

    My biggest problem with understanding this is that I don't see how a compiler can determine whether something is read from or written to again; especially in a large scale project. I could understand the optimization happening in a small scale project with just a few files. however, many of the applications that would need to sanitize memory would be fairly complex programs. I just have a hard time getting my mind wrapped around how a compiler is going to analyze complex run-time behavior. In other words, let's say one class needs to sanitize a memory block but also provides an interface for accessing it. how does the compiler determine at compile time that it will never be accessed again without symbolically executing the source code?

    Moreover, couldn't a standard solution simply be to execute a read operation of one element or bytes of the block immediately after the memset? Wouldn't the compiler then have to come to the conclusion that the write must actually be implemented? So a secureFill function could use std::fill followed by reading one or more bytes to verify that the write occurred. That seems a lot easier then writing a manual loop using a volatile pointer and that would allow you to continue using the library algorithms. I think that we want a solution that allows optimization of the write, but certainly not the complete removal of the write to the memory. thoughts?

  8. #8
    Join Date
    Jan 2006
    Location
    Singapore
    Posts
    6,316

    Re: Secure Memset

    Quote Originally Posted by kempofighter
    My biggest problem with understanding this is that I don't see how a compiler can determine whether something is read from or written to again; especially in a large scale project. I could understand the optimization happening in a small scale project with just a few files. however, many of the applications that would need to sanitize memory would be fairly complex programs. I just have a hard time getting my mind wrapped around how a compiler is going to analyze complex run-time behavior. In other words, let's say one class needs to sanitize a memory block but also provides an interface for accessing it. how does the compiler determine at compile time that it will never be accessed again without symbolically executing the source code?
    That's the point: a compiler might not perform the optimisation, but then again it might, so be aware of the possibility if the optimisation is unwanted. It is as simple as that.

    Quote Originally Posted by kempofighter
    Moreover, couldn't a standard solution simply be to execute a read operation of one element or bytes of the block immediately after the memset? Wouldn't the compiler then have to come to the conclusion that the write must actually be implemented?
    This is mentioned in the second article that you linked to, in the "Touching Memory" section.
    C + C++ Compiler: MinGW port of GCC
    Build + Version Control System: SCons + Bazaar

    Look up a C/C++ Reference and learn How To Ask Questions The Smart Way
    Kindly rate my posts if you found them useful

  9. #9
    Join Date
    Aug 2005
    Location
    San Diego, CA
    Posts
    1,054

    Lightbulb Re: Secure Memset

    Quote Originally Posted by laserlight View Post
    That's the point: a compiler might not perform the optimisation, but then again it might, so be aware of the possibility if the optimisation is unwanted. It is as simple as that.
    If it was that simple, then I wouldn't have started the thread in the first place. Being aware is great, if it is in fact true! But finding a solution is an entirely different matter. Moreover, these kinds of vague and general statements make it difficult to devise a solution. The compiler can't possibly know whether something could be accessed later on in a complex program. There is no way that a compiler should be making that decision based on a few lines of localized code where the memset or fill call is made unless it is a block of data local to a function. Still, the only optimization for something like that that makes any sense to me is to optimize so that the write is as efficient as possible.

  10. #10
    Join Date
    Dec 2011
    Location
    way East Europe
    Posts
    27

    Re: Secure Memset

    so a macro such as next isnt working ?

    #define SAFE_DELETE(p) { if(p) { delete (p); (p)=NULL; } };//

    which is better using: this procedure, or
    assigning a new proc to switch buffers to a new one and using it instead

    let me asume your buffer may be represented as

    char _huge* cTtemp;
    // then
    bool delete (char _huge* cTtemp){
    uint countr = 0;
    char temp2[sizeof(cTtemp)];
    for (countr = 0, counter < sizeof(cTtemp), counter ++){
    strcpy(temp2[countr],cTtemp[countr]);
    strcpy(cTtemp[countr],"\0");
    };
    };//has any safer property ?

  11. #11
    Join Date
    Nov 2003
    Posts
    1,815

    Re: Secure Memset

    >> My biggest problem with understanding this is that I don't see how a compiler can determine whether something is read from or written to again
    Not only can it do that, it can also figure out if removing the code will affect program behavior, regardless of any reads and writes).

    >> how does the compiler determine at compile time that it will never be accessed again
    It may not be able to determine anything, and so the code would be left in.

    >> Wouldn't the compiler then have to come to the conclusion ...
    >> The compiler can't possibly know ...
    >> There is no way that a compiler should ...
    Don't play the "I can out-wit my compiler" game. You'll loose

    >> ... couldn't a standard solution simply be ...
    There's c11's memset_s: https://www.securecoding.cert.org/co...pliantSolution
    I count the "volatilization" solution as a "standard solution" as well.
    Otherwise, consult your compiler's documentation.

    gg

  12. #12
    Join Date
    Jan 2006
    Location
    Singapore
    Posts
    6,316

    Re: Secure Memset

    Quote Originally Posted by kempofighter
    The compiler can't possibly know whether something could be accessed later on in a complex program.
    Suppose a programmer writes a password manager. There is a structure for holding the password data. To ensure that the passwords are not kept in memory after the program has ended, the programmer attempts to overwrite the memory of the structure sometime before the program finishes. It is certainly feasible that despite the intervening code, a compiler could determine (after say, inlining the overwrite function) that this overwrite happens at a point after which the structure is not accessed, or is accessed trivially such that the overwrite can be optimised away, hence defeating the intention of the programmer.

    Quote Originally Posted by john wiley
    so a macro such as next isnt working ?

    #define SAFE_DELETE(p) { if(p) { delete (p); (p)=NULL; } };//
    That is irrelevant (and I would prefer a function template ) because it has to do with making sure that future accesses to the pointer will not access the pointer as if it still pointed to an object (assuming that it checks for a null pointer). If there are no future accesses to the pointer and the setting of the pointer to be a null pointer is optimised away, then there is no problem, unlike for what kempofighter is talking about.
    C + C++ Compiler: MinGW port of GCC
    Build + Version Control System: SCons + Bazaar

    Look up a C/C++ Reference and learn How To Ask Questions The Smart Way
    Kindly rate my posts if you found them useful

  13. #13
    Join Date
    Apr 1999
    Posts
    27,434

    Re: Secure Memset

    Quote Originally Posted by john wiley View Post
    so a macro such as next isnt working ?

    #define SAFE_DELETE(p) { if(p) { delete (p); (p)=NULL; } };//
    That macro is useless here:
    Code:
    void DeleteFunction(char *p)
    {
        SAFE_DELETE(p)
    }
    
    int main()
    {
       char *x = new char[10];
       DeleteFunction( x );
    }
    1) After the DeleteFunction() is called, the pointer x has not changed value -- it still points to deallocated memory and has not been set to NULL. Your SAFE_DELETE macro only works in the scope where the pointer is declared.

    2) The SAFE_DELETE isn't safe if operator new[] was used, as was done in the above example (you need to specify delete[] in the macro, not "delete"
    let me asume your buffer may be represented as

    char _huge* cTtemp;
    // then
    bool delete (char _huge* cTtemp){
    uint countr = 0;
    char temp2[sizeof(cTtemp)];
    for (countr = 0, counter < sizeof(cTtemp), counter ++){
    strcpy(temp2[countr],cTtemp[countr]);
    strcpy(cTtemp[countr],"\0");
    You cannot call your functions "delete", as that is already a keyword in C++.

    Regards,

    Paul McKenzie

  14. #14
    Join Date
    Oct 2008
    Posts
    1,151

    Re: Secure Memset

    Quote Originally Posted by Codeplug View Post
    There's c11's memset_s:
    uhm, I cannot find memset_s on the n3242 draft, has it been dropped ?

  15. #15
    Join Date
    Nov 2003
    Posts
    1,815

    Re: Secure Memset

    >> c11's memset_s
    No "+" characters
    http://www.open-std.org/jtc1/sc22/wg...docs/n1570.pdf

    Perhaps C++ will include "__STDC_WANT_LIB_EXT1__" members at some point.

    gg

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center