Secure Memset

I read some code recently where someone wrote their own function called secureMemset where they used a temporary volatile pointer to the memory block and executed the write operation an extra time to ensure that the data was actually assigned to the new sanitization value. The reasoning for that was that supposedly the library function memset could be completely optimized away by the compiler, depending on how you configure a compiler and whether the compiler decides that the operation was necessary. This was news to me, as I had never heard of that before. Is this also true for the std::fill and std::fill_n algorithms? I tend to prefer the C++ algorithms anyway when I write code, but I wonder if they are also vulnerable to this security problem. Would any C++ security experts out there care to answer this question and provide some more insight. If so, please advise. I read some other forums but I did not see any place where std::fill was discussed as an alternative to memset.
http://stackoverflow.com/questions/2...memory-when-se
https://www.securecoding.cert.org/co...sensitive+data

>> Is this also true for the std::fill and std::fill_n algorithms?
Probably, unless documented otherwise. The problem is still the same, whether you use memset() or std::fill() - the compiler see's an unused stack buffer being filled with 0's at the end of it's lifetime. If the compiler can prove to itself that removing it doesn't alter program behavior, then it's legal to remove it as an optimization.

gg

Surely there must be something more elegant than using the volatile keyword and a manually written loop. Honestly, the aforementioned links don't do a very good job of explaining why the volatile keyword prevents optimization. Moreover, how can the compiler possibly know what is going to happen at run-time? If I try to reset pointers to zero in a destructor after destroying an object, will that be optimized away? That makes no sense to me, but the compiler could determine that operation is pointless since the object is being destroyed just as easily as it could assume that the memset isn't really necessary. Unfortunately, I have not seen a realistic example that proves the theory one way or the other.

Quote:

---

Originally Posted by kempofighter

Honestly, the aforementioned links don't do a very good job of explaining why the volatile keyword prevents optimization.

---

I believe the idea is that due to the volatile keyword, the compiler can no longer assume that there are no further accesses to the variable, just because it does not find any in the code it is given.

Quote:

---

Originally Posted by kempofighter

If I try to reset pointers to zero in a destructor after destroying an object, will that be optimized away?

---

It could be optimised away, though of course we cannot say if it will be.

I think you cannot expect a standard guaranteed solution to this problem, as it relates to the side effects of code on "real" memory whose relation with the "memory" of the abstract machine ruled by the standard is unspecified.

So, IMHO this kind of things should be implemented at the compiler/OS support level, like using SecureZeroMemory on Windows ...
otherwise, if not avaliable on all targets, I'd write a test system used to systematically check for required memory behavior in the final release binary ...

AFAIK, the point of using alorithms like std::fill is that when used with POD data, the meta programming will cause the call to degenerate all the way into a very efficient memset/memcopy.

This means that no matter what type you are using, std::fill is pretty much optimal. In theory, it could even be implemented using micro multi-threading!

----
If I may ask a question though, why would you want to prevent optimization? If the data you are writing to MUST be memset, yet you never actually read that data, then wouldn't the correct solution be to rightfully declare that data volatile.

If your data is not volatile, and you never read it, then the compiler can and will optimize way whatever you do, memset, loops, std::fill etc. For the compiler, it's all the same.

The solution of "secure" memset is merely putting the burden of volatile declaration from the developer to the algorithm. Not a good idea, IMO.

My biggest problem with understanding this is that I don't see how a compiler can determine whether something is read from or written to again; especially in a large scale project. I could understand the optimization happening in a small scale project with just a few files. however, many of the applications that would need to sanitize memory would be fairly complex programs. I just have a hard time getting my mind wrapped around how a compiler is going to analyze complex run-time behavior. In other words, let's say one class needs to sanitize a memory block but also provides an interface for accessing it. how does the compiler determine at compile time that it will never be accessed again without symbolically executing the source code?

Moreover, couldn't a standard solution simply be to execute a read operation of one element or bytes of the block immediately after the memset? Wouldn't the compiler then have to come to the conclusion that the write must actually be implemented? So a secureFill function could use std::fill followed by reading one or more bytes to verify that the write occurred. That seems a lot easier then writing a manual loop using a volatile pointer and that would allow you to continue using the library algorithms. I think that we want a solution that allows optimization of the write, but certainly not the complete removal of the write to the memory. thoughts?

Quote:

---

Originally Posted by kempofighter

My biggest problem with understanding this is that I don't see how a compiler can determine whether something is read from or written to again; especially in a large scale project. I could understand the optimization happening in a small scale project with just a few files. however, many of the applications that would need to sanitize memory would be fairly complex programs. I just have a hard time getting my mind wrapped around how a compiler is going to analyze complex run-time behavior. In other words, let's say one class needs to sanitize a memory block but also provides an interface for accessing it. how does the compiler determine at compile time that it will never be accessed again without symbolically executing the source code?

---

That's the point: a compiler might not perform the optimisation, but then again it might, so be aware of the possibility if the optimisation is unwanted. It is as simple as that.

Quote:

---

Originally Posted by kempofighter

Moreover, couldn't a standard solution simply be to execute a read operation of one element or bytes of the block immediately after the memset? Wouldn't the compiler then have to come to the conclusion that the write must actually be implemented?

---

This is mentioned in the second article that you linked to, in the "Touching Memory" section.

Quote:

---

Originally Posted by laserlight

That's the point: a compiler might not perform the optimisation, but then again it might, so be aware of the possibility if the optimisation is unwanted. It is as simple as that.

---

If it was that simple, then I wouldn't have started the thread in the first place. Being aware is great, if it is in fact true! But finding a solution is an entirely different matter. Moreover, these kinds of vague and general statements make it difficult to devise a solution. The compiler can't possibly know whether something could be accessed later on in a complex program. There is no way that a compiler should be making that decision based on a few lines of localized code where the memset or fill call is made unless it is a block of data local to a function. Still, the only optimization for something like that that makes any sense to me is to optimize so that the write is as efficient as possible.

so a macro such as next isnt working ?

#define SAFE_DELETE(p) { if(p) { delete (p); (p)=NULL; } };//

which is better using: this procedure, or
assigning a new proc to switch buffers to a new one and using it instead

let me asume your buffer may be represented as

char _huge* cTtemp;
// then
bool delete (char _huge* cTtemp){
uint countr = 0;
char temp2[sizeof(cTtemp)];
for (countr = 0, counter < sizeof(cTtemp), counter ++){
strcpy(temp2[countr],cTtemp[countr]);
strcpy(cTtemp[countr],"\0");
};
};//has any safer property ?

>> My biggest problem with understanding this is that I don't see how a compiler can determine whether something is read from or written to again
Not only can it do that, it can also figure out if removing the code will affect program behavior, regardless of any reads and writes).

>> how does the compiler determine at compile time that it will never be accessed again
It may not be able to determine anything, and so the code would be left in.

>> Wouldn't the compiler then have to come to the conclusion ...
>> The compiler can't possibly know ...
>> There is no way that a compiler should ...
Don't play the "I can out-wit my compiler" game. You'll loose :)

>> ... couldn't a standard solution simply be ...
There's c11's memset_s: https://www.securecoding.cert.org/co...pliantSolution
I count the "volatilization" solution as a "standard solution" as well.
Otherwise, consult your compiler's documentation.

gg

Quote:

---

Originally Posted by kempofighter

The compiler can't possibly know whether something could be accessed later on in a complex program.

---

Suppose a programmer writes a password manager. There is a structure for holding the password data. To ensure that the passwords are not kept in memory after the program has ended, the programmer attempts to overwrite the memory of the structure sometime before the program finishes. It is certainly feasible that despite the intervening code, a compiler could determine (after say, inlining the overwrite function) that this overwrite happens at a point after which the structure is not accessed, or is accessed trivially such that the overwrite can be optimised away, hence defeating the intention of the programmer.

Quote:

---

Originally Posted by john wiley

so a macro such as next isnt working ?

#define SAFE_DELETE(p) { if(p) { delete (p); (p)=NULL; } };//

---

That is irrelevant (and I would prefer a function template ;) ) because it has to do with making sure that future accesses to the pointer will not access the pointer as if it still pointed to an object (assuming that it checks for a null pointer). If there are no future accesses to the pointer and the setting of the pointer to be a null pointer is optimised away, then there is no problem, unlike for what kempofighter is talking about.

Quote:

---

Originally Posted by john wiley

so a macro such as next isnt working ?

#define SAFE_DELETE(p) { if(p) { delete (p); (p)=NULL; } };//

---

That macro is useless here:

Code:

---

void DeleteFunction(char *p)
{
    SAFE_DELETE(p)
}

int main()
{
  char *x = new char[10];
  DeleteFunction( x );
}

---

1) After the DeleteFunction() is called, the pointer x has not changed value -- it still points to deallocated memory and has not been set to NULL. Your SAFE_DELETE macro only works in the scope where the pointer is declared.

2) The SAFE_DELETE isn't safe if operator new[] was used, as was done in the above example (you need to specify delete[] in the macro, not "delete"

Quote:

---

let me asume your buffer may be represented as

char _huge* cTtemp;
// then
bool delete (char _huge* cTtemp){
uint countr = 0;
char temp2[sizeof(cTtemp)];
for (countr = 0, counter < sizeof(cTtemp), counter ++){
strcpy(temp2[countr],cTtemp[countr]);
strcpy(cTtemp[countr],"\0");

---

You cannot call your functions "delete", as that is already a keyword in C++.

Regards,

Paul McKenzie

Quote:

---

Originally Posted by Codeplug

There's c11's memset_s:

---

uhm, I cannot find memset_s on the n3242 draft, has it been dropped ?

>> c11's memset_s
No "+" characters :)
http://www.open-std.org/jtc1/sc22/wg...docs/n1570.pdf

Perhaps C++ will include "__STDC_WANT_LIB_EXT1__" members at some point.

gg