What's a good class for writing a Log File Class?

[Silver Ghost]

What I need is a buffer (about 10kb) which can be flushed any time.

Besides in my case I can never know whether the user will write 1 line of text in an hour, or 1000 lines of text in a second, so I need something that is capable of keeping the connection to the file when not writing, and allowing outside sources to use the file, again when not writing...

What's a good class for that?

I tried stream writer, but I am not sure if that's the optimal solution...

[MartinL]

Hm.. I would not keep the log file open all the time...

This is only suggestion, so think about it:

public class CEventLogEngine : IDisposable
{
protected AutoResetEvent evt;
protected Queue store;
protected Thread writeThread = null;
protected bool bThreadQuit = false;
protected string filename;

protected bool isMessage()
{
lock (store.SyncRoot)
if (store.Count > 0) return true;
return false;
}

protected string getMessage()
{
lock (store.SyncRoot)
if (store.Count > 0) return (string)store.Dequeue();
return null;
}

protected void WriteThread()
{
StreamWriter sw = null;

while (bThreadQuit == false)
{
evt.WaitOne();

try
{
sw = new StreamWriter(filename, true, System.Text.Encoding.ASCII);

lock (store.SyncRoot)
{
while (isMessage() == true)
{
string sMsg = getMessage();
sw.WriteLine(sMsg);
}
}

sw.Close();
}
catch (Exception)
{
if (sw != null)
sw.Close();
}
}
}

public CEventLogEngine(string fileName)
{
filename = System.AppDomain.CurrentDomain.SetupInformation.ApplicationBase + fileName;
store = Queue.Synchronized(new Queue(0));
evt = new AutoResetEvent(false);

writeThread = new Thread(new ThreadStart(WriteThread));
writeThread.Start();
}

~CEventLogEngine()
{
Dispose();
}

public void Dispose()
{
if (writeThread != null)
{
bThreadQuit = true;
evt.Set();
writeThread.Join();
writeThread = null;
}
}

protected void AddMessage(string sText)
{
lock (store.SyncRoot)
{
store.Enqueue(sText);
evt.Set();
}
}

public void LogLine(string sText)
{
try
{
AddMessage(sText);
}
catch { }
}

public void LogLine(string sFormat, params object[] args)
{
System.IO.StringWriter sw = new System.IO.StringWriter();
sw.WriteLine(sFormat, args);
string sText = sw.ToString();

LogLine(sText);
}
}

Now you can just construct this class and everytime, when you need to add new log line, just call one of the LogLine methods.

This class opens the file only when it need it, so it is possible to use this file from outside the application too. Additionally, if the user makes many action that need to be logged, the class detects how many messages are waiting for logging and writes them all at once...

Martin

[Silver Ghost]

can you show me what "using" statements you use? Because it won't compile...

Can you also explain what lock statement does?'

[Silver Ghost]

Ok I figured them out:

using System;
using System.Threading;
using System.Collections;
using System.IO;


... now it's gonna take me a while to understand that script since it's advanced for me...

[Silver Ghost]

Ok here are my specific questions:

1. I got confused by the following:
public void LogLine(string sFormat, params object[] args)
{
System.IO.StringWriter sw = new System.IO.StringWriter();
sw.WriteLine(sFormat, args);
string sText = sw.ToString();

LogLine(sText);
}

Ok, I see that you create a new string writer, than you write it, but never close it. Than for some reason you convert the OBJECT to string and pass that string to LogLine where ... (see question 2 :)


2. I also got confused by the following:
public void LogLine(string sText)
{
try
{
AddMessage(sText);
}
catch { }
}

I know what try and catch blocks are, but why is the catch block empty? What would that do? My guess is that in case of a thrown exception it will simply catch it and ignore it - so you don't get to see it in your console... am I correct?

3. Ok now I am confused with AddMessage:
protected void AddMessage(string sText)
{
lock (store.SyncRoot)
{
store.Enqueue(sText);
evt.Set();
}
}

What does lock (store.SyncRoot) ? I will try to look up some info by myself, but it confuses the **** out of me :)


That's pretty much it :)
Thank you in advance.

[Silver Ghost]

Ok I've been reading about lock statement... MSDN states "lock ensures that one thread does not enter a critical section while another thread is in the critical section of code."

MSDN keeps talking about store.SyncRoot as some kind of storage which is thread-safe... I don't quite understand that, please explain it to me in your own words ;) Here's my guess:

"store.SyncRoot is some kind of storage where we can hold objects... In our case strings. And since you lock it all the time, which means that only 1 thread at a time will be able to access it (which doesn't make any sense to me since there's only 1 line of code - what can happen), I suppose you want to be protected from some kind of outside proccesses?

Thank you in advance again :)

[MartinL]

Wow, so many question at this early time... :) I am still sleeping... :)

OK, lets try to explain it... :)

usings...
Yes, you are right...

1.) StringWriter is a class designed to build new string objects. It has a couple of methods that allows you to simply put new objects into the string. In the code I post here, it is just WriteLine member. It takes the format string and objects array.
For example

string userName = "some user";
int chatNumber = 23;
sw.WriteLine("The user {0} has been kicked out from the chat room number {1}", userName, chatNumber.ToString());

This code just appends new line containing the formating string where {n} are replaced by the (n+1)th parameter passed to the WriteLine member...

About the closing StringWriter... You don't need to close it explicitely (however, it is gool practice to close it... :) ) because garbage collector cares about it. Immediately when the function LogLine finishes, StringWriter object goes out of scope and garbage collector can release it. During the process of releasing the object, its close method is called to...

2.)
Yes, you are right again... I don't want to see messeges about I don't know what.. So I catch them and ignore them... However, If you want to use it in real project you should minimally log that exception to the system event log.. (EventLog class in System.Diagnostic namespace)...

3.) store.SyncRoot
Generally, SyncRoot is an object that can be used during thread-synchronization... lock keyword is something as critical section...

It is classic problem of thread synchronization... There is a writer thread running in the code I have posted before... That thread uses store object (store is just a container called queue - thats a first-in first-out collection). However, store object can be used also from anothers threads... Generally, you can have 20 threads and each of them use CEventLogEngine object. Each of them calls LogLine method so store object is used in other 20 threads...

That's why we need to synchronized access to the store object. Queue class exposes the property named SyncRoot. It returns the object that can be used to make this synchronization... So if I write

lock (store.SyncRoot)
{
while (store.Length > 0)
store.Dequeue();
}

it is garanteed that while loop will be entered only when there is no other thread that is in lock (store.SyncRoot) block of code. If there is such thread, the calling thread is suspended until another thread leaves the lock (store.SyncRoot) block...

About that one line of code... All this code is compiled to the assembly language... And that one line of code I wrote there is in real world and assembly language a couple of tens lines of code... We can't allow interrupt execution of these lines of assembly language by another thread... So thats why the lock block... Generally just assignement to the single built-in types are thread safe (however only if you are running your code on one-proccessor machine... ) :)

So, if you have another question, feel free to ask...

Martin

[Silver Ghost]

Hmm thanx for everything, but I still don't understand the 1st one...

Ok you write to the file, THAN you call another method which will also write to a file... why would you do that?


... And what will happen if I remove all the lock statements?

[MartinL]

No... I write to the file just one time... In the WriteThread function... StringWriter.WriteLine function writes to the string not to the file... Then I call AddMessage function. AddMessage function puts the string created by StringWriter into the queue and wakes up the thread (WriteThread function). WriteThread function then gets the message (string) from the queue and writes it into the file...

If you will remove all the lock statements, the code may crash... And it probably will crash!

Martin

[Silver Ghost]

I see... thank you.

[dfb78]

I added this class, but now after closing my application, it remains in Task Manager, and doesn't "exit" until I kill the process. Is there something I should add to your class (or mine even) that will take care of this?

Thanks

[MartinL]

Yes...

It is probably due to the grabage collecting. Did you explicitelly call Dispose method of that class before you exit your application?

If not, the threads are still running (even when your application is about to quit). They stop when Dipose() method is called. Dipose method is called by the destructor. However, destructor calling is not deterministic when you leave it on garbage collector. It means you don't know when the destructor is called. So, just call Dispose() method when you don't need that class anymore...

Martin

[dfb78]

Doh! That was it. Thanks.

[petru66]

MartinL,

You write:

"I would not keep the log file open all the time."

May I ask you why?

Petru

[MartinL]

If you use that class just for logging some information for little application and you really know what is done with that file, you can leave that file opened..

However, I can image situations when it is not possible. When some external applications can work with that file (including deleting it or writting own logs there)...

However I think (that is my opinion!) that is even better if you open the file just when you need to write there some information. Why it should be opened for whole run of the application?

martin

[usmarine]

Ideally you should put this functionality in a service add write to it via shared memory. The event log is sort of one example of this but has limitations.

[petru66]

Martin,

Thanks a lot for your answer!

It seems that I found the right person to ask questions :)

It so happens, that I have to write a trace-log class. I have to trace function calls - too fast for TimeStamp to be of any use, but that's another problem, and I solved it somehow.

Anyway, the problem is this: There are hundreds of thousands of messages to be logged, during the whole execution of the application. I keep the file open, because otherwise the overload for opening/closing would be exceedingly high. I am very upset because it is not properly closed. There is no method explicitly called when the application exists, therefore the only place where I could put the close is in Dispose; which is never called!!!

Perhaps you have an idea...

Thanks,
Petru

[MartinL]

Hm... Sure, place it to Dispose() method...

However Dispose() is called from the destructor. We are dealing with garbage collector in .net so destructor calls are not deterministic...

Garbage collector examines so-called application roots maintained by the CLR. When the object is not reacheble from any application roots it is free to deletion. The process of deletion is called Garbage Collecting. (it is not just about deleting unreachable objects. It is also about moving objects among the generations - internal structure of gargbage collector) However you don't exactely know when that occures. It may be immediatelly after you close tha application or even half a hour after quitting your app...

Fortunatelly you can force this garbage collecting. There is a class GC in System namespace. You can use System.GC.Collect() method to perform the garbage collecting immediatelly. It will imediatelly call destructor and Dispose() method of that object.

You need to handle request to quit the application. And you can call GC.Collect() there...

martin

[Harshit]

well may be i m too late but just wanted to share somthing which you guys might be knowing already..but for those who dont..

we actually dont need to invoke the garbage collector manually just for the heck of calling dispose() method of any particular class.. a good solution will be to use using statement like this..

using (Resource res = new Resource()) {
res.DoWork();
}

the above code is as good as the following..

Resource res = new Resource(...);
try {
res.DoWork();
}
finally {
if (res != null) ((IDisposable)res).Dispose();
}

so it calls the dispose() method automatically when u are done with ur object.

-Harshit

[MartinL]

Yes, it is true... However it works only when you exactelly know where you object does the job... It is impossible to use if the object should be alive during the whole programe execution and should be destroyed before you program finishes...

martin

[tillu]

Isn't there some overhead for opening and closing file every time application writes something?

I have a very hihg performance required console application that does a lot of logging. Is it going to slow it down if log file is opened aand losed frequently?

Thanks.

[TheCPUWizard]

Posting my two cents on the "keep open"/"open on demand" issue. I have faced this many times. There are issues with both.

For a top performance class, I will typically use a timer. A write to the file resets the timer. If the timer expires, i close the file.

My timer is typically on the order of 1 second.

Now the file is closed "most of the time", bu remains open between successive calls within a short duration.