Real Time Data collection from Socket and Storing in Excel
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 4 of 4

Thread: Real Time Data collection from Socket and Storing in Excel

  1. #1
    Join Date
    Jan 2012
    Posts
    2

    Real Time Data collection from Socket and Storing in Excel

    Dear All,
    I m new to this forum. Thanks for welcoming in advance
    I m new to VB 2008Express programming. I m into automation development of FMCG products using Micro-controllers. So,VB is pretty new to me. Well , all said..
    I get real time data Barcode 2D from TCP/IP connection. My task is to compare the result do some Computation with them and Store the incoming Barcode and results into a Excel sheet in a folder. I m successful with the basic working.
    Am currently working at slow speed of 1 barcode(22 char info) per second from the TCP/IP data (barcode scanner). Now, comes the real issue. I need to process atleast 5 barcodes per sec ( <200ms per process). I m partially successful with 3 Barcode per second(300 ms per process).
    so, thats like 300ms per processing. I need to achieve less than 200ms for entire processing. I have used threads and timers. but I m unable to achieve.
    Seems like storing in Excel takes more than 250ms.
    Since i come from micro controllers background 1msec is a lotta time to process for me. I m not sure how much time does it take and trust me i find it difficult to guage the time here.
    Please Help me with this. I m stranded with this issue and find no where to go.
    Please advice me on the approach that i should be taking. I m attaching the code.
    Thanks for viewing this.
    Kindly Help me.

  2. #2
    Join Date
    Nov 2002
    Location
    California
    Posts
    4,553

    Re: Real Time Data collection from Socket and Storing in Excel

    Only 3 barcodes per seconds seems remarkably slow.

    You need to identify the culprit. Why do you believe that the culprit is storage to Excel? Have you achieved significantly higher throughput if storage to Excel is bypassed?

    In general, to identify the culprit, I would experiment with different configurations. For example, to eliminate TCP as the culprit, try kludging together a configuration where the exact same 2D barcode data is read repeatedly from a known high-speed source like a local file, and see if your results are significantly faster. Same for the barcode decoding: skip it completely and output a constant answer to see if the program is significantly faster.

    Get back to us when you have determined which component is the bottleneck in processing speed.

    Mike

  3. #3
    Join Date
    Jan 2012
    Posts
    2

    Re: Real Time Data collection from Socket and Storing in Excel

    Thanks Mike for your reply and highly appreciated for the suggestions given.

    I have tried just storing 100 (20 char each) data sequentially with Button_click_event and timer_tick_event (timer1.interval = 100), during this I m able to write only few data onto excel then the app hangs. now if i change the timer1.interval = 300. I am able to store all 100 data. Am doing this experiment using timer event and within it call the function to write in excel. (is this the wrong approach???)
    this made me suspect more on the writing onto excel sheet consuming bulk of the time.

    Like u said, i did simulating a server(as source for Barcode) with another computer. With this setup i made server send me a data at different timer intervals so that i understand if the socket(TCP) is the bottleneck . I m able to collect data with 250 ms intervals and do my basic processing.

    the app hangs if timer interval < 200. I still think my app should handle such speed.

    Can u pls tell what i should be doing. Give me an idea of using threads for the right process and timer too. How much time is consumed to open a excel file, dump the data onto the cell and close the file. Kindly help me on this. I have put up the code. I can share more info if anyone desires too.
    Does the hardware has a part to play here. I m running this App on Intel Pentium 1.6Mhz, 1GB RAM . Will a multi/quad core processor help running multiple threads same time???


    Cheers

  4. #4
    Join Date
    Nov 2002
    Location
    California
    Posts
    4,553

    Re: Real Time Data collection from Socket and Storing in Excel

    I'm still confused about the source of your bottleneck. First you say this, which makes me think that you believe the source of the bottleneck is Excel:
    Quote Originally Posted by benikvarun View Post
    I have tried just storing 100 (20 char each) data sequentially with Button_click_event and timer_tick_event (timer1.interval = 100), during this I m able to write only few data onto excel then the app hangs. now if i change the timer1.interval = 300. I am able to store all 100 data. Am doing this experiment using timer event and within it call the function to write in excel. (is this the wrong approach???)
    this made me suspect more on the writing onto excel sheet consuming bulk of the time.
    But then you say this, which makes me think that you believe the source of the bottleneck is Excel:
    Quote Originally Posted by benikvarun View Post
    Like u said, i did simulating a server(as source for Barcode) with another computer. With this setup i made server send me a data at different timer intervals so that i understand if the socket(TCP) is the bottleneck . I m able to collect data with 250 ms intervals and do my basic processing.

    the app hangs if timer interval < 200. I still think my app should handle such speed.
    So which is it, Excel or TCP? Or do you believe that both are bottlenecks?

    I agree that this type of processing should have the capability of moving much faster. Your hardware setup seems more than adequate. From what you describe, extra threads etc will not help you.

    If you believe that Excel is the bottleneck, you should look more carefully at Excel automation. In addition, I would not open ond close the file so frequently. Open it once and leave it open.

    Here is one link to start learning Excel automation from VB .NET (which is what I think you have): "How to automate Excel from Visual Basic .NET to fill or to obtain data in a range by using arrays" at http://support.microsoft.com/kb/302094

    If you believe that TCp is the bottleneck, then you should look more carefully at asynchronous socket communications. Here is one link to start learning about asynchronous sockets: "Using an Asynchronous Client Socket" at http://msdn.microsoft.com/en-us/library/bbx2eya8.aspx

    Incidentally, you mention timers. I don't see why you need any timers at all, at least not for the program that receives the barcode (via TCP). Your program should be arranged to listen asynchronously for the receipt of a new barcode via TCP. When a new barcode is received, the asynchrous handler should decode it and write the results to the Excel file, and then continue to listen asynchronously for the arrival of the next barcode. No timers are involved nor are any timers needed.

    Mike

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Azure Activities Information Page

Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center