Printable View
I need to write an interface program in C++ to receive occasional data being sent to me from a different company from across the internet. The company says they will send the data using POST. They have given me a document that lists their data's specific parameters and their meaning. They have told me very little else.
How do I receive this data in my C++ program?
What is the "POST" in this context?Quote:
Originally Posted by ThisIsMe
Probably the POST request method supported by HTTP. See http://en.wikipedia.org/wiki/POST_%28HTTP%29Quote:
Originally Posted by VictorN
See this codeproject article: http://www.codeproject.com/Articles/...P-Wrapper-in-C
Well, I believe the first thing you need here is HTTP server (typically called web server) able at least to do HTTP POST processing. I would strongly recommend to have your web server and interface program separate from each other.
Further design would depend on your dev team skills. For example, personally I would go with Apache + PHP + database, where Apache listens to incoming HTTP POST requests, PHP code extracts posted data parts and put those to database. Your "interface program in C++" would listen to changes in database and visualize those.
In case of some other web skills a solution could be based on HTTP server implemented in C#. Depending on your design of handling data, database may be or may be not a part of your system.
HTTP Server API
HTTP Server Sample Application in plain C.
also:
see C++ REST SDK a.k.a. "Casablanca" on codeplex https://casablanca.codeplex.com/
which is a stand alone framework for C++ based web services without requiring a Webserver (IIS? apache...) running.
the disadvantage is that for the time being, it'll require the POST data to be JSON. Other formats are for "later".
Perhaps I have led a sheltered life and I do not know even the basic stuff.
My program needs to receive data sent to my program by POST. Does that mean I need to implement a web client or a web server? The code project article that you referenced describes a web client. Do I need to implement a client or server?
Or am I messed up in my understanding?
What post do you mean?Quote:
Originally Posted by ThisIsMe
This onegave you links to a web server examples/APIQuote:
Originally Posted by Igor Vartanov
I should have used "reply with quote". I was replying to this "See this codeproject article: http://www.codeproject.com/Articles/...P-Wrapper-in-C"Quote:
Originally Posted by VictorN
I still don't understand. Are you saying that I need to implement a web server. Please help me.
I was looking for this when I replied earlier, but forgot the code name. It's the way to go if it's a rest based interface.Quote:
Originally Posted by OReubens
Ultimately, it doesn't matter what we're saying here. You have enough links to make your mind and understand what you need and choose a way to go. If you still don't, this just indicates that you are not prepared to handling this task, at least right away. Nothing outstanding, nobody is able to know everything.Quote:
Originally Posted by ThisIsMe
And yes, you need a web server. As you might notice, I already said this before.
I misread the original requirements, so a web server is needed.Quote:
Originally Posted by Igor Vartanov
At first I thought you are attempting to post data to the 3rd party's web server. After re-reading the OP, I see that the 3rd party needs to post data to you.
For this you need a web server. So see post #6.
Thank you for helping me. Please understand that I don't yet understand some of the significance here. I am sure there is "meaning" and "understanding" that is way over my head. Please help me to understand.Quote:
Originally Posted by Arjay
There are three entities.
1) Me and my company. I am writing software for my company.
2) My company's client who bought our software. Our software will be installed at this client, at their location and on their computer.
3) A third party that will be sending occasional data using POST. It will be sent to my software. The software is at the client's location.
Am I going to have to tell the third party what web address to send the POST data. Is that correct?
That web address will be a web address of my client. Correct?
Will I have to involve my client's IT department to allow this or set this up or something? Will this just work or are there issues?
I am very confused.
Lets say my client has a website "www.widget.com". They have a full web site setup there. So now one of their departments buys my software. My software needs to receive this POST data from a third party. How does this work? I just don't understand.
It seems OP never listens to what I say, but anyway...
Your software is not obliged to receive POST data. And any web server exposed to internet or customer's intranet can do that for your software by piping the received data to database. The database is very handy in this case, as it provides persistence for the data, and is able to collect the posted data even when your interface program is down. So you never miss a bit of the data because of being shut down. Besides, in this case the interface program just deals with database connection, which simplifies programming things a lot.Quote:
Originally Posted by ThisIsMe
Yes, but from reading the OP posts, I think this is the issue that the OP doesn't understand how this is done for this particular data that is sent to them.Quote:
any web server exposed to internet or customer's intranet can do that for your software by piping the received data to database.
if your app is the one that "hangs around doing nothing" until the other side decides to send you some data, then yes, you are the server side and the other is the client/consumer.
If both sides are "hanging around"... THen you have a server and client on each end.
there are other approaches for communication, but since you're talking about POST and thus HTML, those are irrelevant.
note that some webservices are "extensions" to a full web server (such as ISAPI technology for IIS)
a webservice could also have it's own communication framework built in and run without requiring other software to be installed first.
given a webservice is present, then simple webservices can even be implemented as a dynamic HTML page, this is typically done via scripting features of the webserver, IIS uses ASP.NET for this.
Or it could even be an oldschool cgi approach. (essentially you make a normal console program and take your input from stdin, and output the result to stdout, the webserver will take care of redirecting input and output).
There are many questions that need to be asked before coming up with an architecture. Unfortunately we don't know what your app does, or how it interacts with the 3rd party app. We don't know if all your customers that use this app also have to interact with the 3rd party app. We don't know the frequency of the 3rd party post. We also don't know if more than one instance of your app is run by the customer. Fill in a few more details and we should be able to come up with a few suggestions.Quote:
Originally Posted by ThisIsMe
Assuming you are sticking with the approach...
correctQuote:
Originally Posted by ThisIsMe
yesQuote:
That web address will be a web address of my client. Correct?
Depending on who and what your clien is... yes, no, maybe. hard to tell.Quote:
Will I have to involve my client's IT department to allow this or set this up or something? Will this just work or are there issues?
There are literally MANY issues. several dozen if you dive deep into it.
The major ones will revolve around the fact that you and the 3rd party are essentially expecting this client to be a server.
Meaning, they'll have to have a fixed IP address, thus needing a corporate "server" type internet feed
this typically is going to be costly. If they currently use regular "consumer type" internet like regular cable or ADSL, then you can see the cost of that go up by a factor 10 and more.
(yes, ISP's are ripping you off).
They'll need a 24/24 server or at the very least one that is up for the timeframe the third party will be sending data.
They'll need everything properly set up and properly protected with firewalls etc, It your client is a small company, this can quite literally become overwhelmingly difficult, they can hire outside parties to do this for them, but again, that'll be a big cost.
they'll probably want a separate server from their corporate server (if they aven have that) for security reasons. Adding even more complexity.
---
It's possible that realistically the 3rd party should be operating a web server where your client will periodically get data from rather than the other way around. It's hard to tell without much more info, but if that 3rd party will be doing a lot of those posts to many other companies, they're tacking it on wrong now.
---
there's possibilities by using an intermediary where the 3rd party posts to and your client gets from.
At it's simplest this could be handled by e-mail. It depends how fast data needs to be integrated and how fast a post is expected to be followed by a response.
if they have a website already...Quote:
Lets say my client has a website "www.widget.com". They have a full web site setup there. So now one of their departments buys my software. My software needs to receive this POST data from a third party. How does this work? I just don't understand.
The Obvious approach would be to add a wedservice onto the website using the built in means the web server software allows you to do so. If there isn't one, you just add a separate webservice app onto the same server.
however, this means they're running their own actual server and not just renting web hosting running on some ISP's machines.
Thank you to everyone who is helping me. I have learned a lot here and I have been reading elsewhere as well.
The third party will send messages to my software at odd intervals at a rate that will vary by day and by client. On the minimum, perhaps as few one message in a day. The upper limit will be several hundred messages total per day. Probably never more than one thousand in a day. Each message contains independent data. The messages and their data are not related to each other. My software does not need to respond to each message.
My software will be the only connection at a client's site to the third party.
I have now learned that in addition to receiving the data being sent to me by POST, I can optionally send heartbeat messages to the third party. In this case I would send a POST to the third party. Using the heartbeat would be at my discretion. I want to implement this feature. Each heartbeat I send will be be responded by the third party using a POST message. Since I am both receiving and sending POST messages, I assume this means I must implement both a web server and a web client interface.
My company's software maintains a database and talks to and gathers information from other computers and devices. Some of these other devices we manufacturer and others are manufactured by a variety of other companies. The new data coming to us from the third party will be captured and stored in our database. It will be displayed to users if they choose to view it. In the future it is possible that our clients may ask us to act upon the data in some way.
My company's software connects to a wide variety of devices. Usually data is passed around using TCP and custom packets. We also have some serial port interfaces. The concepts of receiving data from outside the client's office, from the internet and through use of the web protocol POST are all new concepts to us. I thank every one that is helping me.
Ultimately I am looking for the simplest interface method. I have learned that this is not as simple as opening a TCP port. I wish it was.
Ah, I see.
I would separate your app from the web service (meaning, your app doesn't host the web service directly per se). Think of your application as a set of 'components' and one of the components is the web service. Really by components, I mean multi-layered architecture.
For the web service, I would expose multiple endpoints (i.e. interfaces). One of the interfaces is an external interface to receive the POSTs from the 3rd party.
Other interface(s) are used by the components that gather the device information. The gathering components would use this interface to read from and write to the database (with this architecture, gathering components don't have access to write to the db, only the web service does).
When the app needs to write to the 3rd party (or parties), it passes the data to the web service and the web service handles dispatching the data to the 3rd party.
Even though your gathering applications are written in C++, I would probably use C# for the web service (and use WCF services). WCF gives you various hosting options, allows for scalability and is easily implemented (much more so than writing a web service with C++).
Is there a framework or a working program project someplace like CodeGuru that implements the web server and web client interface. Something that I can just do some very minor enhancements to handle the 2 or 3 specific POST messages that I must receive. And the one (heartbeat) message that I must send.
I was thinking this framework program would handle the web stuff and then pass the data to our main program using a TCP packet. We could then run this web interface program on the same computer as our regular software. Alternately it could run a different computer on the same local network as our main computer. Our main program would then handle this data the way it handles data from all the other interfaces we deal with.
This seems the simplest implementation. It does what we need and nothing else. Very compartmentalized. Its a web interface to my company's program.
This is an indication of proof that the third party is taking it on the wrong way.Quote:
Originally Posted by ThisIsMe
If they can handle heartbeat requests, this means they already have a server type feature for this.
Why are they then still insisting on posting data, rather than having the clients query them.
THe current approachs seems to enfore that both the third party and your client need to have dedicated servers, which while certainly not impossible, is a bit of a weird premise.
the simplest is that the third party has a webservice, and that you make simple HTTP GET requests to query data periodically.Quote:
Ultimately I am looking for the simplest interface method. I have learned that this is not as simple as opening a TCP port. I wish it was.
With an API like WinInet, this takes a dozen or so lines of code.
In .NEt which has simpler classes for this, it's even less.
The POST data that we are receiving from the third-party is somewhat time critical. Our client would want to have the data in their system as soon as possible. Within a few seconds. Therefore periodically polling to receive data would not be a good approach. The third party company has supplied this POST communication method. I can not change that communication method.
The third-party is taking a very minimalistic view when working with my company. They basically told me:
1) This is the interface we are offering.
2) We have given you a document that describes the details.
3) You should be able to make this work.
They are not helpful other than that. I have tried talking to their technical people. I don't know if they don't want to get caught up in the difficulties of explaining this, or they are very busy or they don't care.
The best thing for me would be a framework program that handles the web server and web client aspect. This program would then forward received data to the rest of our system using a TCP link. Is a basic framework program available from somewhere?
They "don't care", because they don't want to get involved in havign to deal with the support associated with this;Quote:
Originally Posted by ThisIsMe
As I already indicated, there are literally dozens of minor and major issues to resolve to get this to work from a technical POV (and then I'm not even talking about the actual implementation programming).
The first issue it getting this sorted with your clients ISP. If you want to run a server, that pretty much mandates having a fixed IP. Regular customer type Cable/DSL systems have dynamic IP's that typically reset/change at least every 24 to 48 hours. Depending on your contract, it's also possible you are not allowed to run a server, and it's possible the HTTP port is being blocked to enforce this.
The next would be to decide on the overall approach, will you be creating a full server yourself, or writing an extention for an existing webserver software (maybe already installed), or are you going to solve it with scripting based on that webserver software.
there are tons, but you'll need to decide on the overal approach first. A framework for say an IIS type extention isn't going to help you at all if your client runs Linux servers using apache for example.Quote:
The best thing for me would be a framework program that handles the web server and web client aspect. This program would then forward received data to the rest of our system using a TCP link. Is a basic framework program available from somewhere?
I thank everyone who has contributed on this topic. From my viewpoint I have received several answers that take me in slightly different directions. Some answers seem to be only a partial solution. Unfortunately I don't have the expertise to understand which answer is complete and will work.
SUMMARY:
I need to receive a total of three different HTTP POST commands from a third party entity on the internet. I also need to send a POST command to that same entity.
The POST commands will be received at random intervals. Each POST contains separate data and are not related to each other. At most a few hundred commands per day.
The data in the POST commands is time relevant. We want to have and process the data within a few seconds of it generation.
My program is the only entity at its physical location that is in contact with and is receiving data from the third party. There is no need to coordinate with multiple entities receiving data from the third party.
I have been told I need a static internet IP. I accept that this is needed and I except the expense that may be required.
As I understand it... I need to implement both a web server and a web client.
I am searching for a solution for the task as I have outlined. I am not interested in considerations of future enhancements or expansions of functionality.
PLEASE:
Tell me the best overall approach that uses readily available technology. If there is an example program that fits my needs, please point that out.
Tell me an alternate implementation choice that uses C++.
If you want to get up and running quickly and reliably, I would build a WCF service in C# with a basic http endpoint for the 3rd party to post to.
Then use MSMQ between the WCF service and your app. Your app listens for the MSMQ events.
In this arrangement, the 3rd party posts to Wcf and WCF puts the data in the MSMQ queue where your app immediately picks up and processes.
If you need this in C++, and the HTTP server end will run on a windows machine (server edition, but could be a home edition if really necessary).
1) Build the HTTP server side using the Windows HTTP Server API (http://msdn.microsoft.com/en-us/libr...(v=vs.85).aspx)
2) Build the client side web request using WinHTTP (http://msdn.microsoft.com/en-us/libr...(v=vs.85).aspx)
1 and 2 are different things and could be created as separate apps, or if needed you can combine them into a single APP. There are basic samples for both in MSDN.
If the POST happens in JSON and/or requires OAuth, then the C++ REST SDK (Casablanca, see http://casablanca.codeplex.com/) might be an easier alternative to Windows HTTP Server API. A slightly added bonus is that you could use this for both the client and the server side. However if it's not JSON, it could be very complicated. Also note that Casablanca, while very usable, is not a fully matured API just yet.
The above does NOT require the presense of a separate web server application. (I think the WCF solution Arjay posted requires IIS, but I'm not sure, I'm not that into C#)
If there's already a web server running, then there might be even simpler solutions by writing an extention/script for that instead.
But like said, the above is just a part of the picture... the programming side, there are issues not related to the actual programming that will need to be resolved as well. Such as the fixed IP (you can have a server on a dynamic IP, but that'll require some communication between the client and 3rd party to refresh the IP), basic server security, internet security/firewalls, port forwarding etc... What do do when communication fails entirely (no internet for hours), ...
Some of those decisions may have a big impact on the programming side of things.
I have found someone else in our company to do this work. They say they understand and it won't take them very long to do the complete project. They are still thinking about the details, but they will probably host a WCF service.
I WANT TO THANK everyone that has helped explain this to me. Thank you again.
WCF doesn't require IIS.Quote:
Originally Posted by OReubens
WCF can be hosted in multiple hosts:
IIS
Console app (.net)
Windows Form app (.net)
Windows Presentation app (.net)
Windows Service (.net)
Windows Activation Service
The thing with WCF is that it's about exposing full-fleged web service interface, but not just HTTP POST handler. Which is alot different situation for data posting side.
Personally, I would prefer to code up something that is more generic (e.g. WCF) rather than coding up something very specific like a POST handler.Quote:
Originally Posted by Igor Vartanov
Although that maybe all that is needed *today*, we all know that requirements change - especially when dealing with 3rd parties.
There is definitely more than one way to skin a cat, but I generally will pick off-the-shelf solutions when I can.
Talking about personal preferences, I would prefer to discuss requirements and architectural approaches with customers and get into details. But in reality sometimes you find your customers being uncooperative to the level you hardly can explain, or even playing real freaks unwilling to say a word in addition to the couple of lines they call a requirement. Humans are so much humans.. :)
As for POST, I participated in a project where cloud API was built around POST + JSON, and that combination was really generic and flexible.
Agreed, sometimes we are forced to deal with less than ideal requirements.