Merging HTTP reuquests that arrives in several packets

Joseph M. Newcomer

2011-07-24 22:06:12 UTC

The issue is that you ignore the concept of packet entirely. Pretend at the highest level
that you never heard of it.

An HTTP request needs to read a number of bytes. You read bytes until you get the text
that tells you the length of the packet. Then you read no more than that many bytes of
the data packet. If you have to do 20 receives to accomplish this, so what? If you can
do it by using what you have already received, so what? The fact that you even think you
need to care about the packetizing means you are thinking about the problem incorrectly.
HTTP is a stream protocol, and all you do is implement the stream semantics. Let some
lower level of buffering handle the packet issues for you.

There is no magic. If you need to get more bytes, you call some lower-level routine with
a request to read bytes; default to the size of your buffer. Ultimately, you build up a
stream, at a higher level, which is just one HTTP packet, and all packetizing, parsing,
etc. are handled by a separate level which itself does not worry about packets, only
reading streams.

For example, think of this as reading an HTTP stream from a file. You read bytes until
you form a complete HTTP request, even though the file might have 20 of them in it. Then,
having processed that request, you read the next one, and so on. So you have to think
about splitting the levels out in a sane way to accomplish this. But how an fread()
relates to an HTTP packet on disk is the same as Receive() relates to an HTTP packet
coming in over the network.
joe

Post by dushkin
Hi,
I have a task to simulate a web server which receieves http requests.
Now the http requests can arrive, as you surly know, in any kind of
1. All request in one packet
2. A header in one packet and its content on the next one
3. And also multiple fragmets on the requests on multiple packets.
4. (Can they also come in packets including the end of one request
following the start of another one?)
Can you give me some pointers to a smart algorithm (or maybe existsing
class that you know) that can process those packets into a complete
HTTP request?
----------------------------------
Packet1: GET /services/org.openmobilealliance.pres-content/users
rcs_status_icon HTTP/1.1\r\nAccept: */*\r\nContent-Type: application/
vnd.oma.pres-content+xml; charset="utf-8"
Cache-Control: no-cache
Packet3: Pragma: no-cache
User-Agent: Java/1.6.0
Host: 192.168.5.135:8080
Connection: keep-alive
----------------------------------
will give
GET /services/org.openmobilealliance.pres-content/users/
1.1
Accept: */*
Content-Type: application/vnd.oma.pres-content+xml; charset="utf-8"
Cache-Control: no-cache
Pragma: no-cache
User-Agent: Java/1.6.0
Host: 192.168.5.135:8080
Connection: keep-alive
----------------------------------
Of course the packets can be more complicated (starting from the
middle of the packets, or containg some content (like XMLs - that can
be by itseld truncated into several packets)
Many thanks!

Joseph M. Newcomer [MVP]
email: ***@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm