[ back to toc ]


Date: 2002/04/26 09:39

Dear Peter,

I am writing a code for my website to search multi-site and fetch and
compare the prices of them. I am using LWP::parallel::useraget to submit
the search for the prices to multi-site simultaneously. The code did well.
My question is how I can real-time show the results to client. For
example, I fetch three sites A, B, C at the same time, and A site give the
reponse first, I need to show to visitor the result from A instantly,
while waiting the results from B, C.

I really appreciate you if you could answer this question or give me some
hints on it.


Well, you can not. You can fetch the results asyncronous in some other
process started from crontab from time to time or you can perform the
fetch in the CGI process and display only a single page.

However you have to treat the web page that appears as a response to the
query as a single entity appearing at the client at once when the page is
ready. This is the general and safe approach that you should follow. Any
other assumption is dependent on the browser type, network band witdth and
other factors.


Thanks for your prompt response. Please take a look at this website:
http://www.addall.com. When you search a book for the best price, it will
show the following info dynamically in real time to tell you their search
process. What do you think how they implement this? Thanks.

7 bookstores searched .. 27 to go
erhamilton.com: Can't connect to erhamilton.com:80 (Operation now in
12 bookstores searched .. 22 to go
18 bookstores searched .. 16 to go
24 bookstores searched .. 10 to go

As I wrote:

>Any other assumption is dependent on the browser type,
>network band witdth and other factors.

Their solution is not general, works with the web server they developed it
for, most probably not a simple CGI application but an application that
tightly integrated with the web server or uses some special feature of the
web server.

The simplest way is to issue the questions in asynchronous mode, thus the
program does not wait for the answer but issues all other questions in
parallel and waits for any of them to finish. When one finishes the
program can send the output to the client socket (which is handlen in CGI
using the standard output, STDOUT). This data may immediately be displayed
by the browser as soon as it arrives, but it also depends on the HTML
context. Based on the HTML context the browser may (for example Netscape
4.7 in a table) wait for all the content to arrive before rendering the

It is also a question how can you force the web server and the operating
system NOT-TO buffer the standard output but as soon as data comes send it

Thus this is really a very fine tuned, low level, try and error Web page
programming that does not really pay back in my humble opinion.


[ back to toc ]