ISP Cache problem By SW The connectivity problems that I experienced recently were due to NTL's policy of using transparent web proxy caches on all connections. I expect the freeserve problems are something similar. NTL's network (and Blueyonder, Freeserve, et al) intercepts all attempts to connect to the web (by means of the HTTP protocol on TCP port 80), and routes the request through a transparent web proxy cache. NTLworld uses Inktomi Traffic Servers. Blueyonder uses NetApp NetCaches. Obviously, they do this in order to reduce the traffic on their backbone, and to reduce the traffic they have to pay for on other operators' networks. Dial-up users all use one huge common cache, but cable users are treated to individual, personal, virtual proxies. I guess this is to try to buffer the connection a bit and make better use of the bandwidth. As with all caches, there is a problem if the cached pages are stale. Ctrl-Refresh is the usual solution, but herein lies the problem: As of March 2001, transparent web proxy caches in some regions of the NTL network have been reconfigured so that Ctrl-Refresh no longer forces a reload from the original server, unless the original page has been updated. On the face of it, this is fine since there should be no need to refresh the cache unless the original page has changed... BUT what happens when a cached page is corrupt? In my experience, the result is that you can NEVER refresh the cached page from IE using Ctrl-Refresh! However, there is hope. You can use Netscape to bail you out. In Netscape, clicking Reload always causes the proxy cache to conditionally refresh the page from the original server. Clicking Shift-Reload forces the proxy cache to unconditionally refresh the page from the original server. Thereafter, the newly-refreshed, and hence, newly uncorrupted page can be viewed as before from IE. In cases where the transparent proxy cache is a Network Appliance NetCache, such as on the Blueyonder network, to ensure that you get a full refresh from the original server, you should perform a full browser refresh (Ctrl-Refresh in IE, or Shift-Reload in Netscape) twice, with the second one following the first after a gap of at least 1 and not more than 10 seconds. The biggest problem is when the transparent proxy breaks down. Then users lose all contact with the web. Because it is a transparent cache, there is no setting you can adjust on your computer to cause it to go direct to the remote site rather than via the proxy. You can however defeat the transparent web cache by configuring your web browser explicitly to use another proxy cache that accepts requests on a port other than 80 (typically 3128 or 8080). Because the request is made on a port other than 80, the transparent cache does not intervene. NB: there will be performance issues associated with using a proxy cache outside your ISP's network. So you'll want to revert to normal working when your ISP's system has recovered. The following procedure works on all Windows platforms running Internet Explorer 5.x, and the procedure for Internet Explorer 4.x is very similar: * Open Control Panel. * Double-click Internet Options. * Click on the tab Connections. * Click on button LAN Settings. o UNcheck Automatically detect settings. o UNcheck Use automatic configuration script. o Check Use a proxy server but ensure that the boxes are left blank. o Click on button Advanced. * In the row HTTP fill in webcache.bt.net and port 8080. * Ensure that the other rows are left blank. * UNcheck Use the same proxy server for all protocols. * In the Exceptions box enter: 192.168.* * Click OK. o Click OK to exit LAN settings. * Click OK to apply the changes and exit Internet Options. The last few stages in Advanced avoid using the proxy for Secure HTTPS and FTP traffic, and ensure that you retain local access to the Surfboard diagnostics pages and to (e.g.) the Linksys setup pages, if you have a router. It is not necessary to use webcache.bt.net in the example above: any of the following NTL proxy caches might work (using port 8080) even when your default transparent proxy is broken: inktomi1-bel.server.ntl.com (Belfast) inktomi2-bel.server.ntl.com inktomi1-bir.server.ntl.com (Birmingham) inktomi2-bir.server.ntl.com inktomi1-bre.server.ntl.com (Brentford) inktomi2-bre.server.ntl.com inktomi1-bri.server.ntl.com (Bristol) inktomi2-bri.server.ntl.com inktomi1-cam.server.ntl.com (Cambridge) inktomi2-cam.server.ntl.com inktomi1-car.server.ntl.com (Cardiff) inktomi2-car.server.ntl.com inktomi1-dub.server.ntl.com (Dublin) inktomi2-dub.server.ntl.com inktomi1-edi.server.ntl.com (Edinburgh) inktomi2-edi.server.ntl.com inktomi1-gui.server.ntl.com (Guildford) inktomi2-gui.server.ntl.com inktomi1-hud.server.ntl.com (Huddersfield) inktomi2-hud.server.ntl.com inktomi1-lee.server.ntl.com (Leeds) inktomi2-lee.server.ntl.com inktomi1-lut.server.ntl.com (Luton) inktomi2-lut.server.ntl.com inktomi1-man.server.ntl.com (Manchester) inktomi2-man.server.ntl.com inktomi1-mid.server.ntl.com (Middlesbrough) inktomi2-mid.server.ntl.com inktomi1-nor.server.ntl.com (Northampton) inktomi2-nor.server.ntl.com inktomi1-not.server.ntl.com (Nottingham) inktomi2-not.server.ntl.com inktomi1-oxf.server.ntl.com (Oxford) inktomi2-oxf.server.ntl.com inktomi1-ren.server.ntl.com (Renfrew) inktomi2-ren.server.ntl.com inktomi1-win.server.ntl.com (Winnersh) inktomi2-win.server.ntl.com To return to normal operation with no explicit web proxy: * Open Control Panel. * Double-click Internet Options. * Click on the tab Connections. * Click on button LAN Settings. o UNcheck Use a proxy server. o Click OK to exit LAN settings. * Click OK to exit Internet Options. * Beware: the above procedures change the HTTP proxy setting for only Internet Explorer. Other programs which use the HTTP protocol (Netscape, RealPlayer, QuickTime Player, etc) have their own proxy settings which need to be set by their own methods. In my specific case, where I have my own proxy/cache/firewall sitting between my network and my ISP, I was unable to use the solution above since the proxy setting of all my browsers already pointed to my own proxy cache. Instead, I used an extension to my proxy software which allowed me to force my proxy to route all traffic to another external "upstream proxy" which I set up as per the tip above. All web traffic from my network now passes through inktomi1-cam.server.ntl.com since both my local (Nottingham) proxies hold corrupted pages for www.thevalkyrie.com which have never successfully been cleared. So that's how I fixed it..