Diana the Valkyrie

The cost of running a web site

Some people have asked about the cost of running a web site. It really depends on how big it is.

First let me explain what "big" means. It isn't the amount of data and graphics that matters, it's the telecoms that are the main cost. But when you have a lot of content (I have 80 million files on more than 5000 gigabytes of space) then the cost of hardware also becomes an issue.

You can run a web site for free, if you get free web space from your service provider. Typically, you'll get between 1mb and 10mb of space. What they don't tell you, is that if you start getting a lot of hits (people accessing your site) then they change their minds.

For a small site (where you serve a hundred megabytes per month or so) you can go to one of the cheap service providers, and pay $60 per year (you can pay a lot more), to get web space. They call this a personal site, meaning they aren't expecting it to get many hits.

As soon as you talk about a large site, the economics change. Large means when you get a lot of hits, the megabytes of space isn't the issue. Many Internet Service Providers have a single T1 line (AT&T call it a T1.5, it's also called a DS1), which means that their maximum throughput, for all their users combined, is 1.544 (or is it 1.536, I find both numbers, even in the same document) megabits per second, which is about 700 megabytes per hour. You can find out more about T1 lines if you like. Since load patterns are never even (peak load at 7-8 pm is about three times minimum load at 5-6 am) this means that a site that absorbs 600 mb of load during the peak hour, has exhausted the capacity of the line. As a rough rule of thumb, a T1 will support about 10 gigabytes per day.

Take Mindspring for example. $100 per month gets you the biggest service they offer. That gives you 75 mb of disk space (which isn't the issue, but I want at least 5,000,000 megabytes) and 6000 megabytes of hits per month. This web site does that much in three hours. So what happens after the site has served 6000 megabytes? Each extra mb costs $.08 down to $.04, that's $40 per gigabyte. So 10 gigabytes per day is $400/day, $12,000 per month. Wow!!! But that's what happens when you buy retail. I buy my bandwidth wholesale, I pay a lot less than that. If you're thinking of setting up a web server, shop around. I also shop around when I buy hardware, you can often buy computers and hard drives for a fraction of what you'd pay if you bought a big name brand.

After T1, you can get T3, which is 45 megabits (44.736 to be precise). Obviously, that costs more, but not 30 times as much. You can get fractional T3

What I actually have in New York, is the equivalent of six T1 connections (it's more complicated that that, it's actually 10 megabits, the same as you get in most local area networks). This lets me serve 1.25 megabytes per second, in theory. In practice, you can never completely fill a pipe, so I might only be able to do 1 megabyte per second. That's 80 gigabytes per day, up at New York.

But New York is only my backup site, I don't usually use it. My main site is at Watford, where I have 100 megabits capacity, which would let me serve 1000 gigabytes per day. I'm currently doing about 50.

My main link from where I work on the site is a "megastream". That is a permanently-linked connection to the net, so that I can run a Secure Server, and various other things from here.

OK, that's bandwidth. You need three other things to make a web site; servers (CPU power), disk space (gigabytes) and labour.

This site runs on several servers, Fiona, Lorna, Tilly, Judy, Bunny, Yenta, News3 and News2 (they, together, do the News stuff). Jayne (main server), Freya (Message Boards, chat and some other stuff), Vicky (Secure Server). Hattie (ftp and DNS). Jayne also does The Valkyrie Channel and the Listen with Diana.

Then there's the pre-processing (making mpgs, jpgs, and suchlike), so there's also Brunhilde, Faith, Ingrid, Katie, Rhoda, Ethel, Betty, Clara, Sadie and Queen but they aren't public-access. Irene monitors all the public computers, and tells me when she sees a major problem.

And there's lots more, and it keeps changing (the list above is probably out of date by the time you read it, I use a spreadsheet just to track the computers). And each computer of importance, has another one that twins it, so that if it goes down, I can quickly switch to the twin. That should mean that outages will be fairly short. Of course, if something big goes wrong on the internet, it's possible that I could have a long outage.

Why do I need so many computers? Because this is a *big* web site. And it's cheaper to get a computer to do work on it, than to hire people. So, a lot of the computers are there because otherwise I'd need a person. I'd estimate there's about 40-50 computers making up this site.

Disk space; Penny, Rosie, Naomi, Lorna are all Valkyrie Mark 2 computers. These are my own design, and each give me 650 gigabytes of space for about $4000. News3 is the main NNTP server for news, with 500 gb. Bunny and Yenta are Mark 2.4 1200 gb; being built more recently than the Mark 2s, they give me 1200gb for $4000. Judy and Xenia store the older news, and are also Mark 2s.

Why am I using my own design? Well I realised that I don't have to hunt around what's available to find what's closest to what I need. I can buy a case, motherboard, memory, hard drives, and use a screwdriver and about an hour to bolt it all together. It was a wonderful feeling of liberation, and it's quite a satisfying feeling when it works. Plus, it's a *lot* cheaper than buying ready-built.

Processing. For the Newsgroups stuff, I use dual-processor computers, because there's a lot of computing to do. For the other Mark 2s, I use the cheaper Athlon cpus, with 3/4 gb memory (memory is cheap, and helps a lot with speed of processing). I also designed a Mark 3, which is a machine that's small in size, but powerful, and is used as a front end to the Mark 2. So, when you access the web site, you talk with a Mark 2, which gets the data you requested from a Mark 3.

And it isn't just servers. There's firewalls, routers, hubs, switches, UPSes, tape backup, scanners, video capture, VCRs, camcorders, digital cameras and an absolute spaghetti of cables.

Labour. I don't count my own time, but sometimes I have to pay other people to do things. Not often, though. I find that I can do most things myself. Those people need to buy groceries, they can't work for nothing.

And, as you probably already know, this web site is sponsoring Nicole Bass, Andrulla Blanchette, Sheila Burgess, Christine Envall and other female bodybuilders, power lifters and other athletes. And we do grants to help people compete at events, and we sponsor events.

And that's the bottom line. You can do a small site for free, because the costs are low, so they don't signify. A big web site costs money to make.

But it's a lot of fun to do!