[Home]MeatballServer

MeatballWiki | RecentChanges | Random Page | Indices | Categories

Historical discussion

This discussion was based on server load problems from the beginning of 2004. We resolved the problem temporarily by moving to VPS Colo. However, in 2006, after switching to an RDBMS structure underneath the server started to crash repeatedly when it ran out of memory. In August 2006, the file system crashed when the server ran out of hard drive space. Shortly after that crash, SunirShah opened discussion on the MeatballBoard who would become responsible for sorting out this mess.

See MeatballServerLayout for a technical description of the server.

CategoryMeatballWiki CategoryMeatballWikiProposal

During the last months every now and then we run low on memory and disk space. Maybe we should move to another server. Where should we look for one? Country? How much are the current monthly costs? -- MarkusLude

Agreed. VPSColo blows. It's costing me $50/mo. -- SunirShah

University hosting offer

May 26th, 2008: I work at a university as technical staff and I'm sure that it would be possible to get our faculty to host such project as Meatball, cost-free, without bandwidth limits, with the university staff support and pretty good resources. It's also a long-term solution, as once it's agreed upon, it's unlikely to ever be reconsidered again. If the Meatball community is interested in such a solution, I will go and try to get the acceptance of authorities -- namely the dean of our faculty. Please reply if you are interested in the details. -- RadomirDopieralski

I'd be willing to consider that, if I find out more what the implications are, and what kind of freedom of expression we'd have as well as freedom to continue to adapt the server. Eventually, I'd like to move the system to something similar to BibWiki. -- SunirShah

Sorry for not responding so long, I'm in the process of obtaining the exact rules they play by, and as this involves university's administration, it's painfully slow. I'm not sure how BibWiki setup looks like from inside, but I'm sure I can recreate pretty much any sane setup. -- RadomirDopieralski

Seems like there are no set rules and everything is under control of the person who's picking up the phone at the moment. I haven't managed to get any definitive answer or guarantees. -- RadomirDopieralski

Linode offer and reply:

April 25th, 2004: Hi there - I'm a friend of Bayle's. He mentioned you were looking for hosting. I have a consortium of 3 people and we are renting a virtual Linux box over which we have full root access - see [linode.com]. Depending on your requirements, I would be happy to share the box with you for a small fee. For $10/month you would get 12.5Gb/month bandwidth and 2.25Gb disk space. As many domains as you want, any software can be installed. If you need more bandwidth, we can scale up in integer multiples. Eventually I'd like to have enough people to rent a dedicated server for about $100/month (was thinking managed.com for $60/month but they have bad reviews...). Presently I'm hosting [1] including a Tikiwiki [2] and the server load is very low (usually 0.00, occasionally goes up to 0.25) -- MatthewLloyd

Thank you for the offer, but I think we (CliffordAdams and SunirShah) have already found a solution to our hosting needs in the short term. I am currently testing a VPS (virtual Linux box) at [VPSColo]. Both wikiview.com and meatballsociety.org are currently being hosted there using the minimum packagage at $39/month. Unless we run into problems I hope to move usemod.com (including Meatball) around the end of May 2004. VPSColo has very good reviews, and the few times I've needed support they have been quick to respond.

I was planning to post more information about this decision after doing some load-testing in the next couple weeks. Finally, I really appreciate the offer, and I will definitely keep it in mind if VPSColo doesn't work out for us. --CliffordAdams

Previous Announcements

January 18, 2004: I'm no longer considering the "Jimbo" option below. I hope to stress-test the mesopia.com server in the next few weeks. --CliffordAdams

January 1, 2004: I (CliffordAdams) am seriously considering asking (via StephenGilbert) Jimbo (WikiPedia:Jimbo_Wales) to host usemod.com within the next few months. (See the discussion below.) This move would include the Meatball wiki as part of usemod.com. If Jimbo agrees to host usemod.com, I would have no objections to Meatball staying with usemod.com indefinitely. (The Meatball community could always move later, especially if the community acquires a new MeatballDomain.)

Before I send email to StephenGilbert, does anyone have suggestions or comments? I will try to make any move as painless as possible. --CliffordAdams

If hosting with Jimbo doesn't work out, my backup plan is to move usemod.com to a cheaper provider. (I am paying $40/month currently.) [Mesopia.com] is my current favorite. In the next few weeks I hope to test their servers with my brand-new "wikiview.com" domain. (I plan to use this domain to play around with a few post-UseModWiki ideas and as online backup space.) --CliffordAdams

Hmm.. looks pretty sexy. If it works out well, I might switch sunir.org there as well. -- SunirShah

Earlier Comments

[The text below is mostly from early December 2003.]

Will Cliff still be the host?

Cliff, do you want us to actually host ourselves, or is paying for your hosting services an option? Finding the time is as hard as finding the money for us, so if it's almost as easy for you to host two wikis as one, maybe that would be the way to go.

In the near term (early 2004) I was planning to host Meatball anyway, regardless of payment. The main problem is that I plan to be less available over the next year, and later in the year I may take long breaks completely offline (without even checking email for 2-3 weeks). This could be a problem if I am the official contact for technical problems. (For example, suppose a script error causes mb.pl to use lots of CPU. The admins might disable the script and send me email about this.)

In the longer term I would like to conclude my formal involvement with the Meatball wiki before my absence becomes a problem.

The money is a relatively small factor for me, although it is a factor since I am currently unemployed (and enjoying it greatly :-). If anyone would like to donate, I would prefer that they give something extra to a charity and let me know. --CliffordAdams

Can we do it for free?

Another route that we should think about is doing it cheaply, i.e. not actually paying for hosting somehow. For example, we could host it off someone's old computer and a broadband connection.

My old computer is a 166mhz, and my broadband connection is not reliable (sometimes it cuts out and requires someone to call the cable modem company to tell them to reset something or other). And I will not make myself available 24/7 to deal with server problems (in fact, I probably wouldn't be able to deal with them immediately at any time of the day). And, someday, my old computer will presumably break. But, did I mention it's free? Does anyone else have an old computer and a connection to offer?

-- BayleShanks

A better idea would be to ask [ibiblio] to host us. They have a fairly liberal collections policy (basically, it has to be informative, of value to the Internet community, and non-commericial). They're hosting Fred Bauder's Internet Encyclopedia wiki, so surely they'd host somethng that isn't a knock-off of something else. ;-) -- StephenGilbert

Can we reduce server load?

I'm not sure that an old system will even be able to keep up with the current load. Even a simple page load takes about 0.3 CPU-seconds on the current system (a 900MHZ P-3), and loading RecentChanges takes even longer. Peak hourly loads are currently about 1000 pages/hour, but that has been increasing, and a fair amount of the traffic is RSS feeds (which take more CPU than page loads). --CliffordAdams

I am willing to disable RSS feeds. I don't think they are very useful. -- SunirShah

I use them sometimes, although I guess I could use WikiGateway to write a proxy that would translate RSS requests into RecentChanges requests. Does RSS take significantly more processing than loading RecentChanges? If limits are needed, let's limit them to once per hour per IP. -- BayleShanks

Maybe we could run a survey. Is this a sufficiently UgLy link? SURVEYAreYouReadingThisRSSFeed??

Better software might reduce load - even something as simple as reducing the days on RecentChanges or a RecentChangesCache.

I just did a few things that should help:

Hopefully all of this will help a little. The changes are commented in the meatball "config" file, if any other admin wants to change them again. --CliffordAdams

Could we share a dedicated server with other projects?

Another possibility could be for some people who are currently paying for wiki hosting to cooperate and buy a dedicated (possibly virtual) server to host multiple projects. I recall seeing virtual-dedicated offers as low as $70/month on K5, and real dedicated servers as low as $129/month [3]. This would require some coordination between projects, but would be very flexible.

Can we find someone to replace Cliff?

Yet another route is to look for a different site to piggyback with. Alex, is however you are hosting the emacswiki domain available to us? If you are paying, could we move over there and pitch in? Would Helmut or anyone else running a hosting business be an option for us?

Another option is to ask Jimbo Wales about hosting. He has offered various times in the past to set up wikis for people on Wikipedia's servers. Ususually these people have been somewhat troublesome on Wikipedia, trying to turn it into something it isn't, so I'm guessing he would be willing to host a friendly project like MeatballWiki. The downside is that we may have to share space with the English Wikipedia, which is a notorious resource hog. However, the Foundation just put a new, big, bad server in place to take the load off our other two, so things are much better (for now). If we want to look at this option, let me know and I'll talk with Jimbo about it. -- StephenGilbert

I had also thought of asking Jimbo, but I wonder if it would be appropriate for this community. I tend to think of Meatball as more like a private club than an public space. (People who don't follow the local customs are told to leave.) I don't think Meatball would work well with the Wikimedia Foundation, but it might work as a private matter between the Meatball leaders and Jimbo. --CliffordAdams

It wouldn't involve any official association with the Wikimedia Foundation; MeatballWiki just doesn' fit. Jimmy's previous offers were to simply set up a wiki and provide server space and bandwidth. -- StephenGilbert

Or do we have to pay full price?

If we go for the normal commercial solution, we need to determine how to pay for it. I think we should contribute enough to account for two years out, so that whoever is actually paying the bill doesn't have to scramble around for funds every few months. On the other hand, assuming we don't want to go through the rigamarole of setting up a NonProfit, the point man may be bumped into a different tax bracket if they suddenly get thousands of dollars of contributions, so maybe we would want to spread it out as much as possible.

If we want to do this, I'll pledge $500 (1 year's worth) to start out, provided that at least 7 others match me within a reasonable amount of time. In total, that would be enough for 8 years, or about $70/person/year (most likely less if we could get other people to contribute too, even if not as much). I would probably pay in installments rather than $500 upfront. I'd reserve my right to retract my pledge for future installments because of dire need on my end, or a big disagreement with how it is being spent.

I'll also pledge some money if we pay one of ourselves less than $40/mo to host us, but the amount may be different.

-- BayleShanks

Until I create a NonProfit to account for the money, I will not accept pledges at this level of commitment because that opens me up for malfeasance. I'd rather pay for it myself. -- SunirShah

Well, you wouldn't have to be the point man. If you want to pay for it all yourself, so much the better for the rest of us :) You shouldn't have to, though.

-- BayleShanks

Should we find a cheaper provider?

I chose futurequest.net after finding that two other cheap providers could not provide adequate service for usemod.com (even when it was much smaller). Other providers may be able to host meatball more cheaply. For example, one UseModWiki user recommended [lizardhill.com] which would cost only $120/year. (Their terms of service are not very CGI-friendly, but the user thought he could convince the owner to make an exception.) Perhaps other people know site owners who could make similar deals. --CliffordAdams

Another recommended provider was [modhost.com]. They have a plan for less than $100/year which would give 1000 MB of space and 30GB of bandwidth, and their CGI policy seems reasonable (about the same as futurequest.net). Modhost might be overpacking their systems with users, but I think it's worth a try. Since modhost.com has no setup fees, I might just risk 10.75 for a single month and experiment with it. (I will not move usemod.com/Meatball without public discussion, but I might make a private copy for testing.) --CliffordAdams

Discussion

I would suggest that Meatball remain independent of Wales for a number of reasons, and would be willing to offer a degree of financial support in furtherance of that recommendation. I recommend a move to a commercial hosting company.

To be candid, Wales' server situation is, well, a disaster at the present time and has been for almost a year. While he and his volunteers have been working diligently to address that, no net headway has been made---everything is still slow and there is a good deal of downtime. I realize that there are reasons why, but the bottom line remains. Moreover, Wales himself has much else to do and to all appearances doesn't have much time for another project.

With MeatBall hosted by Wales, the idea that so much of the Wiki community could end up with a single point of failure, both hardware-wise and management-wise, does not sit well.

I have found that most of the commercial web hosting providers are realiable and reasonably priced. Unlike Wales they have hundreds of PCs or perhaps thousands. 4 PCs to a blade, 16 blades to a rack, 10 racks to a cabinet, 10 cabinets to a lineup, and 6 lineups in the server room. Hardware problems? They swap blades and reload from last night's tape, and with 36,000 PCs they have ten of them go out every day so they stay in good practice.

The main problem is customer support, which is as a rule poor and varies widely, and getting a reasonable deal that includes the CGI capabilies and traffic that MeatBall needs.

--(anon)

Actually, the more I think about it, the more I agree with (anon). Jimmy might agree to host on on [Bomis] servers (which are stable), but I really think ibiblio is the best option, and then a commercial host if that falls through. -- StephenGilbert

In addition, the purpose of the Wikimedia foundation is supposed to be chiefly open content, so meatball might be a difficult fit from that point of view too. -- MartinHarper

I don't think it is a good idea to introduce stakeholders into this project that aren't 100% aligned with it. Why give up control to third parties when you don't have to? That reduces our freedom to be creative. The last thing that Meatball needs to do is be absorbed by the very processes it occasionally criticizes. -- SunirShah

Proposal

I am part of an arts collective doing WearableComputing. Would it be off-base to use meatballsociety.org to host its project files? I'd very much like to use meatballsociety.org as some sort of shared space for laterally connected projects. I recently installed CVS on the server and am trying to get it up and running, for instance. The long term goal that I would like to follow is to create a laboratory for FairSoftware and other ParticipatoryDesign projects, to see what kind of support they need. -- SunirShah

MeatballSociety? and meatballsociety.org are currently just placeholders for things to come. They may or may not be related to these projects. You have to build a vision how these fit together. I would advise to buy domains for any of these projects (I didn't for a number of my own projects and it is a burden to change to domains later - you have to break a lot of links). People also like separate domains, they add some trust to their projects. -- HelmutLeitner

Alt+I is now the accesskey shortcut for "edit text of this page", and Alt+P is the shortcut for "preview".

We could add more as well, like alt-s to save, alt-r for RandomPage, alt-c for RecentChanges, alt-f for the search box, alt-h for the history. -- SunirShah

This ties in to MeatballWikiSoftwareDiscussion as well. Check out:

http://meatballsociety.org/cgi-bin/mbnew.pl

Concerns:

    1. Moving to the new engine and then BarnRaising text formatting fixes on the text; or
    2. Having someone tweak the engine to suit our needs (volunteers)?
I'm in favor of #1 but willing to go with either if we get a consensus.

Ping. Lack of response does not constitute a mandate. ;) -- SM

Discussion

MeatballWiki | RecentChanges | Random Page | Indices | Categories
Edit text of this page | View other revisions
Search: