Sunday, 10 January 2010

Eve Online: A second server for Eve?

If you follow Eve Online in the slightest way then you will know one of the game's proudest features is its single server. Developers and players alike are rightfully proud of this feature which adds complexity and depth to both the economy and the territorial empire-building game.

If you've played Eve recently you may be starting to suspect that this is not going to last.

The server crashed at about 10pm last night. 54000 players were logged on, the market was bustling, the war in zero,zero rages. The game is starting to not cope with its own popularity.

The recent Dominion expansion places more load upon the server. Planets look gorgeous now and are lushly rendered. A bug has been accidentally introduced with grids that causes entire fleets to black screen and be killed by their enemies while loading.

One Eve veteran enthusiast from the F13 forums had this to say:

I just spent 3 hours in game staring at a black screen. when IT bombed is from 100kms away. This despite everyone else in eve if you fire bombs in heavy lag the bomb explodes in your face. Before we went in AAA lost 40 bombers that way.

With IT they fired bombs and they went 100 kms and exploded right on top of us. Perfectly. Then my logistics ship was shot to bits in real time without me able to do a damn thing.

Then they are able to move around probe down capitals and destroy them with bombers. Then trap titans, siege blow them up etc. While I'm staring at a blank screen and playing stuff on my laptop.


I'm sorry if I come across as mad but in Eve I am a warrior. If I die in a fight, big deal. Dying while not even loading grid for 2 hours?

Sorry for venting here, dominion has brought back the old shitty bob lag. I'll give it it whoever planned this. They knew that they would have a jammed up node when they paid off that SOT corp and knew they would be able to drop a tcu and get it on-lined before the node could be reinforced at downtime. That's a serious exploit as far as I am concerned. And quite frankly I'm tired, I've been up all night for this and I don't feel like giving ccp my money for this crap. And I don't want to face another night like tonight ever again. I'm getting too old for this shit.

I think I'll do something more fun like have an ingrown toenail. Oh and I've just been told the titans never even saw themselves get shot.

I think I'll go install LOTRO now.

Now bugs can be fixed of course and more computing capacity can be purchased. But there's a
limit to how far and how fast Eve can grow. They already run the game on one of the most sophisticated supercomputers in the world.

And I think the Eve playerbase will grow and grow. Sure there are a couple of strong space MMOs coming out this year. SWTOR and STO will draw large crowds. But those games won't draw off many Eve players - Eve is a strategic pvp experience and those games are tactical pve experiences..

Also Eve is designed to support being put on a backburner - if I get gripped by TOR fever then my Eve accounts will sit there nicely on their own, paying for themselves with passive income. The plex system really helps the game here. If you are a plex seller then you're used to spending a lot on Eve in exchange for being awesome. If you're a plex buyer then you play for free and it will probably take minimal effort to cover the monthly gametime. Combine that with a skill system that rewards people for real life time and you have a game where you may as well stay subbed even if you're not actively playing..

So if Eve will keep growing both in player numbers and in graphical complexity then a second server becomes inevitable. And it's not like there isn't some precedent: the Chinese play Eve on a separate server and there is also a separate Test server.

Edit: this issue provoked some very good responses, basically saying the answer is No and well worth reading. Links here in case people skip the comments:


  1. A second server doesn't become inevitable. What becomes inevitable is for them to find a way of running a single system from multiple servers at once to split to the load, and be able to do it dynamically.

    A second server will cost them a lot of subscriptions. It would certainly make me more likely to give the game up

  2. Did you have a look at the Massively article I linked Belamar? They already run one of the most expensive and powerful server set-ups in the world. They already run the system from multiple servers at once to split the load.

    I can see that people won't necessarily want it but if the game's popularity keeps growing then surely it's either open a second server or turn customers away.

    Or possibly you're right and I'm wrong and there is a technological solution.

    But if so they need to implement it now as it certainly feels like the current set-up is struggling.

  3. Belmar means each individual solar system, i.e. node, needs a way to be spread across the blade servers. Right now at best you get one node per server in the cluster.

    My response to your post:

    TLDR: I think you are wrong :)

  4. They run things from multiple servers, but the issue is that a single system has to be physically located on a single server. They can move the system to a more powerful server during DT (which is what the fleet fight notification function is for), but they can't run every single system on a server with the same power as Jita.

    In the shorter term, they will need to find a way of dynamically moving a system to a high powered server without relying on DT. In the longer term, the ability to load share a single system on multiple cores/servers will let them scale dynamically as needed. This will let them effectively scale the game as fast as they grow. They don't have that ability currently, as current hardware limits the maximum population of any given system

    Both of those options are much more feasable than starting again, undermining one of their key selling points and simply delaying rather than solving the original problem.

    The other thing to bear in mind is that the current setup is not the game at its best performance. It had better performance with the last patch, and I'm sure they're trying to address the change that has introduced the heavy lag and get it back to previous performance.

    I can only hope that they are looking for a more long term solution though

  5. The problem is the changes to the 64 bit database. Having played EvE as long as I have the game performed flawlessly once they optimized the code last year( on 32 bit) their issue is that the database is having no it is not a bug but a design issue. Does it affect people..yes it does, grin and bear it.

  6. Dominion introduced a new type of lag which is not simply a result of more players; anyone who has taken part in large battles before dominion and has experienced the new lag or heard it described will tell you it is not the same thing. To me it sounds like a software issue.

  7. Thanks for your comments everyone and for your blog post response Kirith.

    I suspect it's both a hardware and a software issue, I don't think you can simply dismiss the growing busyness of Jita as a patch bug but the grid loading blackout is very clearly a Dominion-related bug.

    What i think has emerged very strongly from both the comments here and the comments at Inner Sanctum of the Ninveah is that Eve players are vehemently opposed to this ever happening.

    Let's hope that the pace of technological progress outstrips the growth in the number of players.

    From a business point of view it's a good problem for CCP to have.

  8. James Harrison has made a blog post that basically says what I was trying to say in my replies to this post. His blog entry says it better though :)

  9. That's terrific, thank you Belamar.

    I'll link to both this blog and Kirith's in my OP just so if people find this blog but don't read the comments they don't miss them.

  10. CCP have marketed a lot on the fact that EVE operates on a single server so opening a second server will be, well, insane for them I think. I can't image anyone would want to play on it anyway :) Everyone would pick the "real" server!

    I'm sure they wil find a way around the problem. They spoke before about wanting 60,000 players online at once so I'm sure they're prepared for it.

  11. Took me about 3 attempts to post a comment on your site btw :( It's keep wiping my comment and not showing me the captcha dialogue :(

  12. As an EVE tourist who played for five days of the welcome back offer after not playing for more than a year I actually though lag had improved. The most striking example was Jita. Last time I played it took up to five minutes to zone in to that hell hole. This Christmas with 50,00 people playing and a significant number of that 50k stuck in Jita I experienced no lag.

    It seems to me that CCP recognised the lag problem in Jita and did something to fix it. That gives me hope they can fix lag issues elsewhere in the game when large numbers of players converge.