[MUD-Dev] re: Sun's Sim Server and Gordon's 10 Reasons (the first one :))
ceo at grexengine.com
Wed Mar 31 13:54:53 New Zealand Daylight Time 2004
At the GDC Sun released a new technology, a protype MMOG server system.
I'm guessing quite a few people on this list saw it (I'd be interested
to know what you thought).
It was built by someone at Sun's "Game Technology Group", who have
had a slightly shakey start, being both a rallying point for the
java games dev community, but also burning a lot of bridges with
said community. I mention this only to cushion against the tone of
the quoted post :)...it came after a separate discussion where their
claims to being the only group who had *ever* produced a system with
things like failover and duplication prevention were countered, and
several people asked them why they were trying to roll their own
rather than working with partners.
The developer apparently hadn't heard of Gordon's "10 Reasons" talk from
GDC03, and seemed to feel that the few key problems solves by their
tech was the holy grail for MMOG development that would make it easy and
cost-effective. I disagreed, and asked for his reaction to all the other
issues. This is his blow-by-blow response; it gives some interesting
insights into what Sun's strategy is with this Sim Server:
(note: pasted the 10 responses to the 10 questions below)
His summary is "So in short, which of Gordon's 10 does the Sim
Server adress? My answer is every one thats real!" which I find
somewhat blinkered. It appears he chooses to ignore all the
non-technical problems, including issues of maintenance, customer
support, ongoing development, etc.
Jeff's 10 Answers to Gordon
« on: Today at 1:39am » Quote Modify
Sicne BBB asked for it, I looked it up. here's my 10 answers to
Gordon's (IMO in some cases naive) 'reasons":
JEFF'S 10 Answers to Gordons 10 Reasons not to do a Massively
#10. Too many are being built. Walton compared the current crop of
in-development games to the "RTS frenzy" of a few years ago. It's
a fine genre, but there are just too many in development.
Answer. This is like saying there are too many single player
games. The category is huge. There are too many almost
IDENTICAL online RPGS being built. That I agree with, but thats
a tiny tiny slice of the potential market.
We are seeing so much "me too-ism" mostly because of the
difficutly of building good scalable games today. There is a
(actually pretty lousy) existing model of a scalable RPG in EQ
and everyones basically tryign to copy that one model for fear
of failure if they try something new.
As in ANY place in the game industry if you stick to the safe
ground you will be in with lots of competiotrs. If you go new
places and do new things there are a wealth of new
opportunities. As for their difficulty, well thats what we are
*solving* with the Sim Server.
This actually becomes a reason TO do Massively Multiplayer
games-- unlike the platform game market there are all kinds of
well known genres that havent even been touched yet. Break out
into a new area and you will have 0 competition.
#9. The craft requires mastery of too many disciplines. These
include managing a huge team of dozens of people, customer
service, community relations, network operations, billing,
marketing, and communication and service coherency. Most MMOGs
fail in at least two of these crucial areas, Walton supposed.
The answer to this is simply not to try to do it all. A
technology like the Sim Server allows non networking or parallel
processing savvy engineers to be fully competant massively
scalable server programmers. It "knows about" databases and
multi-processing and such so the game programmer deosnt have to.
Simialrly rather then trying to do all your own customer
service, there are well known (by the rest of the computer
industry) third party solutions you can engage.
HOWEVER the Sim Server helps here too. It provides a common
back end administration interface to many games at once. It also
shares the laod across those games. The result is that one
operations center, and thus a single operations center team, can
handle ALL the back end administration functions for a whole
raft of games.
This makes the epicenter/hosting model a good model for both
developers and hosting centers and offloads that
responsabiltyifrom the developer. All the back office services,
including customer support, can by handled one outsourcer for
any number of games-- splitting costs between all clients and
taking the load off the developer.
#8. It requires a huge time with multiple, diverse skill
sets. These include client, server, database, and Web programming
skills and generating gobs of content. Walton said a game that is
three times bigger is at least 10 times harder to develop.
Not with the Sim Server. It handles all your persistance. It
handles your scalable server design.
You, as the developer, write what appears to be event driven
monothreaded code. All your sim objects automatically persist
and your code gets automatically scaled out across the entire
The result is deadlock proof, race proof, massively scaled code
that is as easy to write as a mono-threaded app.
#7. Getting the credit card from the customer is hard. Not all
customers have credit cards, and consumers are generally
suspicious of online transactions. New customers don't always
fathom the value proposition of an MMOG until they try it.
I don't see this slowing Sony down. Getting the CC is easy if
the user trusts you. People trust Sony so sony ahs no prolbem
This again is where large hosting providers can help. The
customer trusts them and they hanbdle the billign for you. In
addition, they use existing Sun enterprise billign systems which
are already highly secure adding to that sense of trust as well
as providing other ways to bill.
The last point is just silly. How many "cancel in 30 days"
offers has Gordon recieved. Why does he think they structure
things that way? All packaged MM games come with at least a
"free" bundled 30 days for exactly that reason.
#6. Online games are completely counterintuitive to packaged-goods
game company management. MMOGs are essentially launching all the
time with staggered launches and new content being added, rather
than centering around a single one-shot launch as packaged
I'm sorry but this is insulting to the industry. Its like
saying there is no way a book publisher can publish a
magazine. Sure the model is different. Packaged games, like
books,are all about acquisition-- the original sale. Online
games are exactly like magazines, while acquisition is
important. retention is the key to real money.
This is exactly why they are so good for the game industry. In
the old model every game has to be marketed, sold and then the
process is doen all over. And that marketing is very expensive.
Acquisition is the hrad part of any sales mode. With MM online
games once yo uhave acquired subscribers you keep them ,and keep
makign money, as long as you put a reasonable amount into
keeping the service they paid for interesting to them.
But saying publishers can't grok this is calling them low grade
morons who can only mechanically do what they have done before.
This isn't exactly phd level economics here and there is an
existing successful industry to emulate.
#5. Everything developers know from making single-player games is
wrong in MMOGs. Well known formulas of discovery and secrets don't
apply to online communities, and cheats that in a single-player
game affect only the player who chooses to use one can ruin the
experience of hundreds and even thousands of paying subscribers in
a persistent world game. Walton also pointed out the importance of
documentation and maintenance issues to MMOGs that often fall by
the wayside in single-player-game development.
The last one was insulting to people who publish games, this one
is insulting to those who design and develop them.
"Well known formulas" (formulae, properly) are a road straight
to chapter 11 in ANY entertainment medium. People crave new
experience and the only reliable "formula" there has ever been
in entertainment is "be different."
Good game deisgners and developers are pushing the envelope
daily in their work-- thats their job. Sure there are new
challenges in the MM space but thats what makes it interesting
and fun! And the fact that so much design hasn't been done yet
means theres a ton of low hanging fruit before we get to the "oh
god what do we do THIS time" phase.
#4. The Internet sucks as a commercial delivery platform. Not only
that, when players have a bad Internet experience, whatever the
reason, they blame the game providers.
Frankly, this sounds like a bad artist blaming his medium.
All new game paltforms bring with them new challenges and
limitations. Designing around and for them is the essence of
real game design.
I haven't noticed Sony suffering for the problems of the
medium. or Bioware. Are there new challenegs?Sure. Thats what
we are paid for-- to solve them.
#3. Customer service is hard. Walton cited customer service as the
single biggest cost variable in online game development, and the
ramifications of the customer service strategy and project
planning are far-reaching. Walton pointed out that whereas in most
traditional businesses customer service is a cost center whose
expense is to be minimized (like the call center you phone to
complain about your cable bill), in MMOGs it is essentially the
entire business. And that 24x7x365 business is extremely
people-intensive, which by definition is costly and messy.
Okay a few comments. First one is this: "...in MMOGs it (the
call cnter) is essentially the entire business."
To be blunt: Only if you have a crapy service. The vast
majority of the time the customer should be happily using your
service. If they arent then thwres something very wrong with
If telcos operated like MMOLRPGs today do then yes, their call
center woudl be flooded with complaints/questions/issues. but
they don't. They deliver the service reliable, as promised and
Quality of Service on todays MM games is terrible, there is no
doubt. Thats someplace the Sun Sim Server halps. It provides
the tools necessary to allow game developer to reach the "5
nines" that telcos tlak about-- where the service is up and
functioning 99.999% of the time. We do that for the telcos,
we're going to do it for online game services.
Once you are AT the telco stage the question is how do you
minimize the cost of even that? Well again the telcos (who Sun
is very familair with and works intensely with today) have some
standard answers. The first is automation. As much as possible
is handled by computer systems. On the second line, they go to
ecomonies of scale in large call centers. In many cases they
actually outsource these call centers to companies that
specialize in 24x7x365 service. All you need to do this is a
scale of usership thats appropriate.
AND again thats where the Sim Server comes in, by making the
game hosting epicenter possible. With a single administration
team and call center servicing all games installed in the call
center you can share expenses with all the other game publishers
uisng that provider. The hosting provider handles user
accounts, customer service and the rest and you pay a small
amount per account as "your share."
#2. There are lots of legal issues. These issues range from
terms-of-service contracts to end user license agreements,
frivolous lawsuits, the commonplace use of "volunteers" to help
administer the game, IP protection, and the question of legal
ownership of virtual "property." All these laws and regulations
are in constant flux, which put legal issues so high up on
Walton's list. His advice? Get good lawyers and be sure to budget
to protect your IP.
Where has he been? Certainly not in the US.
Anyone who runs an IP based business without an IP and contract
lawyer is 100% certifiably insane. Any packaged game developer
who signs publishers' contracts without a lawyer's assistance
will be out of business within a year.
My parents have had a 2 person IP company (print text and
photography) for 40 years. I grew up knowing the name of their
lawyer as well as I knew the names of any of their friends.
This is a non-issue because anyone in a real business already
has to deal with this, MM or not. Where it comes to the
specifics of TOS and billinf contracts, again a service provider
who handles that for you will have standard stuff of their own
from their pet shark.
and last but not least....
And Gordon Walton's #1 reason You Don't Want to Make a Massively
#1. They cost too much money to build and launch! This of course
is the ultimate gotcha that turns the best laid plans of mice and
game developers to very costly muck. Development costs continue to
rise, and, in Walton's words, "the faster you go, the slower you
Today, he's right. To go online for massively multiplayer you
need to build a machine room designed to handle your maximum
expected load. Thats very expensive.
We put a million dollars into our machine room at TEN before we
even put in any computers. (Switches, racks, climate control,
telco equiptment, fire supression, statid supression, power back
Even if you cheat on that stuff (which will come back to haunt
you) you still have to build out a machine back end itself
capable of handling your maximal load. Failure to do so can
result in the "success disaster" wher you get too many users to
handle, your system goes down in flames, and you gain a
permenant reputation as a crappy game.
Whats worse, that means a lot of equiptemnt sitting ideal adding
to your ongoing operating costs. Try to do fail-over by simple
replication and youve just doubled that cost. Fill it up past
expected maximal capacity and you are in shard-ville and have
another set of mostly unused hardware again.
The Sim Sever addresses this on two levels. To begin with, you
can invest in a minimal set of hardware. Because it scales
symetrically, handling additional load is as simple as calling
up your Sun rep and slapping in some more blades as your user
More so, when deployed in a hosted environment you can start by
using a fraction of a blade. You pay the hoster a micro-payment
per user account. As your user base grows you automatically
user more resources and pay him for those new accounts. Your
costs scale as your incoem scales and you can make money from
the very first day. In fact, you can make as much money from 5
games that only ever reach 500 accounts as you would from one
game that has 2500 accounts.
As I may have mentioned we already have a very very major
outsourced computing resources supplier (one of the biggest in
the world but I can't name names) who wants to do just this--
encourage a market of hundreds of niche games rather then juat a
few big ones. The ones of those that take off and become huge,
thats gravy for everyone involved.
So in short, which of Gordon's 10 does the Sim Server adress? My
answer is every one thats real!
MUD-Dev mailing list
MUD-Dev at kanga.nu
More information about the MUD-Dev