[MUD-Dev] TECH: DBM vs BDB speeds (was: AmigaMud DB questions)
bruce at puremagic.com
Wed May 2 01:35:25 New Zealand Standard Time 2001
Jon Lambert wrote:
> The only work I know of that has looked at optimizing massive
> insertions in particular has been TDBM and dxStore. I believe
> similar problems prompted the creation of both. Neither is quite as
> mature as GDBM or BDB (meaning possibly buggy).
Hmmm, I had forgotten about dxStore. :) TDBM looks like it is well
past the point of being inactive into looking dead. A shame.
> I created a special object called $catalog. It's sole function is
> maintaining those global symbols. And by globals I mean names of
> objects. Anyways since its just another object it lives in cache
> with the rest of objects. And the high access frequency tends to
> keep it there. I suppose it might be a useful concept for Cold as
> it would necessarily remove any objname->objnum index records, and
> make the in memory hashtable unecessary. Of course in your
> particular case, that would probably be one massive object. :-)
Mmmmmm. Way too massive. :)
I don't mind the objname->objnum index records. And we still have to
have the objnum->(size,offset) index, so it isn't a lot of extra work
to maintain both.
> I guess maybe if the problems of why it's necessary to do a
> decompile and recompile are resolved when one does a driver upgrade
> then it wouldn't be that big of an issue. I assume that data
> changes in some dramatic way between driver updates? It shouldn't.
> Which is to say that I don't think it should at this late stage in
> development. And if it did some sort of automatic upgrading would
> be more useful.
Well, usually, you can get away without that decompile/recompile.
Just that there are some dependencies within the bytecode values on
the driver configuration, the set of ops/functions, and the set of
native methods. In development over the last couple of months, we've
added new ops/functions as well as new native methods. Furthermore,
most of the serialization code was rewritten, including some changes
in the on-disk format to make things more efficient. Those types of
changes necessitate a re-creation of the binary database.
Furthermore, there was -no- reason for it to take as long as it was,
and is usually a quick and painless process. It just wasn't scaling
well. I'm confident that with some work, we could get the current 2
hour time for a 1.5G DB down to under an hour, but that's not time
that I have at the moment.
MUD-Dev mailing list
MUD-Dev at kanga.nu
More information about the MUD-Dev