The Dolmexica Engine [Moddable 2D Fighter]

23 Apr 2011 14:58
Hey there friends, enemies and frenemies!

Remember this April's Fool thread, which was voted unanimously to be the lamest Senile Team thread of 2011?

For those who haven't read it, it features the introduction of a moddable 2D fighter called the Dolmexica Engine, similar in style to Elecbyte's M.U.G.E.N. What you did not know was that the Engine is real. Not only that, but also pretty advanced already, if I may add.

Just in case you are wondering why I am announcing this here, on this rather small forum, it is not just to show it to Senile Team, but also because a lot of Modding and/or Dreamcast enthusiasts visit this place on a regular basis (especially when Lavalit is down). So now that all of the engine's basic functions are working, but at the same time are still relatively easy to change, I have come to the conclusion the time has come to ask for some feedback, ideas and suggestions.

Before I share with you the secrets of modding for the Dolmexica engine, I would like to stress that it is neither Senile Team's Beats of Rage nor Mugen. Since it was written from scratch (with the help of KallistiOS) by me, I can safely assure you that neither of these moddable milestones were an influence for me, instead I have used a different approach to the aspect of moddability.

Which brings us to the main aspect of the engine: How on earth does creating a new character work?
----------------------------------------------------------------------------------------------------------

Okay, so you've gone through the process of ripping/drawing the sprites.
The Dolmexica Engine uses two semi-independent files for new characters: One .CHR file, which stores all the media related to the characters and one .MVT, which is best compared to a manual which tells the game what the character is supposed to do, more on that later.

First you have to create a .CHR file. Since the engine is optimized for use with the Sega Dreamcast, the individual frames are stored in the RGBA 4444 KMG format, compressed with the QuickLZ compression archive, resulting in a file format called .PKG (Portable KallistiOS Graphic). Currently I employ 128x128 PNG textures (which are then converted and compressed before using them with the game). You might wonder why someone would want a texture with a width of 128 pixels, the thought behind that one is that the all close combat moves take place in this box, so even long-range attacks like Dhalsim's stretching attacks fit in there. Just in case you wonder, those 128 pixels may look a bit small at first, but it's enough to contain even the detailed Street Fighter III: 3rd Strike characters. Also keep in mind that they are stretched to 256x256 pixels on the screen, which is a lot comparing it to the overall screen size of 640x480. One thing that I am still unsure about is wether the engine should use RGBA4444, RGBA5551 or both, with RGBA4444 being my current favorite.

After you have successfully converted your PNGs to PKG, it is time to work on the hit detection. The Dolmexica Engine uses a fifth image layer for that one, colloquially dubbed the Crash Layer and is stored in a .CRA file, which is nothing more than a 128x128 array of ones, zeros and other funky numbers, afterwards compressed with QuickLZ. Since writing 16384 numbers is nothing but a pain, I have also written a tool which converts a normal (24bpp) .BMP image to the .CRA format. White (0) is nothing, Black (1) is vulnerable, Red (3) is the area which HURTS, Blue (2) is blocking. In case you haven't noticed, this hit detection is pixel perfect.

Regarding Sound effects and other small sounds, I am currently evaluating the possibility of using some ADPCM format. Or just the normal .WAV one, the most important thing aspect is that the sound effects will be streamed.

Those three filetypes will be merged into the aforementioned .CHR archive. It probably sounds very confusing right now, but believe me, thanks to the automatic tools I have written, a new attack or animation can be added in less than five minutes.

The .MVT file is a tad more complex. As mentioned before, it serves as a manual to the game, which tells the game which moves are available. First we have to differ between two different types of manuals: The one used by the CPU and the one used when the character is controlled by a human player.
Should the character be controlled by the evil CPU, the .MVT is ordered by the type of Attack, whereas the Humanoid .MVT is ordered by the input buttons which trigger the animation.
Both types are being "compiled" from a readable .TXT file. For normal attacks, values which have to be declared are the overall amount of frames, the speed of the animation, the strength (attacks only), the Enemy Acceleration, how much the enemy gets knocked back (attacks only), sound effects and the AnimationType (attack, throw, blocking...).
Then there are also special attacks like throws, combos and so on, which have values declared for each individual frame, including some new variables: How much the Enemy's or the own Char's Position change, how much damage each individual frame does and others. This is the stuff I'm working on right now, so expect a lot to change in the special moves department, but this is how I am expecting things to work.

Naturally you can also fiddle with things like the character's jumping height and speed. The physics are something I really put a lot of emphasis on and a lot of work into. My primary goal is to make it feel like Street Fighter III, which in my opinion is the very best 2D fighter on the DC.

Anyway, when you are done with creating the .TXT and converting it to .MVT, you have everything you need to have a new working character. My two testing characters right now are Sagat (Street Fighter II ver.) and Johnny Cage (Mortal Kombat ver.), replacing my previous test char, Stickler (The Best Game Ever).

Since releasing the engine just by itself would be pretty uninteresting, I am currently thinking about using it to create a Street Fighter vs. Mortal Kombat game, because it is the only Street Fighter crossover that has yet to happen and the Dolmexica engine is probably able to fix some of Mortal Kombat's grave gameplay issues (such as the hit detection). In addition to this all-star cast, it will also feature two original characters from The Rad Adventures of Captain DC, a full-fledged 2D fighter which will utilize a modified version of the Dolmexica engine. Don't hold too much of your breath for TRAOCDC though, unlike the Dolmexica engine it might go the Maid Fight way and never become a reality.

Of course modifying the source code will be possible too, since the Dolmexica engine will be released under the GPL. As a matter of fact, I have to release it under the GPL, since that is a requirement for using QuickLZ for free. Fair trade if you ask me.
----------------------------------------------------------------------------------------------------------

Now, what do you think? Anything you would like to see added in the final version? Something I have overlooked? Whatever it is, please let me know.

During the next few weeks, I will also use my rusted PHP/SQL skillz to set up a site with a small forum dedicated to modding for and improving the Dolmexica Engine, so stay tuned.

Oh, and thanks for reading this wall of text.
Like mentioned before, I'd be really glad if you could post some feedback, suggestions or ideas you have.

Quote
23 Apr 2011 16:21
Sorry, but your post is tl;dr.  Something about a GPL'd Mugen-like fighting game engine?

Quote
23 Apr 2011 16:30
I'll try and be shorter now. 

Plombo wrote:
GPL'd fighting game engine?

Yes.

Plombo wrote:
Mugen-like

No. It's moddable, but that is the only similarity.

Quote
23 Apr 2011 16:32
Plombo wrote:
Mugen-like

No. It's moddable, but that is the only similarity.



Okay, good.  Maybe you shouldn't have put it in bold, then.

Quote
Senile Team Big Cheese
Avatar
24 Apr 2011 06:57 / last edited 24 Apr 2011 08:48
I may have a tip for you.

After you have successfully converted your PNGs to PKG, it is time to work on the hit detection. The Dolmexica Engine uses a fifth image layer for that one, colloquially dubbed the Crash Layer and is stored in a .CRA file, which is nothing more than a 128x128 array of ones, zeros and other funky numbers, afterwards compressed with QuickLZ. Since writing 16384 numbers is nothing but a pain, I have also written a tool which converts a normal (24bpp) .BMP image to the .CRA format. White (0) is nothing, Black (1) is vulnerable, Red (3) is the area which HURTS, Blue (2) is blocking. In case you haven't noticed, this hit detection is pixel perfect.

Pixel perfect hit detection is easy enough to implement, but there are good reasons why it is very rarely used. The first and most obvious one is that it drains memory resources. The second is that it takes (wastes) a lot of processing power to check so many pixels. Maybe you should consider reducing the resolution of your arrays from 128x128 to 32x32. That will reduce the load to about 6.25%, and it is unlikely that players will ever notice the difference between "pixel perfect" and "nearly pixel perfect".

Also, if you store the collission maps as 3 arrays of 32 x 32 bits, you can save an additional 62.5% of storage space compared to arrays of bytes. Moreover, you will be able to use bitwise ANDs to check for collission, potentially speeding up your collission checking code by another 96% (on or about).


(I assumed that you are currently using arrays of 128x128 bytes. If this is not the case, the percentages may differ from reality.)

Quote
newt model
Avatar
24 Apr 2011 08:23
Dolmexica Engine? Does this mean you were the Dolmexica release group in a previous life?

Quote
24 Apr 2011 19:46
Roel wrote:
I may have a tip for you.

Pixel perfect hit detection is easy enough to implement, but there are good reasons why it is very rarely used. The first and most obvious one is that it drains memory resources. The second is that it takes (wastes) a lot of processing power to check so many pixels. Maybe you should consider reducing the resolution of your arrays from 128x128 to 32x32. That will reduce the load to about 6.25%, and it is unlikely that players will ever notice the difference between "pixel perfect" and "nearly pixel perfect".

Back when I started working on the engine, there were three things my fighter was supposed to have, no matter what:
  • A hit detection with an inaccuracy no higher than 2 pixels.
  • No loading before or after the Continue/Try Again Screen.
  • The ability to handle even Street Fighter III characters.

Those were the things which I found Street Fighter Alpha 3 to be lacking.

According to #1, the lowest Crash Layer resolution I would consider using right now is 64x64. According to my personal tests and experiences, anything more than those 2 pixels of inaccuracy is noticable and annoying. Memory resources aren't much of a problem for the engine, since the compressed .CRA files are stored in RAM and decompressed on the fly, so most of them are only about 1000 bytes or less anyway. Not to name any final numbers, but as of right now, the engine can handle ~1200 overall frames calculating with about 8MB of free RAM usable for storing them. It's the processing power which is going to pose a real threat. Back when the Crash Layer was still compressed with gzip, I was unable to maintain a stable framerate of 60fps. Right now the frames themselves are still compressed with gzip, so should there be speed problems again, I would probably replace that one with QuickLZ first (at the cost of memory) and if that fails too, reduce the resolution.

Roel wrote:
Also, if you store the collission maps as 3 arrays of 32 x 32 bits, you can save an additional 62.5% of storage space compared to arrays of bytes. Moreover, you will be able to use bitwise ANDs to check for collission, potentially speeding up your collission checking code by another 96% (on or about).

Bitwise Operators? Man, that's genius! I can't believe I totally forgot about those!
In case there will indeed be only four values used for hit detection, I could also use a single array and simply divide the eight bits of each byte into pairs of two. That way, I can also display the four values and save even 75% of memory. So the Bitwise AND is also faster than the normal comparison?
Thanks for your tips, I guess when it comes to moddable engines (or game programming in general) you are still the number one authority! 

jeroen wrote:
Dolmexica Engine? Does this mean you were the Dolmexica release group in a previous life?

Oh right, almost forgot about them.

Nah, I never met those guys. They didn't invent the name Dolmexica either. It is the name of SEGA's super secret power source from the video game Segagaga. As a matter of fact, this warez group has stained the Dolmexica by making it associated with piracy and illegal actions. Now my engine will not only be an awesome gaming/modding experience, but also cleanse the name of such connotations. Yeah!

Quote
Senile Team Big Cheese
Avatar
25 Apr 2011 06:15
Bitwise Operators? Man, that's genius! I can't believe I totally forgot about those!
In case there will indeed be only four values used for hit detection, I could also use a single array and simply divide the eight bits of each byte into pairs of two. That way, I can also display the four values and save even 75% of memory. So the Bitwise AND is also faster than the normal comparison?
Thanks for your tips, I guess when it comes to moddable engines (or game programming in general) you are still the number one authority! 

Thanks! You are too kind.

The problem with using pairs of two bits is that you won't be able to use the bitwise operators efficiently. Store the "hurt", "block" and "damage" values in separate arrays of 32-bit or 64-bit rows and you will be able to check an entire row with minimal effort:

Code:
//
// Check for collission on 32 or 64 bits all at once (depends on the type of the "hurt" and "damage" arrays).
//
if((hurt[y0] & (damage[y1] >> relative_x)) != 0) {
	//
	// We've got a hit
	//
	return 1;
}
//
// No hit, check the next row
//

Note that this code is only valid if the horizontal position of the "damage" mask is equal to or greater than the horizontal position of the "hurt" mask. That's because you shouldn't shift bits by a negative offset (because the result is undefined and may differ accross compilers). So if relative_x is a negative value, make it a positive one and enter a different loop.

Also note that both arrays must be of unsigned integer types!


Good luck!

Quote
26 Apr 2011 22:07
Roel wrote:
The problem with using pairs of two bits is that you won't be able to use the bitwise operators efficiently. Store the "hurt", "block" and "damage" values in separate arrays of 32-bit or 64-bit rows and you will be able to check an entire row with minimal effort:

Ah, thanks a lot for posting this example, it really helped me to understand what you meant by using bitwise operators efficiently. Sadly, "long long long long long is too long for GCC" (best compiler error ever), and thus I would have to tone down my resolution to the size of a long long, 64 bits. Of course you already knew this, but it's always something new and exciting for me

Anyway, due to the excessive file I/O I have added for the Special moves, I have troubles sustaining a constant framerate of 60fps, so the 64x64 resolution may become a necessity soon enough. From what I can gather, it is really easy to implement, the only thing that worries me a bit is how I should reverse the bit order (one of the characters is mirrored after all) in an effective way. Oh, but don't waste your time coming up with a code example for me, it's better if I figure it out on my own. And let's not forget, it's more fun.

Thanks again for helping me out with the Bitwise operators, I can't stress how much it helped (not only for the hit detection)! Taking your time to help someone with a programming problem, now that is epic fan service

Quote
Senile Team Big Cheese
Avatar
27 Apr 2011 05:37
From what I can gather, it is really easy to implement, the only thing that worries me a bit is how I should reverse the bit order (one of the characters is mirrored after all) in an effective way. Oh, but don't waste your time coming up with a code example for me, it's better if I figure it out on my own. And let's not forget, it's more fun.

Alright, have fun trying and let me know if you need any help.

Quote
20 May 2011 15:50
Roel wrote:
Alright, have fun trying and let me know if you need any help.

Sheesh, finally managed to implement it. It's only fun if it works!

Weeeeeell, I took the easy way by including both the normal and the mirrored long longs in the CRA. Mirroring 64 bits on the fly is just wasting too much speed, so I gladly accept the RAM used there. Even the fastest code I could find come up with took 30 operations (programmer lingo?) which would result in 3840 additional moves. 
And besides, the new crash layer you recommended takes up so little space once compressed, that even with the mirrored layer included, it takes up considerably less space. Yay!

Quote
Senile Team Big Cheese
Avatar
20 May 2011 16:41 / last edited 20 May 2011 19:56
Great!

But... I know of a way that is fast and does not require pre-rendered mirroring.

Your characters are never upside-down, right? But you can mirror your bit maps that way (just read the array of 64-bit values in the opposite direction). So, here's my solution: turn your bit maps by 90 degrees. In other words, store columns rather than rows in the 64-bit values. That way you can easily mirror the maps horizontally.

Quote
26 May 2011 20:00
Sorry for my late reply, but I was busy doing brilliant things. Mostly.

Well, looks like you've done it again.

Quite the genius idea you have presented there, and once again, I wish it had been mine. Seriously: How can you be both a designer and a great programmer at the same time?! I thought that was a yin-yang, "either/or" thing.     

Despite the fact that it seems relatively easy to adapt the code to use columns, I'll have postpone that one. While the measly screenshots I posted seemed to buy me some time, voices demanding a trailer have gotten more numerous lately. That's why my next final big implementation (and pretty much the last real speed-sucker) is the Artificial Intelligence.

Sadly, semi-random movement and attacking while in range won't cut it this time around, but I've been heavily analyzing various Fighter AIs and also have some real nice ideas up my sleeve. What could possibly go wrong? 

Answer: Everything!

Quote
Senile Team Big Cheese
Avatar
27 May 2011 06:23
Admirable! I'll keep an eye out for that trailer!

Quote
newt model
Avatar
27 May 2011 07:21
I didn't even know there was a screenshot, but a trailer would be nice 

Quote
27 May 2011 21:02
Roel wrote:
Admirable! I'll keep an eye out for that trailer!

Thanks for the kind words!

I doubt it is going to look as final as the Age of The Beast preview trailer you made. Just out of curiosity, how did you make it look so good?

The D-Engine trailer will feature characters from two of the DC's technically most advanced fighters, Street Fighter III and JoJo's Bizarre Adventure respectively. Don't dismiss the latter because of its unknown-ness, the JoJo character which to be included in the trailer has 38 frames, and that's only for the normal stance animation! Took me hours to rip this beast! 

Well, enough of my modding endeavours, it's programming time again! Yeah!

jeroen wrote:
I didn't even know there was a screenshot, but a trailer would be nice 

The reason why I haven't posted the screenshots here, is because I'm not that happy with them. Sadly, the emulator I used didn't seem to recognize most of my button input, so all I could take snapshots of were the stance position, crouching and jumping. Now THAT's exciting, isn't it?

dbimage.php?id=306727

Looking at the screenshot again, I should really put some more effort in the GUI before I make a trailer. So much to do, so little time. 

Quote
newt model
Avatar
28 May 2011 10:39
I must say that a simple gui is often best, but this one is too simple

Quote
11 Jun 2011 10:06
jeroen wrote:
I must say that a simple gui is often best, but this one is too simple

Okay, okay, I have tried to improve it:

dolscreen1.png

If only the emulator button input worked properly, I'd be able to show the red, semi-transparent life(less)bar below the yellow brick of doom. The GUI is still nothing flashy, but good to work with (and more importantly: it actually works ). My main problem here is keeping everything moddable... 

Quote
newt model
Avatar
04 Aug 2011 11:53
Any Dolmexica news?

Quote
04 Aug 2011 20:22
jeroen wrote:
Any Dolmexica news?

Glad you asked! (No seriously, I was about to register a second account to prevent the shame of double posting. )

To start with something, there was something I totally forgot to mention in this thread: The planned release date, August 5th, which happens to be tomorrow. Of course that date was thoroughly miscalculated and has proven to be improbable, so to make up for that and the lack of visual stimuli like screenshots or trailers, I decided to release a playable beta that day!

It will pit Ryu against his eternal rival Ken Masters, two characters which not only include all the basics of the genre, but also save me a lot of work, since they have the same moves (and subsequently the same hit detection. Ha!).

On the technical side, things have improved considerably. Most impressively, the new character max resolution is 256x256.
Example:
RYU-SF4k.gif
Yeah, it's just for showing off. Street Fighter IV on the Dreamcast is as likely as Crazy Taxi on the Master System.

How is that possible you ask? Weren't there speed troubles with the 128x128 chars already? While the latter question would have to be answered with "Yes", the decompression speed is twice as fast now (or even more). All I'm gonna say now is that next time I'll actually read the compressor's manual before attempting to use it.

Of course that's not quite enough to process four times the pixels, so the texture size had to be cut using something that was supposed to be included from the very beginning: Paletted textures.

The usual game, either 255 (+1 transparent) or 15 (+1) colors. Since the KMG format doesn't support paletted images, I had to write my own encoder and modify KallistiOS to accept paletted KMGs. To be honest, I'm kinda proud of that. Sure, it's nothing compared to writing your own library, but it still feels pretty good. 

Finally, the music stuff. This part of the engine is incredibly uninteresting, since it only consists of the sound processor's most basic functions. Preferring quality over size, the music format is Redbook CDDA, or Microsoft Wave files. Might seem a little awkward for a moddable engine to use CDDA, but there is a deeper plan behind that, just wait. 

Sound effects are the Yamaha ADPCM format. The only problem is that they mustn't be longer than 0xffff samples. But I'm sure I'll find something to go around that eventually. 

Long post short: Beta tomorrow. Engine stronger. Sound better.

Thanks for reading this whole novel! 

Quote
Copyright © Senile Team. All rights reserved. | About Senile Team | Contact us
Wii and WiiWare are trademarks of Nintendo. © 2012 Nintendo.
Dreamcast™ is a registered trademark of SEGA Corporation.