Performance - Ported Build to C# XNA(WP7/xbox 360)

[IceColdDuke]

Hey,


I posted this on the duke4 forums but didn't get any response...I ported build to XNA(100% C# verifiable code), which means its now possible to port any build game(and expansion) to the 360, WP7 or any platform that supports .NET Mono(PS3, Linux, etc) Since my last post(on duke4) I fixed a lot of bugs(including the wall bug), speed improvements(on the phone it runs 8-15fps rather than 3-5 :P) and added voxel support. More or less the only thing I really have to implement is mirrors and automapping(two things I just relized I over looked). And same as before the engine loads VOC files but not midi(I used the OGG files from the 3dr ftp, if you don't want to do all that to get music to play just comment out the music code).

I removed all the pointers from the code and instead keep track of positions in arrays through variables instead. My problem is...performance. Build has a lot of while loops with pointers which I had to replace with arrays, and the bottleneck is in pretty much all the A functions where I have to pull values from the managed arrays(C# arrays are very very slow). So I was wondering if TerminX or someone from the Eduke team still travels these boards, basically to speed things up I'm going to have to start consolidating loops, merging if statements, etc.

But anyway as a test I choose to work with the shadow warrior codebase because it obviously pushed the Build engine performance wise and if the code works in SW maps its good to go.

SVN: https://xna-shadow-warrior.googlecode.com/svn/trunk

I've been playing around with the com interop hack on the WP7 device(since WP7 is my primary platform), which is why the project is silverlight based but it doesn't take much to port it back to XNA(change WriteableBitmap to Texture2D and some minor music sound and FS code). But all the interop hack code is commented out.

But I was wondering if anyone on here would be down to work with me on this(perferably someone that has a indepth knowledge of C#)?

[Malvineous]

Wow, that's an amazing achievement! That must've been a huge amount of work.

Not being familiar with any of this, I'm curious why you went with C# rather than C++? It's my understanding that .NET can be used with both these languages (and more), so presumably sticking with a C-based language would have meant you could keep the pointer loops more or less unchanged? Or was there a particular reason why it had to be C#?

Do you know exactly where the bottlenecks are? There are code coverage tools you can use to identify where in the code most of the time is spent, so these could hint at what exactly needs to be optimised.

[Kristian Joensen]

"which means its now possible to port any build game(and expansion) to the 360, WP7 or any platform that supports .NET Mono(PS3, Linux, etc)"

Have you tested this under Mono?

[IceColdDuke]

Originally Posted by Kristian Joensen

"which means its now possible to port any build game(and expansion) to the 360, WP7 or any platform that supports .NET Mono(PS3, Linux, etc)"

Have you tested this under Mono?

No I haven't but I have worked with Mono.NET in the past and there wouldn't be a reason why it wouldn't work. The only things that would have to be changed is a couple lines in the filesystem code(related to titleconainer), sound code(again only a few lines), and if you wanted to use something else besides writeablebitmap(Texture2D for the 360 for example), all you have to do is replace writeablebitmap, its that simple. The rest of the code doesn't use really any .NET functions, its all standard C# code.

Not being familiar with any of this, I'm curious why you went with C# rather than C++?

If you want to port a game to the new Windows Phone you HAVE to use XNA C# you don't have the ability to use C++ on the phone. But that reason aside, if you have a XNA "pro?" license, its the only way to get your game on the 360 without a XDK/JTagged xbox, and as I stated before it will run on multiple platforms with minor changes.

Do you know exactly where the bottlenecks are?

Yes I do actually. If you have ever looked at the Build engine source code you should notice a A-C.c/A.ASM file. These functions perform the pixel blitting. Apps for the Windows Phone have to be verififiable code, which means you can't use pointers...period(if you want your app to pass certification anyway). So what I had to do is create seperate int variables to keep track of where data should be gotten and set from in a array.

So for example lets take a look at vlineasm1, this function handles vertical line drawing. I just added the comments now to show based on the .NET profiler which parts of this function is taking a long time.

public static void vlineasm1(int vinc, int paloffs, int cnt, uint vplc, byte[] gbuf2, int basebuf, int pbase)
        {
	        gbuf = gbuf2;
            gbufpos = basebuf;   
            gpalpos = paloffs;

// INCLUSIVE 4.5%
	        for(;cnt>=0;cnt--)
// END
	        {

// INCLUSIVE 21.5%
                Engine._device._screenbuffer.Pixels[pbase] = Engine._device._palette._palettebuffer[Engine.palette.palookup[gpalpos + gbuf[gbufpos + (vplc >> glogy)]]];
// END
                pbase += bpl;
                vplc += (uint)vinc;
	        }
        }

Now if you look at the line that has "Engine._device._screenbuffer.Pixels[pbase]", that line by it self is causing a massive bottle neck, NOT because of the math in between the brackets, but because it has to look up the value at whatever point in the array. This normally wouldn't be a problem, but .NET ensures that your index value falls between valid ranges, so instead of just doing a mov <addr> <val> it adds extra instructions to handle out of range exceptions. This again would NOT normally be a problem but because that line is called so many times, it adds up after awhile, and the same thing pretty much goes for every function in A.C.

You will also notice the for loop is also taking up tics. Basically I'm going to need to start consolidating loops, and trying to find ways to speed up the code but its looking more like I'm going to need to start re-writing some of Ken's algorithms so they would be more efficient for .NET, and this I'm going to need some help with, because I have a pretty good understanding of how things work but not enough to go inside and start redueing everything :P.

Now before someone brings up "Why didn't you port polymost/polymer", I haven't looked at Polymer at all really, but with polymost Ken create geometry on the fly rather than in index/vertex buffers(I'm basing this off of Jonof's code), and with XNA Dynamic anything will cause massive slow downs due to the poor way XNA handles things.

[IceColdDuke]

I fixed a nasty bug in my scansector code that was causing the game to draw sectors it shouldn't have(forgot to cast a variable to a uint :/), and I came up with a way to mostly fix my performance problem. I only do rendering code every other frame, and let the game code run every frame, this seems to fix a huge bottleneck in the silverlight code.

At 320x200 on my WP7 phone I get 18-30 FPS, and in the emulator I get 40+fps. When I try to switch it to 640x480 I only get 8fps on the device, but thats still better than before were I only got 2fps at 640x480 .

Now I just have to get automapping and mirrors in and its more or less complete(as far as the build engine).

[Malvineous]

Hmm, I'm not particularly fond of C# or .NET at the best of times, but it seems silly to force a bounds check on an array at runtime. During development and testing, sure - but by the time your app is ready for distribution you should be able to turn off all those checks as they are no longer required. I don't like the fact that they baby you so much, but then I guess that's the idea behind .NET in the first place.

Does the compiler have any performance optimisations like unrolling loops? It would make the code a little larger but could help with the speed of some of the loops.

I'm amazed you've kept the original 'software' renderer though - I expected that to be too slow! I guess that's the only sure-fire way to boost the speed - to rewrite parts of the engine to use the hardware's 3D capabilities.

[IceColdDuke]

Originally Posted by Malvineous

Hmm, I'm not particularly fond of C# or .NET at the best of times, but it seems silly to force a bounds check on an array at runtime. During development and testing, sure - but by the time your app is ready for distribution you should be able to turn off all those checks as they are no longer required. I don't like the fact that they baby you so much, but then I guess that's the idea behind .NET in the first place.

I agree completely, but its a security feature which I see the point off, but when MS is forcing developers to use C# and XNA for game development on the phone they need to make better alternatives.

I actually really like C#, the problem isn't so much the managed language its the XNA framework :/.

Does the compiler have any performance optimisations like unrolling loops? It would make the code a little larger but could help with the speed of some of the loops.

Nope, the JIT compiler automatically unrolls/inlines stuff for you, and it does really bad job at it. I can't inline functions without actually copying the code into that function.

I'm amazed you've kept the original 'software' renderer though - I expected that to be too slow! I guess that's the only sure-fire way to boost the speed - to rewrite parts of the engine to use the hardware's 3D capabilities

I actually tried that at first, I got build levels tessalated and built into vertex/index buffers but I couldn't get the UV's right :/. I expected it to be slow too(and still is at higher resolutions), but the biggest bottleneck atm is the XNA/Silverlight framework, they do so much hand holding that all the powerful functions are abstracted around a gigantic mess. But I just created a web project and it loads up Shadow Warrior in Internet Explorer and runs at 60fps. I'll upload the project later on tonight.

[IceColdDuke]

I have a question, something similar was done for quake with silverlight and they hosted the shareware version of the game on there site would I be able to do the same thing for sw?

[IceColdDuke]

nukeland.map - 2d view

custom map - 3d view



Hey,
I just the build editor ported over to silverlight and has about 80%functionaility and it is a bit buggy, but you can build build maps online, save them to your hard drive and load them back up in game. The game and the editor run at 640x480 at 60fps, I was hoping to get a few people to try it just to confirm there aren't any performance issues. What I eventually want to do is create a mini online game maker with all the build tools ported over and with a scripting language, you build the game online press a deploy button and it creates XBOX 360, Windows Phone 7, PC and Silverlight projects.

Due to browser restrictions and comforance I had to change a lot of key combinations. The ALT key just won't work in I.E. because of the menu bar, and silverlight really only picks up one key stroke at a time(reliably) so I choose these key combinations(remember press control only once till you see "control(alt) active" message than use these keys:

CTRL KEY ONCE +
Hollow out a sector(ALT-S in the original build) - "O"
Change object hi-tag - "H"
Change object lo-tag - "L"
Change object palette - "P"
Change object Visibility - "V"
Change object Visibility - "S"

Other Keys(no ctrl):
Set tilename - "V"
Set slope left - "keypad num4"
Set slope right - "keypad num4"
Move object Z - "pgup, pgdown"
Create Sprite "S"
insert point on wall "INSERT"
set start xyz pos - "HOME"
delete wall/sprite - "DELETE"
move around - arrow keys
strafe "A/D"
start draw wall(2d) - space
move up and down(3d) - space/c

To load a map from the stuff.dat file press escape and click the "load map from wap" and type any of these without the extension:
ASCBOARD
BOARDS
EVILAL
KENSIG
NSNOAL
NUKELAND
You can modify these maps, save them and load your modified ones in game.

I did notice some bugs with wall drawing i'll address those in the morning but my concern is I want to make sure everyone is getting 60fps constant(I sped up alot of my code and kens code so it more or less will run perfectly as a managed application).

-Justin

Online build editor/game binaries: http://jvsoftware.tk/
SVN: https://xna-shadow-warrior.googlecode.com/svn/trunk

[IceColdDuke]

Editart Screenshot





Hey,

I Just got editart ported over, you can now import PNG images(must have correct palette set before import, transparent parts of PNG import as transparent) and export tiles to PNG. You can also save art and map files to grp files. To start a project click new project(grp file) and save it to your HD, to load a grp from your HD click the load grp file from HD button. You can load Duke Nukem and Shadow Warrior GRP map/art files edit them and export them back out or you can create your own.

To save maps directly into a grp file you need to click save to grp button and than save grp button. I had to put them as two seperate buttons because you can only open savedialog if its called directly from a button event :/.

-Justin

[Darkman 4]

Originally Posted by IceColdDuke

I have a question, something similar was done for quake with silverlight and they hosted the shareware version of the game on there site would I be able to do the same thing for sw?

Check the licence included with the shareware version of SW. If it's like other shareware games, then I think you can do it. Check anyways, though.