Windows 8 Store App

It has been quiet for too long now since the previous post. Hence this post to update you all about the status of the engine developments and future plans.

Last October 26th Microsoft released its latest operating system (OS) Windows 8 and Windows 8 RT (for ARM devices) supporting a wide variety of touch enabled devices. This release is a great and important milestone in their history. A couple of days later they also released Windows Phone 8. These three OS’es share the same Windows 8 core, making it easier for developers to create apps: a new ecosystem.

Brand new is the Start screen that you are welcomed with at startup of your device. It displays all important information you would like to see in one place. It does so using a grid of live tiles. Each tile can display the latest news and updates in your social media. Windows Phone 7 users will already be familiar with this concept, they already had the metro interface on their devices. Ofcourse there is much more to tell about Windows 8, I will now focus back on the engine, as thats the topic of this blog.

Next to the release of the Windows 8 based OS’es, the new Visual Studio 2012 has been officially released. It now runs smoothly now on my machines, and I replaced 2010 in favor of this new version for my engine developments. Speaking of which, I started a big refactoring project a month ago to factor out all OpenGL calls in the engine. They are combined in interfaces and moved to a DLL which is loaded at runtime depending on the settings. With OpenGL out of the way, it should be doable to make a similar interface on top of DirectX. During this refactoring I also had to make some decisions about old hardware support and decided to remove the support code for older hardware that did not yet support pixel shaders. It makes the code better readable and maintainable. I also have to look at the resources. Currently actors and the world load their images, sounds, etc. in both the client and server processes. For the server process this is a waste of memory. To solve this problem these objects have to be split up into separate modules, e.g. physics and graphics modules. The graphics module will only be added by the client process.

All in all, there still lies a lot of work ahead which I do not expect to be finished soon. I will keep you updated using more blogs and twitter. Thanks for reading!

Windows 8 interface

Last month you probably haven’t seen much repository or any other activity related to JEngine SSE. That’s correct, I haven’t been doing too much work at the moment directly related to the engine. Lately I have been playing games with friends like League of Legends and Guildwars 2, more that programming.

You probably already read about somewhere on the internet about the new Windows 8 release next October 26. For developers and other enthusiast they already released some previews last year of the new system. I was one of them and currently run the release preview on my machine. Currently you have the nay-sayers which totally hate the new windows interface (former metro) they introduced for touch devices and pc. Actually I really like the new interface and find is easy and intuitive to use. Unfortunately my IIyama touch monitor does not support all the swipes necessary for fluent touch navigation. Luckily they keyboard and mouse navigation also simple.

I also did do some programming, though not yet for the engine.I installed the newest visual studio 2012 to make an app myself. It is possible to write apps in several languages including html with javascript, c# and c++. As gaming had my interest due to the engine, you have to use c++ and DirectX. They even extende the c++ language and named it c++/cx. New features include partial and ref classes. Partial classes are class separations over multiple cpp files. This is mainly used for XAML based application, where the XAML compiler generates one part of the class and the user can implement the other part in another file. In this file are mainly the logic and event handlers. There are also ref classes which stands for reference counted classes. So, instead of putting an instance in a smart pointer yourself, the compiler encapsulates it. Another cool feature are the tasks for concurrent programming. You can now easily chain actions which then either run on the background. They are used for faster response times in your apps.

Learning DirectX 11 helps me porting the engine to the wide range of Windows 8 devices, including PC, tablet and phone. It has been some taken me some time to figure out how to use the DirectX interface, which is totally different from OpenGL that I have been using so far. I pulled out an older book I have lying on a shelve about making a RTS in DirectX 9, so I have to convert that on the fly to the latest version. I now got myself a terrain rendered using an in vertex- and pixel shaders implemented texture splatting algorithm!

It will probably still take a long while before I make the big step to porting the engine to DirectX. I hope in the meantime that I can also work a bit on the UI and world editor. But we’ll see that in the next post.

Unicode support

In the past month a lot happened. A new beautiful daughter was born, limiting my available development time for a good reason. Next to that I new game got my attention: League of Legends made by Riot. This great tower defense game, is a multi player game with two teams of five persons who control their champion to destroy the base camp of the other team. Tactics and skype fun with friends make this game very addictive!

With a lot of changes to the code base, toughing almost all code, I managed today to finish up the conversion of the standard library strings to the custom string written on top of the UnicodeString of ICU. With this change the engine comes closer to a full Unicode support, meaning that any language can be used for messages, user interfaces, etc..

Next to Unicode support the original idea behind this work was to make a string cache for the scripting engine. Using this cache it becomes possible to optimize the virtual machine even more as the Variants that are currently used than can be swapped by a simple union. This saves us a lot of memory (de-)allocations and required operations.

When the string cache is operational, a new version 0.4.8 will be released so everyone can toy around with the much improved scripting language while I continue working on other things. One of these bigger things that really needs attention is a level editor. That means that many UI widgets have to be implemented to make this possible. But without a proper level editor, the use of this engine (even though the editor from an earlier version can be used).

Blogging continues

You might have noticed and heard that the standard blogging service hosted by SourceForge.net has been discontinued. Luckily they have been so kind to allow to install apps in the project web space and therefor I can continue blogging about the progress of the game engine in this brand new WordPress installation.

Let me also use this blog to give you an update about the progress itself. Some time ago I made a new branch in which I am trying to improve the garbage collection of objects that are no longer referenced by another object. Until now I used reference counting objects; objects that deletes itselfs after nobody referenced it anymore. Although the concept is nice, it has some serious drawbacks among which is the terrible cyclic dependency islands. Say you have object A and B which both reference eachother. In that case if nobody references A or B, the objects will still not be deleted as their reference count remains one. Besides this example there are many more reasons to abandon this scheme and move on to a more professional garbage collection algorithm.

One of the better known is the Mark and Sweep algorithm which I have choosen to implement for my garbage collector. The basic idea is that there are some root objects, in the case of this engine this could be the client or server and all static objects and object that are currently on the stack. While traversing these root object, each object on the way is marked. After the marking phase, the garbage collector iterates over the collection of all objects, removing those objects  that where not marked. I leave it up to you do reason why this algorithm also would free up the object islands mentioned earlier.

During the implementation of this algorithm many changes have been made to the engine and virtual machine as well that greatly improves the performance. One of the biggest changes is actually the removal of the Game script class. The engine now creates the client and server process from native code (more on this later). In this special branch the frame rate has already increased from around 50 to 150 per second!

Next step to maturity

As promised in the previous post, a new version dedicated to Linux support has been released as version 0.4.7 yesterday! From this release on Linux is fully supported as gaming platform. It is even possible to make multi-player games that are hosted on a Linux machine and connected to by Windows based clients. Of course Linux support is not all that got in. Also a long list of bug fixes and improvements have been implemented, including changes to the Actor animation loading.

Next steps

The plan is to create some more documentation and tutorials about the engine on the wiki. With this information it will be easier for you to get started with your own game.

Development wise there are also some plans for the next release,  including the already mentioned network improvement feature, a property system for the script language and some more UI components to prepare for the new world editor. As stated before I am open for other feature requests.

Demo: whaaaagh!

In case Grudge sees you, he will go berserk and start hitting you on the spot! When a player has been sighted by the enemy model and is close enough for hitting, he will enter attack mode. Currently while in attack mode, only the attack animation is rendered repeatedly.

Implementation wise the patrol state queries the actor class for its attack state, so that each type of enemy has it’s own behavior. The same will probably be implemented for the patrol state, so that for example a path can be followed.

for example

Memory leaks

Some days ago I was attempting to jump around a bit in my sample game thats comes with the engine. Unfortunately during that jumping around another process decided to hang (hopefully not caused by my game ;-) ), so I opened the task manager. While looking for the hanging process I noticed some interesting increase of memory usage by the game executable. In my head it went like “Ow nooess, a memory leak!”. Luckily I already had some  suspicions on some parts of the code but without tools this could become a tough job. After spending some time on the Internet I found a nice library “Visual Leak Detector” should be able to aid finding the leaks. It’s an incredible package I must say! It reports loads of memory leaks, including a memory dump and complete interact-able call-stack. The only drawback is that it does not handle shared pointers well when chucks of memory are deleted in a later stage. Still with this package at my disposal I already found and fixed many memory leaks and the counter is still increasing.

As discussed in the previous blog the engine is finally capable of running on a Linux platform again. As this seems such a big improvement to the engine, I have decided to prepare for a new version sooner without waiting for other new developments. This way more people can test and use the game framework to make their own game. So, stay tuned for a new release in the near future!

Sample game update

As a new part of this blog, I would like to regularly update you on the progress of the sample game that comes with the engine. In this first part I will briefly look at the enemy AI state machine.  This state machine puts the characters in different modes depending on the situation (e.g. player visible and within range). Currently there is only one enemy in the game called Grudge. This big guy currently is equipped with a club and should crack the head of the player. This character now only has a patrol state in its state machine, which lets him patrol a small part of the map.

With the new line of sight implementation that was added recently to the engine,  Grudge is now capable of detecting the player! The PatrolState class is now almost done which means that I can start with the CombatState class soon. This class will be responsible for hitting, shooting or throwing exploding barrels at the player. Note that this all is scripted behavior and only needed some C++ adjustments to expose the required functionality like line of sight detection, etc.

Feature request

The developments of version 0.4.7 have already started, including fixes of  bugs already found in 0.4.6. One of the important features for this next version is the ability to successfully compile and run the engine on a Linux machine. It has been a while since I had Linux installed on a machine, therefore the latest releases were not tested on Linux. Time has come to make it a truly cross-platform engine again. I still have to get used to Linux, so it might take a little longer for me then for people who work on Linux daily.

Next to this also an improvement to the networking code is on the list. Messaging in script has already been implemented and improved in one of the later change lists. The idea for the engine part is to implement a new algorithm, from the Game Programming Gems book series, that improves the reliability of the network connection and bandwidth usage.

In the meantime I am trying to migrate the development methodologies towards agile/scrum due to personal and professional interests. I do this to get more familiar with the methodology and to improve the quality of the engine. This started some time ago when I introduced unit testing. Not to long I go I started using the tool TestLink for the bookkeeping of the requirements and test cases. Now I am trying to introduce the ‘modified’ version of product owner: people who supply features that users want. For this I started a topic on the forums where I hope you and other interested people can enter their needed features. Those features that fit the vision of the engine will then be added to the road-map (product backlog).

Version 0.4.6 released

I hope you have already read this elsewhere, but for those that have not I am proud to announce that the latest version 0.4.6 has been released last week. With this version comes the newest addition to the engine: the brand new scripting language, which I still haven’t decided upon what it’s name is going to be, and some big re-factorings have taken place affecting almost all parts of the engine. The end result is a more stable and better extend-able/maintainable engine.

The script language is currently based on the Java and C# language. In case you are familiar with either one of these getting familiar with this language should be a no-brainer. For those who are not it might take a while longer to get a grasp of it. It is a fully object oriented language using both classes and interfaces. In later versions more functionalities and language constructs will be added so that it becomes easier to model and script your games. Unfortunately often with good things also come some bad things. In this case the bad thing is that the language is completely incompatible with the Lua scripts. Those of you who already have written Lua code will have to port it to the new language.

Developments for the next version 0.4.7 has already started and some improvements and bug fixes have already been implemented. As mentioned in the previous blog, the idea is to release the next version sooner with lesser additional functionalities so that you have more time to play with it and suggest the next features I should be working on. So JEngine SSE can use you input as well to improve it. Let me know what you think of the engine and what features you would like to see in it. You can do so on the forum (accessible from the support page). Bugs you find are also welcome! Please fill in a bug report with as much information as possible so that I am able to reproduce the problem. This helps to get your problems solved earlier and having a more stable engine for everyone!

Big re-factorings have taken place all over the place to improve the stability of the overall engine.

Refactoring

The last few weeks I have been working on refactoring large parts of the engine. Many things have been changed and resulted in the biggest change list of JEngine SSE (not taking integrations and initial branch creation into account). Some of the chances include:

  • The scene graph has been removed completely including the SceneObject. The graph will be replaced eventually by another algorithm so that only objects that are potentially visible are updated and drawn. A good candidate is the quad-tree.
  • No more Object and Creature class; these have been replaced by the Entity and Actor classes. This makes it easier to distinguish between the C++ Object and the scripted Object class.
  • Entity loading has been moved to separate classes. The new ContentManager has been introduced to load all entities. Later other types of entities should be added there, like the particle systems, etc.

With these changes the largest part of the refactoring is finished. I am now wrapping things up in order to create an intermediate version. It will be the first release using the new scripting language. I am planning to release versions whenever a couple of new functionalities have been implemented so people can be play-tested the engine earlier. With these frequent versions I hope to get more bug-reports and functionality requests so the engine becomes even better. So, if you have questions, bug-reports or requests don’t hesitate to file them in the bug tracker or the forum.

After the coming version is released I will continue improving the code base and script language. Two of the things currently high on the list is better (real) generic types support and improved networking. More on these changes later!

Unit Testing

This is going to be a post about most topics discussed so far in this blog. Let me first rephrase what I was working on before and then move on to the new features.

In the last post I mentioned the Windows Phone (WP) and porting the engine to it. In the past months I have spend considerable time in porting the classic Wolfenstein 3D game to the WP. During this period I learned a lot about C# and XNA (the DirectX for the phone, xbox, etc.). For this game to work on the phone I had to write custom content importers to be able to load the world and image (have a 5551 format which isn’t supported correctly in the default texture importer) files. So, I know now how to do that for the world files of JEngine SSE as well.

Besides porting Wolfenstein 3D I picked up other developments as well again. Some weird logical part of my brain remembered me that actually having a UI is way more important in this phase of the engine. Currently there is no way to edit levels or design any user interfaces as was possible before the migration to the new script language. Thus I picked up the development of the UI and implemented the Image widget which can display, yes think hard, an image! It’s a little step in the UI, but a great leap forward for the engine. Many more widgets still have to be implemented. But the current couple of widgets show that the script language is capable of supporting a complex UI. In order to make the UI code as clean as possible I am considering to implement inner classes. More on this in a later blog.

Test Driven Development

Yesterday I also introduced unit testing in the engine. I wanted it in already for a long time and thought I had made a start with it long ago. After some research I turned out to be wrong. So, from revision 188 and up you will have unit tests available for testing the new and old features. I planned to write unit tests for the important parts of the existing engine. For the new developments I will go and try to use the Test Driven Development (TDD) development process. TDD is split up in a couple of phases:

  • Write the test which should be validated to fail.
  • Write some code to make the test pass (of course the functionality should be working as well).
  • Refactor the code to clean it up. Re-running the test should validate that you did not break any functionalities.

I already foresee some learning curves in TDD. Testing the OpenGL parts seems pretty hard to me as those are mainly visible parts on the screen. But time will tell where this ends up. If you have idea’s or suggestions, please let me know!