Coherent UI in CryEngine 3 (Redux)

After the article on CryDev.net about Coherent UI, a lot of people read our previous blog post about our experiment with CryEngine3 and suggested that we should use Hendrik Polczynski's Plugin SDK to get low-level access to the CryEngine3 renderer. The problem we previously had was that we couldn't find a proper way of creating a texture and "inserting" it into the engine, having an ID and whatnot. I talked with Hendrik, he gave me some pointers about the features of his SDK and how it solves our problem. As a result I present you an improved version of Coherent UI in CryEngine3! You'll find the link for the source code at the end of this post.

This is a video of the end result with more of the same as last time. Except this time it's much more fluid.

First I'll start with some notes about the usage of the code:

• We used CryEngine 3.4.0 (3696) Free SDK for our tests. If you want to use 3.4.3, you need to change the version string in Code/Plugin_SDK/inc/IPluginBase.h to "3.4.3". The code in the Coherent UI plugin doesn't have any version specific parts so it should be ready to use with the new SDK, although we haven't tested it.
• The 64-bit build hasn't been tested neither because we updated our developer machines with Windows 8 and the CryEngine executables do not start. We had an issue with the 32-bit executables as well - the login dialog just froze when either the Launcher or Editor were started but that turned out to be caused by the AV - after excluding the executables we could finally start using CryEngine again.
• CryEngine 3 is one of the top engines on the market - as such it squeezes every bit of resources the machine has. On the other hand, Coherent UI is designed as an out of process interface framework that needs some CPU/GPU as well. There's a chance that CryEngine will take most of the system resources and the UI can appear laggy. The best way to alleviate the problem in the free SDK is to check your FPS (use r_DisplayInfo 1 to show dev info if not already shown) and set a maximum framerate value (using sys_MaxFPS value) to something a little lower than what you're getting from the FPS counter.
• When in-game, you can hide the default CryEngine interface using g_ShowHUD 0
• The Editor isn't fully supported; while in the Launcher we create and destroy Coherent UI Views in OnLoadingStart/OnUnloadComplete callbacks, these are unsuitable for the sandbox because OnLoadingStart is called even if you aren't editing a level and OnUnloadComplete isn't called at all. To work around this limitation, we're creating the views in the OnActionEvent callback when the user enters game mode. This has some drawbacks since we're registering the player event listener in the same callback, but in another listener that is called before the one in the Coherent UI DLL. This means that player notifications will not work in the editor (e.g. the health change notification).

Plugin installation

The Coherent UI plugin requires the Plugin SDK and D3D Plugin to function properly (and Coherent UI itself, of course). They are included in the sample code (with no modifications, except for the version string so it matches the 3.4.0 SDK) at revisions d2feebc5d09ca367bc63d42589de6c2cf606b0d8 and c654e77255344a1ebe72c2365144c3d0908fd903, respectively.

First, setup the include and library directories for Coherent UI. To do so, get the package from our Download page and extract the contents of the lib directory to your CryEngineRoot/Bin32 dir (you should have CoherentUI.dll in the Bin32 directory now). Then create a new folder in the CryEngineRoot/Code folder named CoherentUI and extract the contents of the include directory of the package there. You should end up with the following directory tree: CryEngineRoot/Code/CoherentUI/Coherent. Now you're ready to use the Coherent UI framework. Next is the plugin setup.

If you download the whole repository with the Coherent UI plugin, you just need to copy the files in the root directory and build the provided solution.

If you download the plugin only there are a few more steps. Follow these instructions for installation of the Plugin SDK. After you're done, you need to set up the Coherent UI plugin. To do so, add the following to the CGame class:

Add the following action event handler as well (it's used for registering a player event listener provided by the Coherent UI plugin that does the binding between C++ and JavaScript methods):

You'll also need to setup include and library paths for the CryGame project.

 Plugin limitations

When using a DirectX9 renderer, Coherent UI Views can only use shared memory (and not shared textures), since CryEngine uses a IDirect3DDevice9, while shared textures require a IDirect3DDevice9Ex. Creating a shared texture without an Ex device will just fail.

When using a DirectX11 renderer, HUD Views cannot be drawn, because the D3D plugin doesn't hook to the IDXGISwapChain::Present method. This is because there is no specific swap chain associated with the device which means you cannot use the device to get the swap chain. I thought of hooking to IDXGIFactory::CreateSwapChain and using the device and HWND that I have to determine the instance of the swap chain so I can hook to the present method, but that's left for the future. Shared textures can be used, though. Generally, it's recommended to use the DX9 renderer so you can experience all the features.

Plugin architecture

The Coherent UI plugin follows the Plugin SDK architecture and exposes just a tiny bit of functions that are needed by CryGame. These functions are:

• Initialize
• Shutdown
• GetPlayerEventListener

The first two are self-explanatory; GetPlayerEventListener is used only because we can't register a player listener outside CryGame since the CPlayer class is not exported.

Everything else related to the UI is encapsulated in the Coherent UI plugin DLL.

Class list

• CoherentGeometry - Loads geometry in .obj format and intersects rays with the loaded geometry, returning distance to the hit point and barycentric coordinates. This class is used because I couldn't find a way to get the texture coordinates out of the CryEngine provided methods (the physics query did not return texture coordinates)
• CoherentInputEventListener - IInputEventListener implementation that converts CryEngine input events to Coherent input events.
• CoherentPlayerEventListener - Used for demonstrating binding of engine events to UI events.
• CoherentSystemEventListener - UI system events listener, required by Coherent UI.
• CoherentUISystem - Deals with the communication with the UI system, managing Coherent UI views, creation of rendering resources and all UI logic.
• CoherentViewListener - Reports events for Coherent UI Views and performs low-level drawing; it can also own a CoherentGeometry object that is considered the collision geometry for the View.
• CPluginCoherentUI - Implements the plugin interface.
• FullscreenTriangleDrawer - Does exactly what the name implies. Used for drawing the HUD.

Code walkthrough

The classes that do most of the work are CoherentUISystem and CoherentViewListener, so the focus will be on them. We'll just skim through the CoherentInputEventListener briefly, so you know what's happening when you press buttons. The input listener does nothing fancy - it just converts CryEngine input events to Coherent UI input events. Examine the code if you're interested how to do that.

It maps the following keys to some useful functions:

• Numpad0 stops player input and allows forwarding to Coherent UI Views
• Numpad2 toggles drawing the Coherent UI HUD
• Numpad3 hides and shows the mouse cursor while playing.

The input is forwarded to the Coherent UI View that you last moused over (it helps showing the mouse so you know where the cursor is).

To summarize, when you want to send input to a Coherent UI View, press Numpad0 to stop player input and enable forwarding to Coherent UI, press Numpad3 to show the cursor, and mouse over the view that you want to send input to.

On to the CoherentViewListener. This class has a pointer to a DirectX texture that it draws to when it receives the OnDraw notification. It's important that we draw in the Render Thread or otherwise two threads could simultaneously access the DirectX device and cause a corruption. The OnDraw notification is called in the same call stack that Coherent::UI::UISystem::FetchSurfaces is - that means FetchSurfaces must be called from the render thread. We do that by using the D3D plugin which hooks to functions that are always called in the needed context. The drawing itself consists of copying one surface onto another, nothing fancy there.

The listener is also notified when Coherent UI needs to create or destroy textures. These notifications are in the Coherent::UI::UISystem::Update call stack, which is in the main thread. This means we can't use the device immediately and we need to ask the CoherentUISystem object to do the job in the rendering thread - that's no problem because Coherenet UI supports asynchronous surface creation. The last thing the listener does is providing a convenient method for ray casting through the listener's collision geometry, returning the texture coordinates of the hit point (if any).

Last, the CoherentUISystem class - it's pretty much the alpha and omega of the plugin. It receives DirectX notifications from the D3D plugin, executes tasks scheduled for the render thread (such as creating and destroying textures), manages the Coherent UI Views and encapsulates all the UI logic. It can also make coffee if you ask nicely :).

I tried to keep the UI logic as realistic as possible and this is what I came up with: the UI system itself is initialized in the plugin when the game starts and is active throughout the its lifetime. When a new level starts loading, view listeners and their corresponding Views are created for the HUD and objects. The object names are hard coded for the purposes of the example and are relevant for the level provided. Also during loading the input and player listeners are registered. Everything that is created when loading is destroyed when unloading a level.

The management of Coherent UI Views mostly consists of creating and destroying textures, requested by the views and providing help with the drawing of HUD Views. The CoherentUISystem maintains a collection of requests for texture creation that need to be executed on the render thread. These requests are processed in the CCoherentUISystem::OnPostBeginScene every frame.

Views that are placed on objects are updated in the CCoherentViewListener::OnDraw method - a texture is rendered to and later used by CryEngine, no further work required. When displaying a HUD, there's a bit more to it - it's a separate texture that needs to be drawn after all post-processing and is not associated with any object. For this purpose we're doing that in the CCoherentUISystem::OnPrePresent callback - we just render a full screen triangle with the HUD texture mapped onto it. The textures from Coherent UI come with premultiplied alpha so before drawing the triangle we need to set the blending modes to One/Inverse Source Alpha for the source/destination targets respectively. Note that as we mentioned this callback is not available in the D3D plugin for the DX11 renderer, so no HUD there. Creating the textures used by CryEngine is done through the D3D plugin. Examine the CCoherentUISystem::SetTexturesForListeners method to see exactly how.

Check out the full source code for the plugin at github. Bear in mind that the limited version supports only one Coherent UI View, so you'll have to choose which one you want. There's a friendly error message during compilation that will direct you to the code that you need to change.

If you want to see all of this in your CryEngine, go to our Download page to start experimenting!

6 reasons to be at GDC 2013

Is it worth being at GDC 2013? 2 months left till the biggest game developers conference in San Francisco and many game devs are still asking themselves this question. Even the ones who have already bought passes.

The game developers who haven't been to the event yet may not know about all the benefits of being there. This is why we wanted to share with you our personal experience of being there before and point out the top 6 reasons to be at GDC 2013 (according to us). So if you find yourself benefiting from any of them then you should definitely go and meet us there together with more than 22 500 geeks passionate about making games.

1. Networking

GDC is probably the best place to meet people from the game industry. If you're new in the business this is your chance to get to know people. It's not rare for game development companies to find publishers or investors or for students to find a way to break into the industry. Veterans find it also very useful to network with other attendants. You never know which contact will be of use for you so this is why you should try to meet as many people as possible. If you're lucky you might even get invited to private parties and special events.

2. Getting coverage

Many game developers manage to meet bloggers or journalists at GDC and find a way to intrigue them about their projects. As a result they not only get coverage but manage to build long-lasting relationships with key people from the Media.

3. Learning

GDC is also your key opportunity to learn about the newest technology from the biggest players in the industry. This includes AAA developers like Activision Blizzard and BioWare talking about their latest and most successful projects. So far the organizers has revealed talks about Assassin's Creed III, SSX, Dead Space 3, Diablo III , Hitman: Absolution, Mass Effect 3, The Walking Dead and many more.

There is also a big focus on indie game development. If you're into free to play gaming, you might find it useful to listen to what Zynga is going to say about it. One of the talks we are looking forward to the most is the one of Valve about Virtual Reality and its future implementation into gaming.

4. Inspiration

Meeting, talking and learning from so many people from the game industry will definitely inspire and motivate you to build better games and software in general. You will get so many new ideas you could implement instantly into your project that it will be probably best to write them down on spot. Otherwise you might forget and lose great opportunities.

5. Rethinking your business model

GDC will help you to rethink your business model as well. You will learn more about current and new platforms and how to use them for your game. You will also have the chance to learn about new ways to monetize your game or software and better ways to promote it among your target audience.

6. Hiring talent

The people that are willing to go to GDC are really serious about their career. So if you're looking for talent this might be the best place to find it. Keep in mind that if you're still not a well-known company it will be probably harder for you to attract potential employees, so you will  really need to show them the benefits of working with you.

If you believe you might benefit from any of these points, it might be a good idea to get a pass for the conference and to use this opportunity. If you decide to go or already have, we invite you to visit us there at booth #241 (near the IGF booth). You might as well request a meeting in advance since we don't know yet how busy our schedule will be. In all cases, we'd love to meet you there and learn more about your projects.

Converting a gyp library to premake

Here at Coherent Labs, we have a large and cross-platform project. The build system is crucial for a project like ours - it pretty much defines the programmer workflow and iteration time. How we chose our build system deserves a blog post of its own. But at the end - we chose premake - it is very high level, generates almost perfect Visual Studio project files and you have a complete programming language for describing how to build your project.

The idea for this post was to show how to convert a classic C library using autotools (./configure && make) to a premake-based project. But half of the post would be a repetition of Converting a C library to gyp, because the most complex part is generating all the config files for the current platform. So I decided to take node-lame, a node-gyp based project and convert it to premake.

Premake projects are written in plain Lua, typically in a file called premake4.lua. The "Hello World" project file looks like:

Next step is to include the right config.h file. To make our lives simpler we are going to use "token replacement", a  relatively new feature, available in the premake-dev branch. Token replacement allows

the declarative syntax of premake to fallback to imperative functions. So we write a global function and place  a call to it inside %{} in the include directory specification. This function will be called for each concrete configuration of the project and the %{} is replaced with the result.

libmpg123 has several functions written in assembly for x64 platforms, but they cannot be used on Windows. Here again we use Lua to avoid duplicating the lists of defines and files:

The next step is to define the dependencies between the projects. We include the projects in the solution and  for each project declare that it links with its dependencies. This will ensure that all projects will be built in the correct order and the correct outputs will be used for linking. Unfortunately the include paths dependencies are not solved so easily. bindings depends on mpg123 and includes mpg123.h, so it needs deps/mpg123/src/libmpg123 as include path. But mpg123.h needs config.h, so bindings needs mpg123's config directory as include path, too. So each project has to know not only the directories of its dependencies, but their include directories also.

This is a point where gyp shines - 'direct_dependent_settings' allows you to add include and library paths to dependent projects. Premake has a planned feature - "usage projects" that will provide similar functionality.

As I said node-lame uses node-gyp, which takes care of installing the correct headers and libraries of node.js for your platform. With enough scripting in Lua we can achieve the same effect, but, for simplicity, we are going to add an option to premake4, specifying the location of the node.js source tree with built libraries. 

At last lets see where the high abstraction of premake makes it so good:

This section sets linking against the static run-time and enables debug symbols on Windows and the architecture for Xcode. 

And the same for premake (but on all platforms)

You may ask why bother converting third party libraries to your own build system. Well - to be sure they are build with the correct (same) settings as your own project and to avoid building all variants and committing them under source control - here we already have 18 combinations  of configurations and platforms - and no one wants to maintain that many binaries.

Dump References VS add-in

Motivation

In our current products we have to deal with very big build projects, some of are divided in more than 200 libraries. This poses huge challenges on the build process and their overall management.

On Windows we build everything with Visual Studio and msbuild, however I have often stumbled upon two very annoying quirks in them:

1. If you build a Solution through msbuild and project A in it depends on another project B that is NOT in that solution, project B will be compiled and linked in it's default configuration. Usually this is the Debug configuration. This might lead to some Debug libraries being linked in a Release project or some other config inconsistency.
2. If project A in a solution references project B, also in the solution, then Visual Studio fails any compilation attempt with the cryptic "The project file ' ' has been renamed or is no longer in the solution." giving no actual info on what's wrong. There is a bug on MSDN about this. It's unclear if and when it'll be fixed and most solutions suggested by users involve removing all projects, adding them one-by-one and testing which one fails. This is unfeasible in large solutions.

The second problem was for us very serious. We use premake to generate our projects on different platforms and often change dependencies, upgrade third party libraries etc. GUIDs however are only used in VS projects and they keep getting out-of sync often.

We knew we needed an automatic way to check those issues. One solution would have been a program that just walks and parses all VS projects and issues the warnings we are interested in. I opted however to create a VS add-in because the data we are interested in is already readily parsed and available in the IDE itself.

Dump References Add-In

Dump References is an add-in for Visual Studio that prints the references of all projects loaded in the solution, checks their GUIDs and prints eventual issues. It is intended for C/C++ solutions.

The add-in is fairly simple - it walks all projects in the solution and their references and prints:
 - all projects currently in the solution
 - all references that each project has
 - all unique references in the solution
 - possible issues: referenced projects missing from the solution; projects available in the solution but referenced by other projects with another GUID.

To dump the diagnostic just right-click on your solution in the Solution Explorer and click "Dump References".

The results will be available in the Output window in the Dump References pane.

The add-in's source is available on GitHub and licensed under the MIT license, so feel free to use it.