Coherent Blog

Thursday, February 21, 2013

Adding a HTML5 minigame in Unity3D with Coherent UI (Tutorial)

Today, we're starting a new series of blog posts with educational purposes. These posts will essentially be short tutorials for some cool new usage of HTML in your game. Let's get right to the point and see what this first tutorial has to show.

Goal

Embed an existing HTML5 minigame that opens a door upon completion.

For this demo we've chosen a minigame called "Memory Box" - you can try it out on this URL. All credit for the game goes to its authors. There's only one change in the code - we added an inline CSS style style="background-color:rgba(0,0,0,0);" to the <body> element and removed the background-color: #000; line from the CSS at the top of the HTML file to make the background transparent.

If you're very eager to see the final result, the video is at the bottom of the article.

Prerequisites

The AngryBots scene that comes with Unity3D
A cube mesh (the Unity3D box has flipped texture coordinates and shows Coherent UI textures upside down). We used this one.
Coherent UI for Unity3D (we'll also assume that you've already imported the package in the AngryBots project)

Scene setup

The first thing to do is to pick a door that will be opened when the user completes the minigame. In the AngryBots scene most of the doors are prefabs that have pre-attached scripts. I picked an internal door near the hologram around the entrance. It uses the TriggerOnPresence.js script and I'll refer to it throughout this article, so keep that in mind if you choose another door (external doors use a different script, for example).

Now that you have picked a door, add the cube-textures mesh somewhere around the door. It will be used as a surface where the minigame will be rendered. You don't need the whole cube, just the front face will suffice, so feel free to delete the unneeded ones. It's helpful to rename the new GameObject to something meaningful such as "CoherentSurface". That's how we'll be referring to the object from now on.

After you're happy with the placement of the object, add a CoherentUIView component to the surface so we can display HTML on it. Configure the URL, the resolution and make sure to tick the checkbox for transparency (you might not need it, depending on the minigame) and "Click to focus". The latter redirects input to the CoherentUIView after you click on the surface and is useful if you can't be bothered to write input forwarding code :).

If you hit play now you'll already have the minigame up and running, but it won't open the door yet. The setup so far should look like this:

CoherentUIView setup for the minigame

Short digression: if you tried playing the minigame, you'll notice that the player is firing when you click the dots. This can be distracting, so if you want to stop the firing you can add this code to TriggerOnMouseOrJoystick.js's Update method:

This will toggle the functianlity of the script (which is to fire bullets only) when the user presses the spacebar. Feel free to add any other logic, such as toggling firing when the CoherentUIView gains or loses focus or anything else that you might like.

Door opening logic

The door's opening logic defined by the TriggerOnPresence.js script does the following: when the user enters the object's collision volume, the OnTriggerEnter function is called, which in turn notifies all the subscribers for the specified signal. When the user exits the volume, the OnTriggerExit function is called and notifies the subscribers for the exit signal.

When inspecting the prefab, you'll find out that this signaling breaks down to sending the "OnPlay" and "OnPlayReverse" messages to the "AudioSource" and "slidingDoor" sub-GameObjects. Now that we know how to open and close the door, we can start making changes.

We won't need the proximity triggers anymore so just disable the TriggerOnPresence component from the sliding door GameObject. We'll also need to be able to identify the door's "AudioSource" and "slidingDoor" sub-objects, so we have to rename the door to something unique, such as "interiorDoorSlidingMinigame". After we have this we can open the door with the following one-liner:

GameObject.Find("interiorDoorSlidingMinigame/slidingDoor").SendMessage("OnPlay");

Great!

Wiring the minigame completion to opening the door

First, we want to have means for sending messages to the game from JavaScript. To do that, add the coherent.js script that comes with the Coherent UI Unity3D package somewhere in the top of the HTML like so (the location of the src property may be different):
"<script type="text/javascript" src="coherent.js"></script>".
This will define the "engine" object, which you can use for communication with the game.

Next, we have to find the code that's executed when you complete a level. That's at the bottom of memorybox.js, in the "if(this.isLastCorrectSquare)" conditional. If the script is totally unreadable to you, you can process it using jsbeautifier.org or some other service. Add the line "engine.call("OpenDoor");" at the end of the if block. This will send the message "OpenDoor" to the corresponding ViewListener. That's all for the HTML/JS side - just 2 lines of code!

Last, we have to handle the message sent by JavaScript in the game. To do that we make a new script, MinigameDoorOpener, and add it to the GameObject with the CoherentUIView (the CoherentSurface object). This helper script will subscribe for the View Listener's ReadyForBindings event and bind a handler for the "OpenDoor" message. The handler will simply open the door as we discussed previously. Its code is as follows (the sample script is written in C#, but you can use anything that Unity3D allows):

Now everything is wired and ready to use! After opening the door you can continue playing the minigame if you like it. You can also enhance the demo by enabling the TriggerOnPresence component of the door so it behaves like a normal door after you unlock it instead of just staying open.

Here's the video of the final result:

That's all for today's tutorial on adding big value to your game in 10 minutes. Stay tuned for more!

Friday, February 1, 2013

CryEngine 3 minimap made with Coherent UI

This is just a small update on our demo in CryEngine 3. We added a minimap using OpenLayers giving us all sorts of cartography goodness.

It's bird! It's a plane! It's a map alright!

Update: an interactive visualization of the map in your browser! (the marker uses the default OpenLayers marker image)

Move marker

Rotate marker

The movement and rotation are done automatically in the demo of course :).
[End update]

Our sample barely touches the surface of the possibilities offered by OpenLayers but it's enough for our purposes. For simplicity's sake, I used a single non-tiled image for the map (which presents artifacts when enlarging the map). The image I used was the one I found in Game\Levels\Singleplayer\Forest\ - Forest.tif (which I converted to a format that doesn't take 16MB). The image isn't fabulous per se (it's blurry and has a lot of discontinuities) and if you want something really cool you can use an orthographic camera in the Editor and take snapshots of different tiles at different zoom levels. When you have the snapshots you can follow this tutorial to make OpenLayers use your local tiles and have a map with real zoom.

Most of the work for the map is done in JavaScript/CSS. You can toggle it by pressing Tab. The C++ part is basically just firing events for the player position and orientation.

The other C++ part (that's not actually related to the cartography) is forwarding the input to the HUD view - I made a new class (CCoherentHUDViewListener) that does the job. You can examine it if you're interested. As a side note, when forwarding input to transparent Coherent UI Views make sure that you set the ViewInfo::SupportsClickThrough to true when creating the view.

The interesting bits of JavaScript for the map are in Bin32/TestPages/hud/js/map.js. There's actually not much to it - we create an OpenLayers map, add a marker representing the player and wire some events that will come from the engine. Here's the whole script (stripped down to the essentials)
Just a few notes about the code - I found the bounds of the area depicted in Forest.jpg by flying around in the Editor and writing down some approximate numbers. You'll notice that I used negative coordinates on the Y axis - that's because it allows easier conversion from the CryEngine coordinate system to the OpenLayers one. If I used positive numbers, I'd have to write a fully fledged linear transform (a number and a minus in this case) instead of just the minus :).

The second note is about the orientation of the player marker - OpenLayers does not provide means for rotating the marker image so we have to use CSS. Fortunately, the marker object gives us all the information about the DOM element that contains the marker so that's no problem.

The HTML code for adding the map is simply <div id="map" class="smallmap">, where the smallmap class defines the size of the div element.

If you want to make the map circular you can clip it using a simple border-radius trick like so:
Here's an example of what this does (may not work on all browsers):

Original

Using border-radius

Or you can use some more advanced HTML/CSS technique that fits your needs.

That's all for today's update, the updated code is on GitHub so feel free to try it out and extend it. We'll be preparing the CryEngine3 showcase for GDC this year, so if you're in the Bay area at the end of March feel free to visit as at booth #241 and check it out live!

Tuesday, January 29, 2013

Coherent UI in CryEngine 3 (Redux)

After the article on CryDev.net about Coherent UI, a lot of people read our previous blog post about our experiment with CryEngine3 and suggested that we should use Hendrik Polczynski's Plugin SDK to get low-level access to the CryEngine3 renderer. The problem we previously had was that we couldn't find a proper way of creating a texture and "inserting" it into the engine, having an ID and whatnot. I talked with Hendrik, he gave me some pointers about the features of his SDK and how it solves our problem. As a result I present you an improved version of Coherent UI in CryEngine3! You'll find the link for the source code at the end of this post.

This is a video of the end result with more of the same as last time. Except this time it's much more fluid.

First I'll start with some notes about the usage of the code:

We used CryEngine 3.4.0 (3696) Free SDK for our tests. If you want to use 3.4.3, you need to change the version string in Code/Plugin_SDK/inc/IPluginBase.h to "3.4.3". The code in the Coherent UI plugin doesn't have any version specific parts so it should be ready to use with the new SDK, although we haven't tested it.
The 64-bit build hasn't been tested neither because we updated our developer machines with Windows 8 and the CryEngine executables do not start. We had an issue with the 32-bit executables as well - the login dialog just froze when either the Launcher or Editor were started but that turned out to be caused by the AV - after excluding the executables we could finally start using CryEngine again.
CryEngine 3 is one of the top engines on the market - as such it squeezes every bit of resources the machine has. On the other hand, Coherent UI is designed as an out of process interface framework that needs some CPU/GPU as well. There's a chance that CryEngine will take most of the system resources and the UI can appear laggy. The best way to alleviate the problem in the free SDK is to check your FPS (use r_DisplayInfo 1 to show dev info if not already shown) and set a maximum framerate value (using sys_MaxFPS value) to something a little lower than what you're getting from the FPS counter.
When in-game, you can hide the default CryEngine interface using g_ShowHUD 0
The Editor isn't fully supported; while in the Launcher we create and destroy Coherent UI Views in OnLoadingStart/OnUnloadComplete callbacks, these are unsuitable for the sandbox because OnLoadingStart is called even if you aren't editing a level and OnUnloadComplete isn't called at all. To work around this limitation, we're creating the views in the OnActionEvent callback when the user enters game mode. This has some drawbacks since we're registering the player event listener in the same callback, but in another listener that is called before the one in the Coherent UI DLL. This means that player notifications will not work in the editor (e.g. the health change notification).

Plugin installation

The Coherent UI plugin requires the Plugin SDK and D3D Plugin to function properly (and Coherent UI itself, of course). They are included in the sample code (with no modifications, except for the version string so it matches the 3.4.0 SDK) at revisions d2feebc5d09ca367bc63d42589de6c2cf606b0d8 and c654e77255344a1ebe72c2365144c3d0908fd903, respectively.

First, setup the include and library directories for Coherent UI. To do so, get the package from our Download page and extract the contents of the lib directory to your CryEngineRoot/Bin32 dir (you should have CoherentUI.dll in the Bin32 directory now). Then create a new folder in the CryEngineRoot/Code folder named CoherentUI and extract the contents of the include directory of the package there. You should end up with the following directory tree: CryEngineRoot/Code/CoherentUI/Coherent. Now you're ready to use the Coherent UI framework. Next is the plugin setup.

If you download the whole repository with the Coherent UI plugin, you just need to copy the files in the root directory and build the provided solution.

If you download the plugin only there are a few more steps. Follow these instructions for installation of the Plugin SDK. After you're done, you need to set up the Coherent UI plugin. To do so, add the following to the CGame class:

Add the following action event handler as well (it's used for registering a player event listener provided by the Coherent UI plugin that does the binding between C++ and JavaScript methods):

You'll also need to setup include and library paths for the CryGame project.

Plugin limitations

When using a DirectX9 renderer, Coherent UI Views can only use shared memory (and not shared textures), since CryEngine uses a IDirect3DDevice9, while shared textures require a IDirect3DDevice9Ex. Creating a shared texture without an Ex device will just fail.

When using a DirectX11 renderer, HUD Views cannot be drawn, because the D3D plugin doesn't hook to the IDXGISwapChain::Present method. This is because there is no specific swap chain associated with the device which means you cannot use the device to get the swap chain. I thought of hooking to IDXGIFactory::CreateSwapChain and using the device and HWND that I have to determine the instance of the swap chain so I can hook to the present method, but that's left for the future. Shared textures can be used, though. Generally, it's recommended to use the DX9 renderer so you can experience all the features.

Plugin architecture

The Coherent UI plugin follows the Plugin SDK architecture and exposes just a tiny bit of functions that are needed by CryGame. These functions are:

Initialize
Shutdown
GetPlayerEventListener

The first two are self-explanatory; GetPlayerEventListener is used only because we can't register a player listener outside CryGame since the CPlayer class is not exported.

Everything else related to the UI is encapsulated in the Coherent UI plugin DLL.

Class list

CoherentGeometry - Loads geometry in .obj format and intersects rays with the loaded geometry, returning distance to the hit point and barycentric coordinates. This class is used because I couldn't find a way to get the texture coordinates out of the CryEngine provided methods (the physics query did not return texture coordinates)
CoherentInputEventListener - IInputEventListener implementation that converts CryEngine input events to Coherent input events.
CoherentPlayerEventListener - Used for demonstrating binding of engine events to UI events.
CoherentSystemEventListener - UI system events listener, required by Coherent UI.
CoherentUISystem - Deals with the communication with the UI system, managing Coherent UI views, creation of rendering resources and all UI logic.
CoherentViewListener - Reports events for Coherent UI Views and performs low-level drawing; it can also own a CoherentGeometry object that is considered the collision geometry for the View.
CPluginCoherentUI - Implements the plugin interface.
FullscreenTriangleDrawer - Does exactly what the name implies. Used for drawing the HUD.

Code walkthrough

The classes that do most of the work are CoherentUISystem and CoherentViewListener, so the focus will be on them. We'll just skim through the CoherentInputEventListener briefly, so you know what's happening when you press buttons. The input listener does nothing fancy - it just converts CryEngine input events to Coherent UI input events. Examine the code if you're interested how to do that.

It maps the following keys to some useful functions:

Numpad0 stops player input and allows forwarding to Coherent UI Views
Numpad2 toggles drawing the Coherent UI HUD
Numpad3 hides and shows the mouse cursor while playing.

The input is forwarded to the Coherent UI View that you last moused over (it helps showing the mouse so you know where the cursor is).

To summarize, when you want to send input to a Coherent UI View, press Numpad0 to stop player input and enable forwarding to Coherent UI, press Numpad3 to show the cursor, and mouse over the view that you want to send input to.

On to the CoherentViewListener. This class has a pointer to a DirectX texture that it draws to when it receives the OnDraw notification. It's important that we draw in the Render Thread or otherwise two threads could simultaneously access the DirectX device and cause a corruption. The OnDraw notification is called in the same call stack that Coherent::UI::UISystem::FetchSurfaces is - that means FetchSurfaces must be called from the render thread. We do that by using the D3D plugin which hooks to functions that are always called in the needed context. The drawing itself consists of copying one surface onto another, nothing fancy there.

The listener is also notified when Coherent UI needs to create or destroy textures. These notifications are in the Coherent::UI::UISystem::Update call stack, which is in the main thread. This means we can't use the device immediately and we need to ask the CoherentUISystem object to do the job in the rendering thread - that's no problem because Coherenet UI supports asynchronous surface creation. The last thing the listener does is providing a convenient method for ray casting through the listener's collision geometry, returning the texture coordinates of the hit point (if any).

Last, the CoherentUISystem class - it's pretty much the alpha and omega of the plugin. It receives DirectX notifications from the D3D plugin, executes tasks scheduled for the render thread (such as creating and destroying textures), manages the Coherent UI Views and encapsulates all the UI logic. It can also make coffee if you ask nicely :).

I tried to keep the UI logic as realistic as possible and this is what I came up with: the UI system itself is initialized in the plugin when the game starts and is active throughout the its lifetime. When a new level starts loading, view listeners and their corresponding Views are created for the HUD and objects. The object names are hard coded for the purposes of the example and are relevant for the level provided. Also during loading the input and player listeners are registered. Everything that is created when loading is destroyed when unloading a level.

The management of Coherent UI Views mostly consists of creating and destroying textures, requested by the views and providing help with the drawing of HUD Views. The CoherentUISystem maintains a collection of requests for texture creation that need to be executed on the render thread. These requests are processed in the CCoherentUISystem::OnPostBeginScene every frame.

Views that are placed on objects are updated in the CCoherentViewListener::OnDraw method - a texture is rendered to and later used by CryEngine, no further work required. When displaying a HUD, there's a bit more to it - it's a separate texture that needs to be drawn after all post-processing and is not associated with any object. For this purpose we're doing that in the CCoherentUISystem::OnPrePresent callback - we just render a full screen triangle with the HUD texture mapped onto it. The textures from Coherent UI come with premultiplied alpha so before drawing the triangle we need to set the blending modes to One/Inverse Source Alpha for the source/destination targets respectively. Note that as we mentioned this callback is not available in the D3D plugin for the DX11 renderer, so no HUD there. Creating the textures used by CryEngine is done through the D3D plugin. Examine the CCoherentUISystem::SetTexturesForListeners method to see exactly how.

Check out the full source code for the plugin at github. Bear in mind that the limited version supports only one Coherent UI View, so you'll have to choose which one you want. There's a friendly error message during compilation that will direct you to the code that you need to change.

If you want to see all of this in your CryEngine, go to our Download page to start experimenting!

Thursday, January 24, 2013

6 reasons to be at GDC 2013

Is it worth being at GDC 2013? 2 months left till the biggest game developers conference in San Francisco and many game devs are still asking themselves this question. Even the ones who have already bought passes.

The game developers who haven't been to the event yet may not know about all the benefits of being there. This is why we wanted to share with you our personal experience of being there before and point out the top 6 reasons to be at GDC 2013 (according to us). So if you find yourself benefiting from any of them then you should definitely go and meet us there together with more than 22 500 geeks passionate about making games.

Networking

GDC is probably the best place to meet people from the game industry. If you're new in the business this is your chance to get to know people. It's not rare for game development companies to find publishers or investors or for students to find a way to break into the industry. Veterans find it also very useful to network with other attendants. You never know which contact will be of use for you so this is why you should try to meet as many people as possible. If you're lucky you might even get invited to private parties and special events.

Getting coverage

Many game developers manage to meet bloggers or journalists at GDC and find a way to intrigue them about their projects. As a result they not only get coverage but manage to build long-lasting relationships with key people from the Media.

Learning

GDC is also your key opportunity to learn about the newest technology from the biggest players in the industry. This includes AAA developers like Activision Blizzard and BioWare talking about their latest and most successful projects. So far the organizers has revealed talks about Assassin's Creed III, SSX, Dead Space 3, Diablo III , Hitman: Absolution, Mass Effect 3, The Walking Dead and many more.

There is also a big focus on indie game development. If you're into free to play gaming, you might find it useful to listen to what Zynga is going to say about it. One of the talks we are looking forward to the most is the one of Valve about Virtual Reality and its future implementation into gaming.

Inspiration

Meeting, talking and learning from so many people from the game industry will definitely inspire and motivate you to build better games and software in general. You will get so many new ideas you could implement instantly into your project that it will be probably best to write them down on spot. Otherwise you might forget and lose great opportunities.

Rethinking your business model

GDC will help you to rethink your business model as well. You will learn more about current and new platforms and how to use them for your game. You will also have the chance to learn about new ways to monetize your game or software and better ways to promote it among your target audience.

Hiring talent

The people that are willing to go to GDC are really serious about their career. So if you're looking for talent this might be the best place to find it. Keep in mind that if you're still not a well-known company it will be probably harder for you to attract potential employees, so you will really need to show them the benefits of working with you.

If you believe you might benefit from any of these points, it might be a good idea to get a pass for the conference and to use this opportunity. If you decide to go or already have, we invite you to visit us there at booth #241 (near the IGF booth). You might as well request a meeting in advance since we don't know yet how busy our schedule will be. In all cases, we'd love to meet you there and learn more about your projects.

Monday, January 21, 2013

Converting a gyp library to premake

Here at Coherent Labs, we have a large and cross-platform project. The build system is crucial for a project like ours - it pretty much defines the programmer workflow and iteration time. How we chose our build system deserves a blog post of its own. But at the end - we chose premake - it is very high level, generates almost perfect Visual Studio project files and you have a complete programming language for describing how to build your project.

The idea for this post was to show how to convert a classic C library using autotools (./configure && make) to a premake-based project. But half of the post would be a repetition of Converting a C library to gyp, because the most complex part is generating all the config files for the current platform. So I decided to take node-lame, a node-gyp based project and convert it to premake.

Premake projects are written in plain Lua, typically in a file called premake4.lua. The "Hello World" project file looks like:

Next step is to include the right config.h file. To make our lives simpler we are going to use "token replacement", a relatively new feature, available in the premake-dev branch. Token replacement allows

the declarative syntax of premake to fallback to imperative functions. So we write a global function and place a call to it inside %{} in the include directory specification. This function will be called for each concrete configuration of the project and the %{} is replaced with the result.

libmpg123 has several functions written in assembly for x64 platforms, but they cannot be used on Windows. Here again we use Lua to avoid duplicating the lists of defines and files:

The next step is to define the dependencies between the projects. We include the projects in the solution and for each project declare that it links with its dependencies. This will ensure that all projects will be built in the correct order and the correct outputs will be used for linking. Unfortunately the include paths dependencies are not solved so easily. bindings depends on mpg123 and includes mpg123.h, so it needs deps/mpg123/src/libmpg123 as include path. But mpg123.h needs config.h, so bindings needs mpg123's config directory as include path, too. So each project has to know not only the directories of its dependencies, but their include directories also.

This is a point where gyp shines - 'direct_dependent_settings' allows you to add include and library paths to dependent projects. Premake has a planned feature - "usage projects" that will provide similar functionality.

As I said node-lame uses node-gyp, which takes care of installing the correct headers and libraries of node.js for your platform. With enough scripting in Lua we can achieve the same effect, but, for simplicity, we are going to add an option to premake4, specifying the location of the node.js source tree with built libraries.

At last lets see where the high abstraction of premake makes it so good:

This section sets linking against the static run-time and enables debug symbols on Windows and the architecture for Xcode.

And the same for premake (but on all platforms)

You may ask why bother converting third party libraries to your own build system. Well - to be sure they are build with the correct (same) settings as your own project and to avoid building all variants and committing them under source control - here we already have 18 combinations of configurations and platforms - and no one wants to maintain that many binaries.