Getting started with Coherent UI for Unity3D (Video Tutorial)

We are proud to present to you our first training video for Unity3D. This tutorial covers the basics of using the Coherent UI package, from importing and installing the package to assigning UI Views to different surfaces. You'll see how to render your HTML files projected on the camera or on a 3D object inside the world of your game. You'll also learn how to use our custom coui:// protocol to display files from your local filesystem.

We'll put the cherry on the top by showing you a peek at the Coherent UI Debugger which allows live editing of your HTML. It gives you much more flexibility than you'll see in the video of course, but showing them requires a tutorial of it's own... which is coming soon

For the purposes of this video tutorial, we have used our Standard package of Coherent UI.

You can request your free trial from here, or purchase a Mobile, Basic, Standard or Pro version of Coherent UI for Unity3D from here.

Please stay tuned for more video tutorials, coming out soon :)

Coherent UI for .NET PRO is now available as a standalone product

We at Coherent Labs, have always been working towards one main goal - to meet your needs and provide a great solution, no matter if you are developer, artist or producer. Less than two weeks ago we released our Mobile Beta version for Unity3D and now we have already prepared something new for you.

We are glad to introduce you our new standalone product - Coherent UI for .NET.

Main features:

• .Net API for the native Coherent UI library

• fully cross-platform web browser control for WinForms, WPF and MonoGTK, covering Windows, Mac OS X and Linux.
• automatic binding of C# objects to the JavaScript universe.

Coherent UI for .Net browser control allows you to embed a browser inside your .Net application and still run on all desktop platforms using a single API. Coherent UI for .Net provides standards compliant HTML5 browser with SSL support.

Coherent UI for .Net also integrates seamlessly in your XNA, MonoGame, SharpDX or SlimDX game with the complete features of the native library. You can use Coherent UI for .Net to add HTML5 user interface, in-game browser and video player to your game or integrate it with the social networks.

The professional version is available for purchase on our website for $899. We also encourage you to try our Trial version, which is available for download here.

Coherent UI is now available on Unity3D Asset Store

We’re really happy to announce that Coherent UI for Unity3D is now available on the Asset Store. Buy Basic or Standard version or download a free trial from Coherent Lab’s website.   

All versions of Coherent UI for Unity3D can be used for the creation and implementation of HUDs, menus, dialogs and in-game browsers. Our GUI library currently supports Windows (32 and 64 bit) and Mac OS X and soon we’re going to add Linux as well.

The Basic version covers all the primary needs of game UI developers allowing them to have both HUD and in-game browser at the same time. Standard gives more creative freedom because it allows unlimited number of views, SSL support, on-demand views and control view framerate, features needed for more complex projects.

We encourage Unity3D developers to take a look at our GUI library and to give us their feedback about it. All of your reviews and ratings will help us a lot with the future development of the product. We are always open to discuss ideas for new features and tools that will be of benefit for game UI developers.

For a quick start guide we suggest a look at our previous post Coherent UI in the Unity3D editor - Introduction. For more resources and tutorials follow the Unity3D tag on our blog or @CoherentLabs on Twitter.

Coherent UI with new release 1.1.5

Coherent Labs is happy to announce that after a few months of hard work we are releasing a new version of Coherent UI for desktop platforms. The new things are official support for Mac, performance improvements and many bug fixes. For a complete list check out our changelog.

Coherent UI will be available in the Asset Store

There are good news for Unity3D developers as well. We have submitted Coherent UI for both Windows and Mac platforms to the Asset Store and soon enough you will be able to purchase our game ui middleware directly through it. We are also going to keep the option to buy it on our website.

License and pricing options

The update comes with some changes in our pricing and license options. We have created two product suites: Coherent UI native and Coherent UI for Unity3D. Both of them support all desktop platforms and come with 32 and 64 bit versions.

1. Coherent UI native - for C++ & .NET projects on Windows/Mac/Linux

We provide 3 types of licensing options:

• Indie - $99/seat

This version is available for game development companies with teams of 5 or less employees with annual gross revenue of $50 000 or less for the previous fiscal year.

• Pro - contact us

The Pro version is designed for larger studios and has no limitations.

• Trial

We allow developers to try Coherent UI by downloading a trial of the Pro version. As in the full version you are allowed to take advantage of all features and you don’t have any run-time restrictions. However, it can’t be used for commercial purposes.

2. Coherent UI for Unity

• Basic - $150/seat

Basic is limited to only 2 run-time view and has to be purchased per Unity3D Pro seat as is the official asset store policy. This is ideal for users who want to integrate either only an in-game browser and/or an HTML5-based UI.

• Standard - $350/seat

Standard has a no limits whatsoever. It is the best option for a team of one or several people that need a bit full freedom with their UI - you could have both an in-game browser and an HTML5 UI as well as UI + video, etc.

• Pro (per title) - contact us

Pro is designed for larger studios using Unity3D as a game engine that want to take advantage of all of Coherent UI's features and need other licensing options.

• Trial

We also allow game developers to try Coherent UI for Unity3D for free. The trial is based on the Standard version and can't be used for commercial purposes.

All versions come with extensive documentation, sample projects and quick-start guides so you can start implementing your new UI right away.

For those of you who are looking forward to try our game UI middleware, please don’t hesitate and download it here for native and here for Unity3D.


Stay tuned for our new releases coming very soon. We continue to improve Coherent UI, to add new features and platforms. We need your feedback to make it even better. Try it and let us know what you think!

Adding a HTML5 minigame in Unity3D with Coherent UI (Tutorial)

Today, we're starting a new series of blog posts with educational purposes. These posts will essentially be short tutorials for some cool new usage of HTML in your game. Let's get right to the point and see what this first tutorial has to show.

Goal

Embed an existing HTML5 minigame that opens a door upon completion.

For this demo we've chosen a minigame called "Memory Box" - you can try it out on this URL. All credit for the game goes to its authors. There's only one change in the code - we added an inline CSS style style="background-color:rgba(0,0,0,0);" to the <body> element and removed the background-color: #000; line from the CSS at the top of the HTML file to make the background transparent.

If you're very eager to see the final result, the video is at the bottom of the article.

Prerequisites

• The AngryBots scene that comes with Unity3D
• A cube mesh (the Unity3D box has flipped texture coordinates and shows Coherent UI textures upside down). We used this one.
• Coherent UI for Unity3D (we'll also assume that you've already imported the package in the AngryBots project)

Scene setup

The first thing to do is to pick a door that will be opened when the user completes the minigame. In the AngryBots scene most of the doors are prefabs that have pre-attached scripts. I picked an internal door near the hologram around the entrance. It uses the TriggerOnPresence.js script and I'll refer to it throughout this article, so keep that in mind if you choose another door (external doors use a different script, for example).

Now that you have picked a door, add the cube-textures mesh somewhere around the door. It will be used as a surface where the minigame will be rendered. You don't need the whole cube, just the front face will suffice, so feel free to delete the unneeded ones. It's helpful to rename the new GameObject to something meaningful such as "CoherentSurface". That's how we'll be referring to the object from now on.

After you're happy with the placement of the object, add a CoherentUIView component to the surface so we can display HTML on it. Configure the URL, the resolution and make sure to tick the checkbox for transparency (you might not need it, depending on the minigame) and "Click to focus". The latter redirects input to the CoherentUIView after you click on the surface and is useful if you can't be bothered to write input forwarding code :).

If you hit play now you'll already have the minigame up and running, but it won't open the door yet. The setup so far should look like this:

CoherentUIView setup for the minigame

Short digression: if you tried playing the minigame, you'll notice that the player is firing when you click the dots. This can be distracting, so if you want to stop the firing you can add this code to TriggerOnMouseOrJoystick.js's Update method:

This will toggle the functianlity of the script (which is to fire bullets only) when the user presses the spacebar. Feel free to add any other logic, such as toggling firing when the CoherentUIView gains or loses focus or anything else that you might like.

Door opening logic

The door's opening logic defined by the TriggerOnPresence.js script does the following: when the user enters the object's collision volume, the OnTriggerEnter function is called, which in turn notifies all the subscribers for the specified signal. When the user exits the volume, the OnTriggerExit function is called and notifies the subscribers for the exit signal.

When inspecting the prefab, you'll find out that this signaling breaks down to sending the "OnPlay" and "OnPlayReverse" messages to the "AudioSource" and "slidingDoor" sub-GameObjects. Now that we know how to open and close the door, we can start making changes.

We won't need the proximity triggers anymore so just disable the TriggerOnPresence component from the sliding door GameObject. We'll also need to be able to identify the door's "AudioSource" and "slidingDoor" sub-objects, so we have to rename the door to something unique, such as "interiorDoorSlidingMinigame". After we have this we can open the door with the following one-liner:

GameObject.Find("interiorDoorSlidingMinigame/slidingDoor").SendMessage("OnPlay");

Great!

Wiring the minigame completion to opening the door

First, we want to have means for sending messages to the game from JavaScript. To do that, add the coherent.js script that comes with the Coherent UI Unity3D package somewhere in the top of the HTML like so (the location of the src property may be different):
"<script type="text/javascript" src="coherent.js"></script>".
This will define the "engine" object, which you can use for communication with the game.

Next, we have to find the code that's executed when you complete a level. That's at the bottom of memorybox.js, in the "if(this.isLastCorrectSquare)" conditional. If the script is totally unreadable to you, you can process it using jsbeautifier.org or some other service. Add the line "engine.call("OpenDoor");" at the end of the if block. This will send the message "OpenDoor" to the corresponding ViewListener. That's all for the HTML/JS side - just 2 lines of code!

Last, we have to handle the message sent by JavaScript in the game. To do that we make a new script, MinigameDoorOpener, and add it to the GameObject with the CoherentUIView (the CoherentSurface object). This helper script will subscribe for the View Listener's ReadyForBindings event and bind a handler for the "OpenDoor" message. The handler will simply open the door as we discussed previously. Its code is as follows (the sample script is written in C#, but you can use anything that Unity3D allows):

Now everything is wired and ready to use! After opening the door you can continue playing the minigame if you like it. You can also enhance the demo by enabling the TriggerOnPresence component of the door so it behaves like a normal door after you unlock it instead of just staying open.

Here's the video of the final result:

That's all for today's tutorial on adding big value to your game in 10 minutes. Stay tuned for more!

CryEngine 3 minimap made with Coherent UI

This is just a small update on our demo in CryEngine 3. We added a minimap using OpenLayers giving us all sorts of cartography goodness.

It's bird! It's a plane! It's a map alright!

Update: an interactive visualization of the map in your browser! (the marker uses the default OpenLayers marker image)

Move marker

Rotate marker

The movement and rotation are done automatically in the demo of course :).
[End update]

Our sample barely touches the surface of the possibilities offered by OpenLayers but it's enough for our purposes.  For simplicity's sake, I used a single non-tiled image for the map (which presents artifacts when enlarging the map). The image I used was the one I found in Game\Levels\Singleplayer\Forest\ - Forest.tif (which I converted to a format that doesn't take 16MB). The image isn't fabulous per se (it's blurry and has a lot of discontinuities) and if you want something really cool you can use an orthographic camera in the Editor and take snapshots of different tiles at different zoom levels. When you have the snapshots you can follow this tutorial to make OpenLayers use your local tiles and have a map with real zoom.

Most of the work for the map is done in JavaScript/CSS. You can toggle it by pressing Tab. The C++ part is basically just firing events for the player position and orientation.

The other C++ part (that's not actually related to the cartography) is forwarding the input to the HUD view - I made a new class (CCoherentHUDViewListener) that does the job. You can examine it if you're interested. As a side note, when forwarding input to transparent Coherent UI Views make sure that you set the ViewInfo::SupportsClickThrough to true when creating the view.

The interesting bits of JavaScript for the map are in Bin32/TestPages/hud/js/map.js. There's actually not much to it - we create an OpenLayers map, add a marker representing the player and wire some events that will come from the engine. Here's the whole script (stripped down to the essentials)
Just a few notes about the code - I found the bounds of the area depicted in Forest.jpg by flying around in the Editor and writing down some approximate numbers. You'll notice that I used negative coordinates on the Y axis - that's because it allows easier conversion from the CryEngine coordinate system to the OpenLayers one. If I used positive numbers, I'd have to write a fully fledged linear transform (a number and a minus in this case) instead of just the minus :).

The second note is about the orientation of the player marker - OpenLayers does not provide means for rotating the marker image so we have to use CSS. Fortunately, the marker object gives us all the information about the DOM element that contains the marker so that's no problem.

The HTML code for adding the map is simply <div id="map" class="smallmap">, where the smallmap class defines the size of the div element.

If you want to make the map circular you can clip it using a simple border-radius trick like so:
Here's an example of what this does (may not work on all browsers):

Original

 

Using border-radius

Or you can use some more advanced HTML/CSS technique that fits your needs.

That's all for today's update, the updated code is on GitHub so feel free to try it out and extend it. We'll be preparing the CryEngine3 showcase for GDC this year, so if you're in the Bay area at the end of March feel free to visit as at booth #241 and check it out live!

Coherent UI in CryEngine 3 (Redux)

After the article on CryDev.net about Coherent UI, a lot of people read our previous blog post about our experiment with CryEngine3 and suggested that we should use Hendrik Polczynski's Plugin SDK to get low-level access to the CryEngine3 renderer. The problem we previously had was that we couldn't find a proper way of creating a texture and "inserting" it into the engine, having an ID and whatnot. I talked with Hendrik, he gave me some pointers about the features of his SDK and how it solves our problem. As a result I present you an improved version of Coherent UI in CryEngine3! You'll find the link for the source code at the end of this post.

This is a video of the end result with more of the same as last time. Except this time it's much more fluid.

First I'll start with some notes about the usage of the code:

• We used CryEngine 3.4.0 (3696) Free SDK for our tests. If you want to use 3.4.3, you need to change the version string in Code/Plugin_SDK/inc/IPluginBase.h to "3.4.3". The code in the Coherent UI plugin doesn't have any version specific parts so it should be ready to use with the new SDK, although we haven't tested it.
• The 64-bit build hasn't been tested neither because we updated our developer machines with Windows 8 and the CryEngine executables do not start. We had an issue with the 32-bit executables as well - the login dialog just froze when either the Launcher or Editor were started but that turned out to be caused by the AV - after excluding the executables we could finally start using CryEngine again.
• CryEngine 3 is one of the top engines on the market - as such it squeezes every bit of resources the machine has. On the other hand, Coherent UI is designed as an out of process interface framework that needs some CPU/GPU as well. There's a chance that CryEngine will take most of the system resources and the UI can appear laggy. The best way to alleviate the problem in the free SDK is to check your FPS (use r_DisplayInfo 1 to show dev info if not already shown) and set a maximum framerate value (using sys_MaxFPS value) to something a little lower than what you're getting from the FPS counter.
• When in-game, you can hide the default CryEngine interface using g_ShowHUD 0
• The Editor isn't fully supported; while in the Launcher we create and destroy Coherent UI Views in OnLoadingStart/OnUnloadComplete callbacks, these are unsuitable for the sandbox because OnLoadingStart is called even if you aren't editing a level and OnUnloadComplete isn't called at all. To work around this limitation, we're creating the views in the OnActionEvent callback when the user enters game mode. This has some drawbacks since we're registering the player event listener in the same callback, but in another listener that is called before the one in the Coherent UI DLL. This means that player notifications will not work in the editor (e.g. the health change notification).

Plugin installation

The Coherent UI plugin requires the Plugin SDK and D3D Plugin to function properly (and Coherent UI itself, of course). They are included in the sample code (with no modifications, except for the version string so it matches the 3.4.0 SDK) at revisions d2feebc5d09ca367bc63d42589de6c2cf606b0d8 and c654e77255344a1ebe72c2365144c3d0908fd903, respectively.

First, setup the include and library directories for Coherent UI. To do so, get the package from our Download page and extract the contents of the lib directory to your CryEngineRoot/Bin32 dir (you should have CoherentUI.dll in the Bin32 directory now). Then create a new folder in the CryEngineRoot/Code folder named CoherentUI and extract the contents of the include directory of the package there. You should end up with the following directory tree: CryEngineRoot/Code/CoherentUI/Coherent. Now you're ready to use the Coherent UI framework. Next is the plugin setup.

If you download the whole repository with the Coherent UI plugin, you just need to copy the files in the root directory and build the provided solution.

If you download the plugin only there are a few more steps. Follow these instructions for installation of the Plugin SDK. After you're done, you need to set up the Coherent UI plugin. To do so, add the following to the CGame class:

Add the following action event handler as well (it's used for registering a player event listener provided by the Coherent UI plugin that does the binding between C++ and JavaScript methods):

You'll also need to setup include and library paths for the CryGame project.

 Plugin limitations

When using a DirectX9 renderer, Coherent UI Views can only use shared memory (and not shared textures), since CryEngine uses a IDirect3DDevice9, while shared textures require a IDirect3DDevice9Ex. Creating a shared texture without an Ex device will just fail.

When using a DirectX11 renderer, HUD Views cannot be drawn, because the D3D plugin doesn't hook to the IDXGISwapChain::Present method. This is because there is no specific swap chain associated with the device which means you cannot use the device to get the swap chain. I thought of hooking to IDXGIFactory::CreateSwapChain and using the device and HWND that I have to determine the instance of the swap chain so I can hook to the present method, but that's left for the future. Shared textures can be used, though. Generally, it's recommended to use the DX9 renderer so you can experience all the features.

Plugin architecture

The Coherent UI plugin follows the Plugin SDK architecture and exposes just a tiny bit of functions that are needed by CryGame. These functions are:

• Initialize
• Shutdown
• GetPlayerEventListener

The first two are self-explanatory; GetPlayerEventListener is used only because we can't register a player listener outside CryGame since the CPlayer class is not exported.

Everything else related to the UI is encapsulated in the Coherent UI plugin DLL.

Class list

• CoherentGeometry - Loads geometry in .obj format and intersects rays with the loaded geometry, returning distance to the hit point and barycentric coordinates. This class is used because I couldn't find a way to get the texture coordinates out of the CryEngine provided methods (the physics query did not return texture coordinates)
• CoherentInputEventListener - IInputEventListener implementation that converts CryEngine input events to Coherent input events.
• CoherentPlayerEventListener - Used for demonstrating binding of engine events to UI events.
• CoherentSystemEventListener - UI system events listener, required by Coherent UI.
• CoherentUISystem - Deals with the communication with the UI system, managing Coherent UI views, creation of rendering resources and all UI logic.
• CoherentViewListener - Reports events for Coherent UI Views and performs low-level drawing; it can also own a CoherentGeometry object that is considered the collision geometry for the View.
• CPluginCoherentUI - Implements the plugin interface.
• FullscreenTriangleDrawer - Does exactly what the name implies. Used for drawing the HUD.

Code walkthrough

The classes that do most of the work are CoherentUISystem and CoherentViewListener, so the focus will be on them. We'll just skim through the CoherentInputEventListener briefly, so you know what's happening when you press buttons. The input listener does nothing fancy - it just converts CryEngine input events to Coherent UI input events. Examine the code if you're interested how to do that.

It maps the following keys to some useful functions:

• Numpad0 stops player input and allows forwarding to Coherent UI Views
• Numpad2 toggles drawing the Coherent UI HUD
• Numpad3 hides and shows the mouse cursor while playing.

The input is forwarded to the Coherent UI View that you last moused over (it helps showing the mouse so you know where the cursor is).

To summarize, when you want to send input to a Coherent UI View, press Numpad0 to stop player input and enable forwarding to Coherent UI, press Numpad3 to show the cursor, and mouse over the view that you want to send input to.

On to the CoherentViewListener. This class has a pointer to a DirectX texture that it draws to when it receives the OnDraw notification. It's important that we draw in the Render Thread or otherwise two threads could simultaneously access the DirectX device and cause a corruption. The OnDraw notification is called in the same call stack that Coherent::UI::UISystem::FetchSurfaces is - that means FetchSurfaces must be called from the render thread. We do that by using the D3D plugin which hooks to functions that are always called in the needed context. The drawing itself consists of copying one surface onto another, nothing fancy there.

The listener is also notified when Coherent UI needs to create or destroy textures. These notifications are in the Coherent::UI::UISystem::Update call stack, which is in the main thread. This means we can't use the device immediately and we need to ask the CoherentUISystem object to do the job in the rendering thread - that's no problem because Coherenet UI supports asynchronous surface creation. The last thing the listener does is providing a convenient method for ray casting through the listener's collision geometry, returning the texture coordinates of the hit point (if any).

Last, the CoherentUISystem class - it's pretty much the alpha and omega of the plugin. It receives DirectX notifications from the D3D plugin, executes tasks scheduled for the render thread (such as creating and destroying textures), manages the Coherent UI Views and encapsulates all the UI logic. It can also make coffee if you ask nicely :).

I tried to keep the UI logic as realistic as possible and this is what I came up with: the UI system itself is initialized in the plugin when the game starts and is active throughout the its lifetime. When a new level starts loading, view listeners and their corresponding Views are created for the HUD and objects. The object names are hard coded for the purposes of the example and are relevant for the level provided. Also during loading the input and player listeners are registered. Everything that is created when loading is destroyed when unloading a level.

The management of Coherent UI Views mostly consists of creating and destroying textures, requested by the views and providing help with the drawing of HUD Views. The CoherentUISystem maintains a collection of requests for texture creation that need to be executed on the render thread. These requests are processed in the CCoherentUISystem::OnPostBeginScene every frame.

Views that are placed on objects are updated in the CCoherentViewListener::OnDraw method - a texture is rendered to and later used by CryEngine, no further work required. When displaying a HUD, there's a bit more to it - it's a separate texture that needs to be drawn after all post-processing and is not associated with any object. For this purpose we're doing that in the CCoherentUISystem::OnPrePresent callback - we just render a full screen triangle with the HUD texture mapped onto it. The textures from Coherent UI come with premultiplied alpha so before drawing the triangle we need to set the blending modes to One/Inverse Source Alpha for the source/destination targets respectively. Note that as we mentioned this callback is not available in the D3D plugin for the DX11 renderer, so no HUD there. Creating the textures used by CryEngine is done through the D3D plugin. Examine the CCoherentUISystem::SetTexturesForListeners method to see exactly how.

Check out the full source code for the plugin at github. Bear in mind that the limited version supports only one Coherent UI View, so you'll have to choose which one you want. There's a friendly error message during compilation that will direct you to the code that you need to change.

If you want to see all of this in your CryEngine, go to our Download page to start experimenting!

What can Unity 4 and DirectX 11 do together?

With the new version Unity 4 introduces support for DirectX 11 which can really take your graphics to the next level. The company released a video explaining a bit about the new features, however we believe it was not enough for the game developers to learn even slightly for all the possibilities.

Using DirectX 11 with Unity 4 means taking full advantage of features like:

Shader model 5

The main purpose of this feature is to solve a common problem in current game engines: the upsurge in the number of shaders due to the large number of permutations. In other words, for each kind of material and light the game developers must include a shader in order to handle all cases.  

DirectX 11 offers an elegant solution through dynamic shader linkage.
 
Tessellation

Tessellation is one of the biggest features around DirectX 11 and Unity 4. It is simply breaking down polygons into finer pieces which brings profound improvements to 3D graphics. For example, game developers can cut a square across its diagonal to make 2 triangles and use them to depict new information. You can see the difference in the images below.

Without Tessellation

With Tessellation

Compute shaders

Compute shaders provide high-speed general purpose computing and takes advantage of the large numbers of parallel processors on GPU. They provide with memory sharing and thread synchronization features to allow more effective parallel programming methods.

A great example is this demo made by Nvidia showing a real-time simulated ocean under twilight lighting condition.



Unity 4, DirectX 11 and UI

As you may already know, Coherent UI have been recently integrated with Unity. Coherent UI natively supports DirectX 11 rendering and we are really happy that in Unity 4 it is supported too so users will be able to use its features to create better game UI. Inspired by this we integrated one of the Unity 4 samples with Coherent UI components and we made this demo to show you what Coherent UI, Unity 4 and DirectX 11 can do together.

As you can see there is a HUD on the main camera and two more Coherent UI Views on the paintings. You are allowed to scroll and select the paintings, interact with the UI and even play a game within the game. At the same time the performance has improved up to 50% compared to using Unity 3.5 version and DirectX 9.

If you’re interested in using Coherent UI with Unity 4, you’re welcomed to request a public beta of our integration with the game engine from our download page.

Unity3D integration with Coherent UI - Public Beta

Coherent UI is really happy to announce today the official public beta of our integration with the Unity3D game engine!

In other words, with Coherent UI you can build your game user interface in modern HTML5 & CSS3 and trivially integrate it in your Unity3D game.

If you were reading our blog, you might have known already that this was coming. Early this month in Coherent UI in Unity3D - First look we released a short teaser and later on we talked in another post about the upcoming official integration with the game engine.

For the curious ones we have prepared a video tutorial explaining how Coherent UI works inside the editor:





If this gets your curiosity and you'd like to get your hands on the public beta, don't hesitate to request your version on our download page.

We will highly appreciate your feedback about our integration with Unity3D. Feel free to contact us with any problem you might encounter or suggestions you have at 'info at coherent-labs dot com'.

Coherent UI in the Unity3D editor - Introduction

We would like to share a video showing the first version of the Coherent UI integration in the Unity3D editor. In this video we show how to:

• Import the Coherent UI package
• Create an in-game object with a web page on it
• Interact with a web page in the game
• Add a HUD powered by Coherent UI
• Build the game

We plan to release a Beta version of the product in a couple of weeks tops so stay tuned! For any suggestions or thoughts about the Unity integration please leave a comment or visit this thread.

How it works

The integration is very powerful yet very simple to use and requires little or no programming at all as we have wrapped everything in components with editor-accessible properties. However, we also expose to script all the features available in the .NET and C++ version of the library so more custom behavior can also be achieved.

What happens in the video:

1. We load an empty scene and add a floor, a light and a cube on which we'll project what in Coherent UI terms we call a 'View'. A view is something that Coherent UI renders - that could be a HUD element, a projected web page, an animation - anything wrapped in an HTML page.

Simple scene

2. We add a character controller so that we can move around

3.We import the Coherent UI package. The component we are interested in this tutorial is 'CoherentUIView'. We drag it on one of the faces of the cube. All the various properties of the View are editable in the Inspector. By default it will load google.com. Let's hit 'Play'.

A Coherent UI View on an object

4. We can see the web page on the game object in Unity!

5. We change the web page and the resolution.

6. Now we have a nicer page in a better resolution.

An interactable web-page projected on a game object

7. The next thing is to make the page interactable. To do this we add a Mesh Collider component to the mesh. Then a UserScipt component that just handles the raycast and mouse button translation to the Coherent UI View. We also add some code to the camera to stop it from receiving when we hit 'L', otherwise it's very hard to click on anything with the mouse-look on. That's it - the View is now fully interactable.

8. Now for the HUD - we just drag the 'CoherentUIView' component on our Main Camera. We have made the HUD resources and copy them in our project and set the View's Page property to coui://TestPages/demo/demo.html.

'coui' is a special protocol we use to signal that the resource is local and subject to loading through the file handlers supplied by the application. In this way you can use a custom resource manager as well as for instance encrypt your UI data. The HUD looks pixelated because the resolution we set does not coincide with the one of the Editor pane. It will however be OK in the game when we build it. In your games the resolution will always be tied to the actual resolution of the Camera (the back-buffer).

A sample HUD made rendered through Coherent UI

9. The Coherent UI integration handles all cases and detects if the component is attached to and object in the world or a camera. It is compatible with post effects. We can add an effect on the camera. By default that effect will be applied to the View also. However this might not be desirable so by clicking 'Apply After Post-Effects' you can disable them on the View.

Post effects can be applied or skipped on Coherent UI Views

10. We now just have to build the game. Coherent UI resources will automatically be copied and made available at runtime.

The built game

 All Coherent UI rendering happens in the native C++ plugin so it's impact on the performance of your game should be minimal. 

Porting Brackets to a new platform

Brackets is a code editor for HTML, CSS and JavaScript that is built in HTML, CSS and JavaScript. That makes it a fully featured desktop application written using Web technologies. Brackets embeds a browser to show and run the entire editor and extends it with additional functionality used by JavaScript. There are more and more applications using this kind of mixed platform - HTML/JavaScript becomes first class User Interface (UI) (and not only) for Window 8 applications, other tools from Adobe such as Edge Animate, Icenium - a new cloud IDE, recently announced by Telerik. Sencha Animator also is an HTML5 application, running inside QT WebKit. Even the UI of most browsers is written in HTML.
This approach has several advantages:

• fully cross-platform UI - as long as the browser is running on a platform, the UI is going to be running on that platform too
• full UI customization using CSS lots of available application frameworks MVC, MVVM - Backbone.js, knockout.js, you name it
• lots of UI libraries - Kendo UI, jQuery UI, mochaui
• easy development of the UI using Chrome Dev Tools, Firebug, etc.
• Live reload of the UI, without restarting the whole application!
• native access to filesystem and everything without sandboxing
• use existing native APIs
• native performance where necessary
• only single browser to support - this is a huge relief for the HTML/JavaScript developers

Given the advantages, I am sure that more and more mixed native and HTML applications are going to appear.

Brackets Architecture

Brackets consists of two applications running together - an HTML/JavaScript application and a native C++ shell that runs the HTML application. The native shell consists of two processes - one running the HTML application inside a browser, and one running the heavy native part like filesystem operations and number crunching. The communication between the two processes is asynchronous, using JavaScript callbacks and request / response for the native code.

• The shell extends the standard HTML DOM with the following functions:
• Open file or directory using the OS dialog
• Read file
• Write file
• Create directory
• Rename file or directory
• Get file last modification time
• List directory contents
• Open and close browser for Live Preview
• Open extensions directory in OS file browser
• Open developer tools for brackets itself
• Get the system default language
• Time since the application start
• Get the application support directory

Brackets Shell implements the asynchronous communication between the HTML render process and the browser in a very simplistic way: JavaScript executes a function appshell.fs.readdir(path, callback), the render process stores the callback in a mapping from a request id to callback and calls the native process with the name of the function. When the native process is ready it sends back any result of the call together with the request id. The HTML process finds the callback by the request id and executes it with any result.

This architecture is the same as for any good GUI application - the UI never executes any expensive functions and is always responsive and all data manipulation is in a separate thread.

Running Brackets in Coherent UI

Making Brackets run inside Coherent UI is really easy. We start by creating a view that loads www/index.html relatively to the executable. To support Coherent UI we have to include some JavaScript files in the index.html of Brackets

These scripts are Coherent UI dependencies, Coherent UI itself and the abstraction layer between all JavaScript code of Brackets and Coherent UI. Then we have to register our native callbacks for the asynchronous calls:

Brackets native functions always return an error code as first argument to the JavaScript callback. This mechanism can be implemented in Coherent UI, but then the callbacks always have to do two things - handle the correct result and handle the error, which is kind of annoying. engine.Call might take two callbacks - one for the successful result and one for error. Therefore we have to wrap the normal callback in an object with separate handlers for success and error.
gist: javascript callback wrapper
All that is left now is to wrap the callbacks and use engine.Call instead of native function.

and to write the native function

Handling synchronous calls  

Brackets has and some synchronous methods that return to JavaScript immediately. These methods are:

• Get the application support directory
• Get the system default language
• Time since the application start

Coherent UI does not support providing synchronous JavaScript functions by design, so we will have to work around that.

The application support directory remains constant through a single run of Brackets, so we can set it once and for all during application initialization:

Getting the system default language might be implemented in the same way. The last method left is time since the application start and is used only in Show Performance Data menu. This method might be implemented entirely in JavaScript, assuming Brackets is not going to be reloaded during the performance test run.

Porting Bracket to Linux

Since Coherent UI already runs on Linux and we have implemented most of the native functions using the cross-platform boost::filesystem library, all we have to do to get is showing an open file dialog, creating and closing a chrome instance opening an URL and folder in the default OS HTML browser and file manager.

We use the GtkFileChooserDialog and the xdg-open tool. Unfortunately, the Live Preview doesn't work under Linux. Live Preview in Brackets creates a new Google Chrome instance with enabled remote debugger, attaches to the debugger using XmlHttpRequest and WebSockets and controls the instance via the debugger. What happens on Linux is that one of the XmlHttpRequests fails with "DOM Exception" and the debugger is unable to attach to the instance.

Another Linux related issue is that I couldn't find a way to close a Google Chrome  tab on Linux gracefully, so when the developer tools are closed you get the "Ow, Snap" page. In a future version we will stop using Google Chrome for Live Preview and for showing the developer tools, which will fix this problem.

Here is a short video of Brackets running on Linux:

Get a prebuild package or get Coherent UI and start hacking!

Coherent UI in Unity3D - First look

We want to share with you the first video of the integration of Coherent UI with Unity3D:

Coherent UI already support HUD-type views as well as views mapped to objects with input. The integration is designed to be very user-friendly yet powerful. The demo shows the awesome CSS3 periodic table by Ricardo Cabello aka Mr. Doob and in Unity3D it looks even better.

Update: Check out the introduction to the integration of Coherent UI in the Unity3D editor.

Client application multi-threaded rendering support

Our on-going integration effort in Unity3D prompted us to accelerate a feature we've been planning for a long time but didn't have the time to finish until now - support for client applications with a multi-threaded rendering architecture.

Although it is even now possible to incorporate Coherent UI in an application with multi-threaded rendering, it is inherently difficult because the rendering-related events must be performed in the thread that updates the system.

For the sake of simplicity I'll call the thread that performs the logic in the client app the 'update' thread and the one that renders the 'rendering' thread.

There were two major challenges we needed to beat in order to support a separate 'rendering' thread in the client - rendering resource management and draw callbacks.

When Coherent UI needs a rendering resource - usually a texture, it calls the appropriate callback provided by the user. This happens when a new View has to be created or a View gets re-sized. The API expected the result of the operation to be immediately available.

We changed it so that it now uses a 'SurfaceResponse' object that must be signaled with the result of the operation. This signaling can happen at any later time, so the resource could be created in a separate thread. This is analogous to the resource requests (usually file-reads or file-writes) we already support in the same manner.

All new surfaces are fetched while calling 'UISystem::FetchSurfaces' for buffered Views and 'View::FetchSurface' for on-demand Views. This results in calls to the 'ViewListener::OnDraw' callback provided by the user. Usually a copy is made of the received surface for rendering and up until now those methods had to be called in the 'update' thread. Now it is perfectly safe to call them from your 'rendering' thread. This not only allows for easy integration with multi-threaded rendering pipelines but could be used as a performance optimization, as it voids the need to make an eventual copy of the surface to be used later for rendering.

Note that the 'ViewListener::DestroySurface' could now be called from both the 'update' and the 'rendering' thread but Coherent UI has already relinquished any ownership on the surface when it calls the method so it's trivial to dispose it even when the callback happens in the 'update' thread.

The API changes will be available in the next version of Coherent UI.