A high level shader construction syntax - Part III

This article is part of the a series of posts, the first ones can be found here: 

A high level shader construction syntax - Part I
A high level shader construction syntax - Part II

Compile-time conditions

Although the proposed shader syntax minimizes compile-time code branching, it is still needed. Sometimes it is required to perform a calculation only if a specified previous step has been selected by the polymorphic decision system. Such cases should be rare.

To address the issue two new keywords are introduced: CONTEXT_IF and CONTEXT_IFNOT. The syntax for using them is: 

The usage is:
CONTEXT_IF(context.some_semantic) {
...
}

When the CONTEXT_IF is parsed, the translator splats the code in the curly braces if (or not if) the semantic is currently present in the context.

For instance in the snippet above if specular_color has not been calculated (i.e. GetSpecularColor()) was None, then the semantic will be missing from the context. In that case the CONTEXT_IFNOT will be true and the code will be added to the final shader. Keep in mind that all those conditions are evaluated at shader translation time and at the moment the translator reaches the line with the context conditional. If the tested semantic appears after the test in code, it won't influence it.

The implementation supports condition nesting as well as polymorphics in conditions. However conditions can currently appear only in the main body of the shader and not in expanded polymorphics.

Implementation

A sample implementation of a translator that implements the enhanced syntax can be found on GitHub:
http://github.com/stoyannk/ShaderTranslator.
The sample can easily be modified to become a stand-alone library.

The major components are the ShaderTranslationUniverse and the ShaderTranslator. The universe is a holder for all the atoms, combinators and polymorphics. It is highly likely that all shaders in a product share a library of those components but it might not always be the case.

The ShaderTranslator performs the translation itself given a universe and the initial source code. It outputs valid HLSL SM4 code.

The translation process itself is heavily based on regular expression and performs a lot of string operations. To speed things up and reduce memory fragmentation, the temporary strings used during the translation process use a linear scratch allocator. It takes a big chunk of memory and always returns new blocks from it without ever freeing it. At the end of the translation the whole memory region is freed all at once. This scratch allocator is accessed as a thread-local variable. I wouldn't recommend however using the translator at runtime - it is best suited as a step in the build process of the final product.

I wasn't too pedantic at fixing allocations during the translation process so some containers used still access the default allocator. It would be trivial to change those too.
The main program comes with two sample shaders used for testing.

CryEngine 3 Integration

This week we're on a multimedia frenzy and we're presenting another video of Coherent UI :).

This one is about our experimental integration in CryEngine 3. It's experimental because our integration is with the free SDK, which doesn't expose enough resources for maximal performance (e.g. the DirectX device is not easily accessible). We wanted to have it as clean as possible (no hacky stuff!) to understand the problems one might have when trying to integrate Coherent UI in an existing engine. While this led to having sub-optimal performance (technical details below the video), it was still very good and we believe it's worth showing.

The video demonstrates some exciting features of Coherent UI:

• Displaying any website on any surface
• Support for HTML5/CSS3, SSL
• Social integration with Facebook, Twitter, Google+ (well, this isn't exactly demonstrated, but we have it :))
• JavaScript binding (for engine variables and methods)
• Live editing/debugging the interface

Without further ado (and I realize it's been quite an ado :)) I present you Coherent UI in CryEngine3!

Coherent UI in CryEngine 3

Coherent UI in CryEngine 3 (short version of the above)

Developer tidbits

Here's the story of the hurdles we had to overcome. Let me start with this first, the lack of access to the DirectX device was extremely annoying. So, now that we've got that clear, we just started our integration... and immediately ran into a problem.

There was no easy way to create an empty texture with specified size. Let alone create a shared texture for Windows Vista/7. This presented the first small inefficiency in the integration, since we had to use shared memory as image transport mechanism from Coherent UI to CryEngine 3, which involves some memory copying (and using shared textures doesn't). Even access to the device wouldn't help that much here since we'd like to have a valid engine handle for the texture, which means that we can't bypass it. After some experimenting, we settled on creating a dummy material with the editor, assigning a placeholder diffuse texture and getting its ID with the following code:

Having the ID, we can update the texture at runtime using IRenderer::UpdateTextureInVideoMemory. This approach comes with its own set of problems, however. An obvious one is that you need unique dummy material and diffuse texture for each Coherent UI View, which is annoying. Another problem is that this texture is not resizable, so we had an array of textures with common sizes that were the only ones allowed when resizing a View. The least obvious problem was that if the material's texture had a mip-chain, IRenderer::UpdateTextureInVideoMemory did not automatically generate the lower mip levels which resulted in some strange results, because of the trilinear filtering. It didn't perform any kind of stretching either, and that's why we only allowed preset View resolutions. You can see the mips problem here:

Problematic trilinear texture filtering

The placeholder texture

It took some time for figuring out since, at first, we didn't have fancy placeholder textures, but only solid color ones. The solution was to simply assign a texture that had only one surface (i.e. no mips). This presented another small inefficiency.

Ok, we have a texture now, we can update it and all, but how do we draw it on top of everything so it acts as a UI? After some digging in the SDK, we found that the CBitmapUI(and more precisely, IGameFramwork's IUIDraw) class should be able to solve this, having various methods for drawing full-screen quads. The color and alpha channel weights were messed up, however, so we had to call IUIDraw::DrawImage beforehand, which had the weights as parameters, so we could reset them to 1.0. We just drew a dummy image outside the viewport to reset these values, having yet another small inefficiency.

Moving on, to the biggest inefficiency of all - Coherent UI provides color values with premultiplied alpha. This means that transparency is already taken into account. When drawing the fullscreen quad, the blending modes in CryEngine are set to SourceAlpha/1-SourceAlpha for the source and destination colors, respectively, meaning that the source alpha will be taken into account again. What we had to is "post-divide" the alpha value, so when DirectX multiplies is we get the correct result. We had to do this for each pixel, involving both bitwise and floating point operations - imagine the slowdown for doing that on a 1280x720 or even 1920x1080 image. If we had device access, all that would be fixed with a single call for the blend mode but, alas, we don't. Also, if we used DirectX11 renderer, we'd have to do another pass on the pixels to swap their red and blue channels, because the component ordering has been changed since DirectX 10!

Next on the list was input forwarding - we wanted to add means for stopping player input(so we don't walk or lean or anything while typing) and redirecting it to Coherent UI, so we could interact with the Views. This wasn't really a problem but it was rather tedious - we had to register our own IInputEventListener that forwards input events to the focused View, if any. The tedious part was creating gigantic mappings for CryEngine 3 to Coherent UI event conversion. Stopping player input when interacting with a View was easy, too - we just had to disable the "player" action map using the IActionMapManager. We also needed a free cursor while ingame, so you can move your mouse when browsing, which was just a matter of calling the Windows API ShowCursor.

The final problem was actually getting the texture coordinates of the projected mouse position onto the surface below. I tried using the physics system which provided some sort of raycasts that I got working, but i couldn't do a refined trace on the actual geometry nor obtain it to do the tracing myself. And even if I managed to do that, I couldn't find any way to get the texture coordinates using the free CryEngine 3 SDK. That's why I just exported the interesting objects to .obj files using the CryEngine Editor, put the geometry into a KD-Tree and did the raycasting myself after all. For correct results, first we'd have to trace using the physics system so we know that no object is obstructing the View. Then, we trace in the KD-Tree and get the texture coordinates, which can be translated to View coordinates.

On the upside, there are some things that just worked - the JavaScript binding, debug highlights... pretty much anything that  didn't rely on the CryEngine 3 renderer :).

In conclusion, it worked pretty well, although if Crytek gave us access to the rendering device we could have been much more efficient in the integration, but then again, we used the free version so that's what we get. I was thinking of ways to get the device, like scanning the memory around gEnv->pRenderer for IUnknowns (by memcmping the first 3 virtual table entries) and then querying the interface for a D3D device, or just making a proxy DLL that exports the same functions as d3d9/11.dll and installing hooks on the relevant calls, but I don't have time for such fun now.

Now that we've seen how far can we go using the free CryEngine 3 SDK, next on the agenda is full Unity 3D integration (we have device access there!). Be on the lookout for it next month!

Debugging undebuggable applications with PIX

Developers can ask DirectX 9 not to allow PIX to debug their application by calling D3DPERF_SetOptions(1). I knew that and encountered several commercial applications using it. One day, I was fooling around with Portal 2 and wanted to feed my curiosity on how some stuff is done but when I started PIX all I got was “Direct3D Analysis Disabled” and I knew it was the time to find a way to circumvent this little peculiarity. So, let’s see how can we convince DirectX to ignore the request of the said developers.

I started with a simple application:


Let’s check out the disassembly of the D3DPERF_SetOptions(1) call:

_D3DPERF_SetOptions@4:
72EC7402 8B FF             mov     edi,edi  
72EC7404 55               push     ebp  
72EC7405 8B EC             mov     ebp,esp  
72EC7407 83 EC 18         sub     esp,18h  
72EC740A A1 50 92 FC 72   mov     eax,dword ptr [___security_cookie (72FC9250h)]  
72EC740F 33 C5             xor     eax,ebp  
72EC7411 89 45 FC         mov     dword ptr [ebp-4],eax  
72EC7414 A1 54 74 EC 72   mov     eax,dword ptr [string "DirectX Direct3D SO" (72EC7454h)]  
72EC7419 89 45 E8         mov     dword ptr [ebp-18h],eax  
72EC741C 8B 0D 58 74 EC 72 mov     ecx,dword ptr ds:[72EC7458h]  
72EC7422 89 4D EC         mov     dword ptr [ebp-14h],ecx  
72EC7425 8B 15 5C 74 EC 72 mov     edx,dword ptr ds:[72EC745Ch]  
72EC742B 89 55 F0         mov     dword ptr [ebp-10h],edx  
72EC742E A1 60 74 EC 72   mov     eax,dword ptr ds:[72EC7460h]  
72EC7433 89 45 F4         mov     dword ptr [ebp-0Ch],eax  
72EC7436 8B 0D 64 74 EC 72 mov     ecx,dword ptr ds:[72EC7464h]  
72EC743C 89 4D F8         mov     dword ptr [ebp-8],ecx  
72EC743F C6 45 ED 44       mov     byte ptr [ebp-13h],44h  
72EC7443 8B 4D FC         mov     ecx,dword ptr [ebp-4]  
72EC7446 33 CD             xor     ecx,ebp  
72EC7448 E8 D3 A1 F5 FF   call     @__security_check_cookie@4 (72E21620h)  
72EC744D 8B E5             mov     esp,ebp  
72EC744F 5D               pop     ebp  
72EC7450 C2 04 00         ret     4

Some movs, xors, runtime security check and that’s it, nothing with the actual value we passed to D3DPERF_SetOptions... well that was big nothing.

Ok, take two - let’s first start the application with PIX and then attach.
We’ll have to add some code to give us time to attach:


Use something like this, or just a Sleep() for enough time. Now what do we have with the new setup:

_D3DPERF_SetOptions@4:
72EC7402 E9 46 77 AA E8   jmp     HookedD3DPERF_SetOptions (5B96EB4Dh) 

All right, the sneaky PIX has modified d3d9.dll's memory and now it has a jmp in the beginning! The function it now executes takes us inside PIXHelper.dll:

HookedD3DPERF_SetOptions:
5BF6EB4D 8B FF             mov     edi,edi  
5BF6EB4F 55               push     ebp  
5BF6EB50 8B EC             mov     ebp,esp  
5BF6EB52 83 7D 08 01       cmp     dword ptr [ebp+8],1  
5BF6EB56 75 0B             jne     HookedD3DPERF_SetOptions+16h (5BF6EB63h)

We’ve only had one push so far (for the value we passed) and we push ebp, so that’s 2 pushes. After "mov ebp,esp" ebp is the same as the stack pointer, so dword ptr [ebp + 8] would be exactly the value we passed to D3DPERF_SetOptions. We compare that to 1 and if it’s equal some procedures are invoked that stop the execution. If it isn’t - we follow the jump specified by jne. What we have to do is make that jump unconditional - i.e. always execute the jump, regardless of the value passed. We don’t care about the code that pops the message for disabled analysis so we have plenty of bytes to play with; However, we don’t need them as we can see in “Intel® 64 and IA-32 Architectures Software Developer Manuals” - what we’re looking for is the EB cb variant of jmp, the exact same amount of bytes as the used jne instruction. Now all that’s left is open PIXHelper.dll with a hex editor (I used Notepad++ with hex-editor plugin), search for some of the bytes (try “8B EC 83 7D 08 01 75 0B” - I found it only once) and change the 75 to EB. Voila! Now you won’t see that annoying warning anymore.