31 May 2008

Table of contents

• Introduction
• 3D in documents
• Quartz Composer rocks!
• Step-by-step : create an application that can display a 3D model without doing anything itself
• Remaining problems
• Conclusion

Introduction

MacOS X is a great platform. There are many reasons to say that, but one of them is that it provides technologies that allow developers to create very rich applications. And rich application allow me to work more efficiently. An example of such an application is OmniGraffle; another example is Keynote. And a last example is combining both of them.

Omnigraffle allows me to create illustrations, and Keynote allows me to use them in presentations. What I like is that drag'n drop works, transparency works, LinkBack works. Modifying an image may or may not use CoreImage, as a user I do not care, but the result is fast and beautiful (and as a developer I enjoy CoreImage). Step-by-step, MacOS X demonstrates what technology and integration mean.

That's enough praising, let's work a little

3D in documents

As I said, I am a Keynote user. I am a scientist and I often have to present data to the audience. What I miss, for now, is the ability to present 3D data from Keynote. For instance, let's consider some surface :

To make it clear to the audience what the shape is, I need to rotate it. If I use a screen capture in my Keynote presentation, I cannot do that, I have to switch to Grapher. I could use an animation, but I really lack control with it. The idea is :
I would like to import a 3D model in Keynote and be able to rotate it as if it was in Grapher.

I do not want to argue on how many solutions and standards can already be used or developed to do that. But I want to show how useful Quartz Composer could be to bring a MacOS X-wide reusable tool to embedd 3D in documents.

Quartz Composer rocks!

It seems to me that Quartz Composer is actually not well known, or at least not understood enough to be used as much as it could. At first sight, Quartz Composer looks like a laboratory to create beautiful screensavers. Graphically, you can chain components to display images or animations, and you can make a ninja jump over an RSS feed. :-) Much ado for nothing ?

First, what you should not miss is that a Quartz Composition can be saved and used offscreen, as a box with inputs and outputs. Thus, you can save many lines of code by creating a workflow in QuartzComposer and use it in your applications. I have done that successfully in my app MozoDojo. For each image of an image bank, I have to perform some resize/cropping/color space change/histogram analysis, and it is all delegated to an offscreen QCRenderer. Less code, less bugs.

But more important to my eyes : QuartzComposer acts like a bridge between MacOS technologies. It is so easy and magical to drop, move and link boxes in Quartz Composer, that one often forgets how powerful is under the hood. OpenGL can map a cube with a lazy-evaluated CoreImage result as texture, while playing the sound of a QuickTime sequence retreived through http. Yes, Objective-C is great.

Now, let's back to my initial request : I want 3D in Keynote. The good news is that Keynote has a preliminary support for Quartz Compositions (you will see that some problems remain). So, the first step has been done. As long as an application can display a Quartz Composition (that translates as can use a QCView), then the Quartz Composition can display a 3D model represented as OpenGL commands, and rotate on my mouse demand. Magical. But still needs some work.

Step-by-step : create an application that can display a 3D model without doing anything itself

From now, I will present the different steps that drove me to build an application able to display a 3D model stored in a file. What is beautiful is that the application itself has almost no code. It supports drag'n drop from a file, creates QCView subviews, and redefine some mouse events. All the difficult part is in reusable frameworks or quartz plugins.

Step 1 : What is a 3D model ? I want it in OpenGL !

A 3D model should be here a file, written in a standard format, describing a scene with objects, meshes, textures... Fortunately, such files exist. 3D Studio has created .3ds, and a common other format is the OBJ format. The difficulties are to read such a file, and convert it to a set of OpenGL commands (glVertex(), glTranslate()...). I really do not want to do that myself. So , a little Googling gave an answer TooL : The OpenGL-OBJect loader. This is a C++ library that can display OBJ files in a built-in OpenGL viewer.

TooL does not build that easy, it has many dependancies (SDL, boost, python, Jam...) and I do not want to use the built-in Viewer. So, my first work was to strip that distribution and to create a framework that would full fill my needs. My framework is called ObjectLoader and here is its only public header :

//ObjectLoader/ObjectLoader.h
#include 

#ifdef __cplusplus
extern "C" {
#endif

typedef void ObjMesh;

extern ObjMesh* meshCreateFromFile(const char* UTF8Path); //creates an opaque Mesh Object from a file
extern void meshDraw(CGLContextObj cgl_ctx, ObjMesh* cmesh); //draw it in the given OpenGL context
extern void meshRelease(ObjMesh** mesh);//releases the mesh

#ifdef __cplusplus
}
#endif
    

As you can see, I really do not need more for now. You can download ObjectLoader. Since it is not a real project, and is in no way made to be distributed or used, it has absolutely no license, no guaranty, and does not event respect the GPL license of TooL, since I had to cut parts and modify its code to make it compile.

Step 2 : Can I use ObjectLoader from within a QuartzComposer Patch ?

Step 3 : Use it in an application

Now that I have some ready-to-use Quartz compositions that can load and display a 3D model, it is time to use it in a real application. I have called it QC3DApp. The application's document contains a single view. If you drop a .qtz file in it, it creates a QCView subview with the given composition. And if you drop an OBJ file here, it creates a QCView subview loading QCPlugin.qtz, using the OBJ file path as input. And here it is : 3D in your document. With almost no code, since QCPlugin and ObjectLoader are doing all the work. You can download QC3DApp. You will find obj files inside.

Remaining problems

There are still problems to solve to apply that to Keynote.

• Load textures. I did not investigate that much, but textures and ligthing are required for a correct render. ObjectLoader currently fails with textures, it seems.
• Transparency ! I do not know how to make the background of the composition transparent.
• Embedding in Keynote. Currently, Keynote embeds QTZ compositions in a QuickTime media, so it does not respond to mouse events.
• Grapher does not export data as an OBJ model ! But the idea is here.

Conclusion