Skip to content


In development: the TUIOPlaybook


As I said in the last post, I am currently building a small project to be published onto the BlackBerry Playbook. I am building it in FDT 5.5, using RobotLegs, AS3Signals and TUIO. Above is a short teaser of my work in progress.

Posted in ActionScript 3.0, Personal Works, Projects, Technologies.

Tagged with , , , , .


News and plans for spring and summer 2012

Sunny Cali

Summer is almost at our doors, on the calendar at least. It's been a while since I wrote, I have been busy: I've traveled, I've been coding, and I've been trying out a couple of different things. Let me bring you up to date.

Flash Gaming Summit 2012

Flash Gaming Summit 2012 Speakers

In February, GAMES@CODAME held a contest on their Google Plus page to win a ticket to the Flash Gaming Summit 2012 in March in San Francisco. So after a tight race, I ended up winning the ticket! As I said before, video games are something towards which I would like to move, and meeting game devs and hearing them speak of their craft could not have been a better blessing.

I managed to meet with animators, entrepreneurs, devs and all sorts of good people. Some talks were really good, while some others were basically just rants. I personally wished that some of them would have been more technical than business oriented. On that angle, Iain Lobb delivered.

Almost as a fanboy, I made sure that I would meet some public figures in the game devs business, so I chatted a bit with Thibault Imbert, Bruno Fonzi, Iain Lobb, and my compatriot Ryan Creighton.

Later in the day, Kongregate held a party at their offices, where I met other cool devs. When I spoke to Rob Bateman, who had earlier given a quite interesting speech in favor of open source, I had to find a way to brag about my Away3D tutorials. By the way Rob, I'm still interested in writing tutorials for Away3D!

The next day, I was also lucky enough to be invited to visit a game dev studio not far outside of San Francisco. It's quite refreshing to see a team of devs and artists really work together, I hadn't felt a relationship this tight when I was working in an advertising agency.

Working with Float4

Float4's Slideshow

As you might have understood from some of my tweets and posts, I have been working with Float4 in Montréal since January. They specialize in video mapping and interactive surfaces, both with HD resolution or higher. It's quite nice to be able to use high quality graphics rather than being restrained to the small dimensions of the web.

Float4 offers a software, RealMotion, that has many purposes, one of which is to display Flash content. So I mostly have been working in ActionScript 3 for their projects. As Stage3D came to be, we did some tests to see if it would be possible to use within the context of RealMotion. To explain shortly, RealMotion does not incorporate the Flash Player directly. An instance of the Flash Player is launched on a thread, and RealMotion reads the visual output created and displays it on a texture. With Stage3D, that became impossible, since the render is actually done on the GPU directly, and there is no other way to get the render but to try and intercept the visuals directly from the graphic card. When I passed by the Flash Gaming Summit, I asked Thibaut Imbert if he had an idea of how to do this, he referred to PIX, a GPU profiler, but it didn't turn out to be of use on a Windows machine. If you have any suggestion, it would be welcome!

Discovering how to handle multitouch was quite new to me, as I was not used to the possibility that many UI elements could be used at the same time on the same screen. Also, I have been writing for installations where neither a mouse nor a touch screen was used, but rather lasers. In this setup, lasers are scanning a surface in order to determine where the user or users touch.

It may seem like it would not affect how to code, but it's actually quite important! While on a desktop and on a mobile surface it's important to listen to the "up" event (mouse up, touch up), it's not quite logical to do so with lasers, since users would not expect to lift their hand to trigger an action. Therefore, a touch down event acts as a click, and triggers something right away.

In the case of a list–like the iPhone contact list–it's not logical to open the contact as soon as it's touched, since it may be required to scroll. That's why on these device, it's the up event that triggers an action. With lasers, I thought it would be best to introduce the logic of holding the position for a while to launch an action. A timer would start and once a certain amount of time is passed, the action is triggered.

RealMotion communicates with Flash via the TUIO protocol, and its AS3 library, TUIO AS3. The library is open source and quite easy to integrate. Rather than using mouse events, the library comes with its own custom TUIO touch events. What is also quite nice is the guy who manages the library is quite open, listens to comments and adapts accordingly.

But how does one test multitouch during development? It does take time to deploy to the installation every time, and let's be honest, it's not always available to the developer. Luckily, it's possible to install an iPhone app and an Android app that can turn your device into a multitouch controller for your project. Simply start up the app before you launch your AS3 project, and then they are connect via TCP.

As for projects, I have been working on an update and a localization of a project for the Canadian government that needed to be displayed in Davos. I hope to obtain some images and footage to add to my portfolio. I also have been working on a slideshow that showcases Float4's portfolio with different AS3 APIs (Vimeo, Flickr, Twitter). You can see a screenshot of that project above this section, I will soon add screenshots to my portfolio. I also have been working on a library that can communicate with RealMotion, so that Float4 can provide clients who want to develop their own content with an API.

An iPhone app on its way

QRC

For some reason, I decided to go back to try and learn Objective-C. Since there is no better way to learn something than having an actual project, I will soon release my first iOS app! I finally understood how to link visuals to the code, and how to add and remove event handlers. Despite a lot of idiosyncrasies, XCode is a good IDE for the job. The code completion is marvellous, the interface is Apple-like (ie: fine tuned to the bone), but some features for quick coding and ease are missing. I may try and work more in that environment in the future.

A new toy

BlackBerry Playbook

During March, BlackBerry had a 50% off deal on their PlayBook tablet. I never really felt the need to own a tablet, and I thought the iPad was way too big and expensive. After attending the Flash Gaming Summit though, I thought that tablets could be an awesome target for games, and that there is no better way to understand how a device works than own and use one.

To my surprise, BlackBerry offers an awesome AIR SDK to develop for the PlayBook with their own UI. Smart! I had some issues while trying to run it on FDT, but a recent post on the BlackBerry forums brings some good news for the future.

I think that I may try to create a TUIO controller app for the PlayBook as my first attempt!

Going back to hacking

Circuit Bending Workshop

In the spirit of exploring many things digital and analog, I decided to go back to hacking electronics, as I have done for school projects before.

First, I will attend a circuit bending workshop given at the Eastern Bloc in Montréal, and see where that takes me. I hope I will play with my Arduino again. I've also been quite interested into looking into the Raspberry Pi, a more than affordable open source computer.

It may get cloudy

Cloudy

Finally, I thought I should leave you with another tease. I've been doodling and thinking of this little game where a cloud has to water plants and make sure to avoid the sun to not evaporate. The full gameplay is not yet defined, and as you can see I am tackling more than one project at a time. I think this project could be a good excuse to look more into haXe and NME.

What have you been up to?

Posted in Commercial Works, Conferences, Personal Works, Projects.


How to fully fit an Away3D plane in the viewport

Escher - Relativity (Detail)

There has been a lot of people who played with Away3D and built awesome things. A lot of said awesome things were built by people who know, or know someone that knows 3D modeling, which is why the quality has been so awesome. However, it may not always be necessary to build a full 3D world, sometimes a project may simply require some 3D elements over a 2D background. But how do you place an image behind all sorts of 3D objects if Away3D's View3D has a black background? I will try and explain the process in this article. Warning: matrices and trigonometry ahead!

Build it and they will come

Usually, if you want to place an image and make it fit your document dimensions, you simply stretch the image accordingly. In 3D however, it is not as simple, as the world dimensions are not necessarily measured in pixels. While working with the guys from Float4, I came to understand that meters are oftentimes used as measurement unit. With that in mind, we cannot directly use the dimensions of our document.

So let's say we want to place an image in the 3D world and make it fill the viewport (the visual space taken by Away3D). At first, we would need an image, and a plane onto which we would like to place the image. As I wrote previously, materials applied onto shapes need to be in dimensions that are a power of 2, so will will need to rescale our image. A Matrix will fulfill this need.

private const WIDTH:uint = 640;
private const HEIGHT:uint = 640;
private const MATERIAL_WIDTH:uint = 1024;
private const MATERIAL_HEIGHT:uint = 1024;
 
[Embed(source="assets/images/escher-relativity.png")]
private const EscherImageClass:Class;
 
private var _imageData:BitmapData;
private var _matrix:Matrix;
private var _material:BitmapMaterial;
private var _plane:Plane;
private var _view3D:View3D;

WIDTH and HEIGHT represent the dimensions of my document, and also the dimensions of the image. The MATERIAL_WIDTH and MATERIAL_HEIGHT represent the dimensions needed for the material, dimensions that are a power of two. Here I will be using an image from Escher, obtained from Wikipedia. There is no need for a Bitmap per se, but we will need the image's BitmapData, hence the _imageData variable.

Let's do this!

Normally, I resize the material dimensions to a value higher than what I need to display; first the material is stretched up, and then visually fitted to the original dimensions of the image. If we were to scale the material to smaller dimensions than that of the image, the plane would stretch up the material afterwards when displaying the image, resulting in possible loss of smoothness or detail.

_matrix = new Matrix();
_matrix.scale(MATERIAL_WIDTH / WIDTH, MATERIAL_HEIGHT / HEIGHT);

Let's draw an instance of the Escher image into the_imageData. This is where _matrix comes in handy. I wonder what pushed Adobe's engineers to make an architectural choice where they thought users would use a ColorTransform, a BlendMode or a clipping Rectangle more often than a flag to smooth the drawing. I mean, putting the flag as a last argument? Anywho, let's add a lot of null parameters...

_imageData = new BitmapData(MATERIAL_WIDTH, MATERIAL_HEIGHT);
_imageData.draw(new EscherImageClass(), _matrix, null, null, null, true);

Next, create the material. As seen previously, let's use a simple trick to assign whether or not the material is mipmapped.

_material = new BitmapMaterial(_imageData, true);
_material.mipmap = (_imageData.width == _imageData.height);

Apply the material to the plane, add the plane to the View3D, and then add the view to the display list.

_plane = new Plane(_material);
_view3D = new View3D();
_view3D.scene.addChild(_plane);
addChild(_view3D);

Do not forget to listen to Event.ENTER_FRAME and call the View3D.render() method at every update.

addEventListener(Event.ENTER_FRAME, updateHandler);

Run it, and what do we get?...

Nothing.

Uh?

By default, the plane is added at (0, 0, 0), so it should be visible...

It is, it turns out that an API choice—which may make sense in a 3D world, but that I do not understand—puts the plane with its face up on instantiation. What this means is when it is created, we see its side, which has no thickness. Just change the yUp parameter to false, so the new plane will then face the camera upon creation.

_plane.yUp = false;

We should get something like this.

Plane is too close, image is cropped by viewport

If we compare the image and this result, we notice that the image's edges are cropped, which means the plane is too close. Below is a diagram explaining this visually.

Diagram - too big, outside of viewport

Imagine a pyramid drawn from the camera at its top with an angle that is the camera's field of view, up until a certain point. From the camera's point of view, there is always a certain frame that is visible, that is the viewport. However, that pyramid is not infinite, actually what the camera sees is not even a pyramid. The viewing frustum, what is visible within a 3D world, is an area from a certain threshold (the near plane) to another one (the far plane), that's what is visible within a 3D world. What happens with the code above is that the plane is bigger than the frustum, thus being cut.

Below is what we want to achieve:

Diagram - Plane fits into the viewport

SOH-CAH-TOA

This is where the fun begins! How do we fit our plane into our viewport? It is possible to simply move the plane on the z axis and eventually position the plane properly to fit our need. But what if the dimensions change? We'd be back onto square one. This problem requires good ol' trigonometry, and the pythagorean theorem.

You may wonder what this title means. It's a mnemonic trick that's been given to me back in high school on how to remember the rules of trigonometry:

Sine of an angle = Opposite side / Hypothenuse

Cosine of an angle = Adjacent side / Hypothenuse

Tangent of an angle = Opposite side / Adjacent side

Then let's consider this as a trigonometry problem, illustrated below. We have a camera, the Camera3D, which is located at (0, 0, -500) by default. Look at the sources to confirm, it is the case with Away3D, but it may be different in other 3D engines. We have a plane with a position yet undefined, but we do want to make sure that its fits perfectly into the viewport.

Diagram - trigonometry and pythagorean theorem

Where does that 60° angle in the illustration above come from? If we take a look at the Camera3D constructor, we find that it uses a PerspectiveLens by default. In that lens constructor, we can see the default field of view is 60, which is in degrees. It's too bad we need to look at the source to obtain this, it would have been simpler to use a getter to be able to check it, but there is none.

Good, we have an angle, we can calculate dimensions and positions. Let's set an arbitrary z for the plane, we'll see further that it doesn't matter.

var angleY:Number = 60;
var cameraZ:int = _view3D.camera.z;
var planeZ:int = 500;
var distFromCamToPlane:Number = Math.abs(cameraZ) + planeZ;

Since the field of view is vertical, let's measure the height of our plane first. Let's not forget that the field of view covers the whole height, but in order to work with rectangle triangles, we will use only half of that angle. We know the angle, we know the distance from the camera to the plane (the adjacent side), but we do not know the opposite side, so let's use the tangent. The Math.tan() takes radians, so just make sure to convert those degrees to radians.

var planeHeight:int = Math.tan((Math.PI / 180)(angleY * 0.5)) * distFromCamToPlane;

Since this represents only half of the height, let's correct it for what we need further.

planeHeight *= 2;

To obtain the width of our plane, simply apply the aspect ratio of the image. In this case the image is a square, but it may very well be any kind of rectangle.

var aspectRatio:Number = WIDTH / HEIGHT;
var planeWidth:int = planeHeight * aspectRatio;

And now we are ready to create our plane with the proper dimensions and position.

_plane = new Plane(_material, planeWidth, planeHeight);
_plane.yUp = false;
_plane.z = planeZ;

Let's take a look at the whole process at once.

// get the scale to apply to the image
_matrix = new Matrix();
_matrix.scale(MATERIAL_WIDTH / WIDTH, MATERIAL_HEIGHT / HEIGHT);
 
// create the image data
_imageData = new BitmapData(MATERIAL_WIDTH, MATERIAL_HEIGHT);
_imageData.draw(new EscherImageClass(), _matrix, null, null, null, true);
 
// create the material
_material = new BitmapMaterial(_imageData, true);
_material.mipmap = (_imageData.width == _imageData.height);
 
// create the view
_view3D = new View3D();
 
// camera values
// see away3d.camerasCamera3D's default lens' field of view
// in away3d.cameras.lenses.PerspectiveLens
// it turns out that the field of view is vertical
var angleY:Number = 60;
var cameraZ:int = _view3D.camera.z;
 
// this position is arbitrary as we will see further
// useful to calculate the distance between the camera and the plane
var planeZ:int = 500;
var distFromCamToPlane:Number = Math.abs(cameraZ) + planeZ;
 
// since the field of view is a vertical angle, calculate the height of the plane first
// use pythagorean theorem to calculate
// tip for trigonometry: soh-cah-toa
// toa: tan(angle) = oppositeSide / adjacentSide
var planeHeight:int = Math.tan((Math.PI / 180)(angleY * 0.5)) * distFromCamToPlane;
 
// and since it was for a rectangle triangle, thus half the size, double the length
planeHeight *= 2;
 
// useful for resizing the image
var aspectRatio:Number = WIDTH / HEIGHT;
 
// use the aspect ratio to calulcate the width of the plane
var planeWidth:int = planeHeight * aspectRatio;
 
// create the plane
_plane = new Plane(_material, planeWidth, planeHeight);
_plane.yUp = false;
_plane.z = planeZ;
 
// add 3D objects to the scene
_view3D.scene.addChild(_plane);
 
// add the away3d view to the display list
addChild(_view3D);
 
// listen to the updates
addEventListener(Event.ENTER_FRAME, updateHandler);

This should produce a result like this, where the image fits perfectly in the viewport.

Plane fits viewport perfectly

For additional fun, I added an option to click to rotate it, so that it demonstrates that it is indeed 3D.

Sources

As I always do in my tutorials, you can download the sources. This project has been built for Flash Player 11+ with FDT. You may need to adapt the Flex SDK.

Enjoy!

Posted in Tips, Tutorials.

Tagged with , , .