Kinect Cloth Simulation

During my christmas holidays i had time to spare and some ideas to develop, so with all the party-feeling and booze i started working on the one i wanted the most. This is a cloth simulation running on the CPU which you can interact with using the kinect. With the 3d information from the kinect i’m able to change the cloth in some cool ways. The most interesting is plain and simple, offset each connection point by the depth at that point. With that done i had to go with the visual side (fun fun!), so i started playing with per-pixel lighting and some cool patterns.

Then added self-shadowing to the mixture and some color control just for the fun and looks of it.

In the end i thought it would be nice to take my carpet out to clean. Happy holidays!

CLNUI 4 Java (Kinect)

Kinect is a new product from Microsoft for XBox360. With an RGB camera, a 3D depth sensor (structured IR light) and  a multi-array microphone , it is able to compute a depth map, something very interesting for the interactive world of creativity and gaming. Nothing new as technology, but still, interesting and cheap. Since its release people have been trying to work in order to hack it and get it to work with windows and mac OS. Quite a few people have been working on drivers and a way to hack the gadget to get access to the color/depth data. One of these people is called Alex from Code Laboratories, the man behind the PS3eye camera drivers for PC’s. Recently, he released a new package (C and C#) that includes drivers and SDK for Kinect. I took the SDK and with  JNA i wrote a library for java that wraps the original one and also tries to make its use easy for starters.

A very early release is available http://victamin.googlecode.com/files/CLNUI4J_01.zip. This might or might not work out-of-the-box and it sure needs improvement and more work, but as they say, release early, release often. This package includes the library and basic examples running on Eclipse IDE and processing IDE (untested). It is known that this library does not work directly with Processing.

EDIT! Version 0.2 is now available at: http://victamin.googlecode.com/files/CLNUI4J_02.zip.
You can now find a friendly wrapper and a couple of examples to get you started.

I would love to get some feedback, so if you’re a Kinect owner and you’re into java/processing, give it a  spin and let me know how it works out for you.

Have fun.

GML Viscosity

So here it is, the application and source code for GML Viscosity.

I finally took the time to clean up this project and make a public release. In the packages you will find binaries and source code for the application, so if you’re thinking of using it and come up with something nice, let me know. After the first release i have fixed some problems with memory access and crashes. I have also added a simple UI for easy pick of tags from the “data/tags” folder on your hard-drive.

What is it?
– GML Viscosity
is an experiment. It is application used to draw GML tags on a viscous liquid rendered purely on the GPU side.

What does it do?
– It liquifies your tags, in a way you can hardly read them but still looks cool (thats graffiti).
– It is possible to load gml files from the disk or directly from the “lots of zeros”book database.

How can I interact?
– Not implemented on this version. This version selects and draw gml files randomly from the web or from a tags folder. For a smart person it should be easy to add mouse support. The code is also prepared to support multitouch so, it should be easy to implement TUIO and create your multitouch version.
– Use ‘r’ key to randomly fetch a new tag from the website and if available it will draw it on the screen.
– Use the GUI window to pick any tag from the folder “tags” inside data folder.

Where can i get it?
Windows Version | MacOSX Version

Have fun.

MSAOpenCL for Java and Processing

Recently I have been doing some work with OpenCL and i found myself messing up with old code and re-writing most of it .  At first it was interesting and quite funny but no more, so,  I thought it was time to make things easier for myself and just make things reusable by creating a wrapper which would wrap most of the boring and time-consuming parts. This would be a wrapper on OpenCL for Java and/or Processing. Well, a well known guy named Mehmet “Memo” Akten has done such a wrapper in C++ for the OpenFrameworks and Cinder libraries. I have ported it  to Java.

There are some problems with reading/writing from/to GL textures. For some reason it crashes on me. I have been working with JavaCL‘s author, so hopefully it will be working soon.

:: Download

Download it, install it and try it.
If you find any problems or if you have suggestions, let me know.

Have fun.

Evaluate a Cubic Bézier on GPU

I’ve made an application as an example for this thread on how to compute/evaluate a Cubic Bézier Curve using a Geometry Shader. The formula is pretty straightforward as described by this wikipedia article (look for Cubic Bézier Curve). I will not go over the bézier math or theory. I assume you have some knowledge in shader programming (GLSL is the case) and some math background would help, while not really a need. All that said, let’s get to work.

On this case we will need 4 points: 2 anchor points (the line end points) and 2 control points. The control points won’t really touch the curve, they work more as directional information on the curve itself.

As we need to send this data to the shader i have decided to use  LINES as input primitive, 2 points define a line so it’s perfect, we’ll use that for the 2 anchor points. As for the control points 2 different texture units (glMultiTexCoord3f) attached to the line’s vertex data will do. Using geometry shaders besides setting the input primitive type we also need to set the output type. LINE_STRIP is fine, as it works perfectly for what we’re doing. That’s all on the application side.

On the vertex shader side it’s pretty simple:

void main( void )
    gl_FrontColor = gl_Color;
    ControlPoint1 = gl_MultiTexCoord0.xyz;
    ControlPoint2 = gl_MultiTexCoord1.xyz;
    gl_Position = gl_Vertex;

What the code is doing is sending the data further down the pipeline to the geometry shader, where all the magic happens. At this stage, having both Anchor and Control points we can now define the curve by a given detail. Think of detail as a number of step-points along the curve which makes it look smoother or flatten (tesselation, subdivision, smoothing, etc).

Both control points are sent by the application for the geometry shader, still, as in the the vertex shader comes before the geometry shader, we will need to send them down on the vertex-shader, otherwise the GS won’t be able to “see” them (i know, hurray for Cg). We’re now almost done. By using the function from the above link(s) and as shown below we compute the curve with a given detail on the geometry shader, by generating new vertices along the curve purely on the gpu side. I think it to be pretty straightforward and the code should be self-explanatory.

uniform int g_Detail;
varying in vec3 ControlPoint1[];
varying in vec3 ControlPoint2[];
// Found in nvidia sdk
vec3 evaluateBezierPosition( vec3 v[4], float t )
    vec3 p;
    float OneMinusT = 1.0 - t;
    float b0 = OneMinusT*OneMinusT*OneMinusT;
    float b1 = 3.0*t*OneMinusT*OneMinusT;
    float b2 = 3.0*t*t*OneMinusT;
    float b3 = t*t*t;
    return b0*v[0] + b1*v[1] + b2*v[2] + b3*v[3];
void main()
    vec3 pos[4];
    pos[0] = gl_PositionIn[0].xyz;
    pos[1] = ControlPoint1[0];
    pos[2] = ControlPoint2[0];
    pos[3] = gl_PositionIn[1].xyz;
    float OneOverDetail = 1.0 / float(g_Detail-1.0);
    for( int i=0; i<g_Detail; i++ )
        float t = i * OneOverDetail;
        vec3 p = evaluateBezierPosition( pos, t );
        gl_FrontColor = gl_FrontColorIn[0]; 
        gl_Position = gl_ModelViewProjectionMatrix * vec4( p.xyz, 1.0 );

What’s next? That is up to you. You’re not going to leave me with all the work, are you ?

Download the example + source.
You will also need to install Vitamin 0.5.6 as the project is built with it.