Update! The link has been removed as the application was removed from online. I will get a new link ASAP.

]]> http://www.pixelnerve.com/v/2010/11/04/dt/feed/ 1 http://www.pixelnerve.com/v/2010/09/07/gml-viscosity/ http://www.pixelnerve.com/v/2010/09/07/gml-viscosity/#comments Tue, 07 Sep 2010 07:32:25 +0000 victormartins http://www.pixelnerve.com/v/?p=1259 So here it is, the application and source code for GML Viscosity.

I finally took the time to clean up this project and make a public release. In the packages you will find binaries and source code for the application, so if you’re thinking of using it and come up with something nice, let me know. After the first release i have fixed some problems with memory access and crashes. I have also added a simple UI for easy pick of tags from the “data/tags” folder on your hard-drive.

What is it?
- GML Viscosity is an experiment. It is application used to draw GML tags on a viscous liquid rendered purely on the GPU side.

What does it do?
- It liquifies your tags, in a way you can hardly read them but still looks cool (thats graffiti).
- It is possible to load gml files from the disk or directly from the “lots of zeros”book database.

How can I interact?
- Not implemented on this version. This version selects and draw gml files randomly from the web or from a tags folder. For a smart person it should be easy to add mouse support. The code is also prepared to support multitouch so, it should be easy to implement TUIO and create your multitouch version.
- Use ‘r’ key to randomly fetch a new tag from the website and if available it will draw it on the screen.
- Use the GUI window to pick any tag from the folder “tags” inside data folder.

Where can i get it?
Windows Version | MacOSX Version

Have fun.

So imagine we have 2 image buffers Image1, Image2. Usually to change data from Image1, you would simply access it and write directly back to the same positioion. Now, when we’re talking about a shader fragment program we can’t simply do that. You may ask now, what’s the solution? Ping-pong it!

Here is what i’m talking about:

//
// Initialization
//
int W = 100;
int H = 100;
ImageArray = new int[2][W*H];
int CurrActiveBuffer = 0; // Current active buffer index
 
 
//
// Mainloop
//
for( int j=0; j<H; j++ )
{
	for( int i=0; i<W; i++ )
	{
		// ERROR! Write to same buffer. Not possible in gpu shader
		//Image1[i+j*W] = Image1[i+j*W] * 2;  // Mul by 2
 
		// Ping-pong version
		int src = CurrActiveBuffer; // Current active buffer (Input)
		int dest = 1-CurrActiveBuffer;  // Back buffer (Output)
		ImageArray[dest][i+j*W] = ImageArray[src][i+j*W] * 2;  // Mul by 2
	}
}
 
// Swap back and front buffers (read becomes write and vice-versa)
CurrActiveBuffer = 1-CurrActiveBuffer;    // CurrActiveBuffer ? 0 : 1;

As you can see we start with buffer 0. That’s where we get out data from (read) and write it to Buffer 1. Once the operation is done, we swap buffers every frame. This way we’ll be able to use last iteration’s data as input for the next iteration.

Now let’s put this in OpenGL language. I will be using a FrameBufferObject (FBO) and 2 textures here. The framebuffer will be holding to both textures as 2 Color Attachments. So let’s get coding:

//
// Initialization
//
int W = 100;
int H = 100;
int FboID;
int TexID[2];
int CurrActiveBuffer = 0;  // Current active buffer index
 
// Create the frambuffer
glGenFramebuffersEXT( 1, &FboID );
glBindFramebufferEXT( GL_FRAMEBUFFER_EXT, FboID );
 
// Create 2 textures for input/output.
glGenTextures( 2, TexID );
 
for( int i=0; i<2; i++ )
{
	glBindTexture( GL_TEXTURE_2D, TexID[i] );
	glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );
	glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );
	glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE );
	glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE );
        // RGBA8 buffer
	glTexImage2D( GL_TEXTURE_2D, 0, GL_RGBA, W, H, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL );
	if( _hasMipmapping )  glGenerateMipmapEXT( GL_TEXTURE_2D );
}
 
// Now attach textures to FBO
int src = CurrActiveBuffer;
int dest = 1-CurrActiveBuffer;
glFramebufferTexture2DEXT( GL_FRAMEBUFFER_EXT, 
                           GL_COLOR_ATTACHMENT0_EXT, 
                           GL_TEXTURE_2D, TexID[src], 0 );
glFramebufferTexture2DEXT( GL_FRAMEBUFFER_EXT, 
                           GL_COLOR_ATTACHMENT1_EXT, 
                           GL_TEXTURE_2D, TexID[dest], 0 );
);
 
 
 
//
// Mainloop
//
int src = CurrActiveBuffer;
int dest = 1-CurrActiveBuffer;
 
FBO.Bind();
 
glDrawBuffer( dest );
 
glBindTexture( GL_TEXTURE_2D, TexID[src] );
ShaderProgram.SetTextureUniform( 0 );
RenderScene();
 
FBO.Unbind();
 
 
// Swap back and front buffers (read becomes write and vice-versa)
CurrActiveBuffer = 1-CurrActiveBuffer;    // CurrActiveBuffer ? 0 : 1;

Why would you want to ping-pong? Well for instance imagine you are doing a water effect in the gpu. You’ll need to access data from your previous buffer right? Using the CPU that would be trivial as you have the last frame’s memory buffer allocated somewhere which you can read/write access directly. That doesn’t really work on the gpu side that way (but we’re getting closer with CL, CUDA, DC)

Have fun.

Next step was to optimize the cube generation by lowering from a 24 vertex cube to 14 vertex triangle strip. Things got better, but nothing close to my expectations. I was not satisfied, i mean, i had alot of cubes on screen (100K which was not that much) and that was it, nothing else. We’re talking about 20fps or so, for 100.000 cubes (around 1.2 million triangles per frame). Later on i added vertex normals to the geometry shader and started to work on some lighting/shadowing, but i ended up going back on the rendering side of the job. Meanwhile, i was speaking with a friend of mine about this idea and we were discussing ways to compute lighting but i couldn’t stop thinking about my real problem. So it came to me.

Previously, i have done some experiments with opengl hardware instancing, but never got  to do much about it .  What better time than now, so i grabbed the project and took it for a spin. After a few hours i had the same amount of cubes on screen with a much much better framerate. Quickly implemented some eye-candy (coloring, texturing, vertex lighting), some tweaking here and there and as i was listening to Mr. Peter Broderick (hi, i love you man) added some audio analysis to the feature list.
Last but not least, a kind of “Brownian Motion” was used to generate points in space, increased the cube count to 512*512 and watched it flow ( at 20fps ).

In conclusion, Hardware instancing was much easier to implement  and performance seems much better at first sight. Above is a video of 262.144 audio-reactive cubes with GPU animation and basic lighting at around 20fps. For my video card i think that is very good. On a sidenote, i have not given up on the geometry shaders. I am not sure what will be my next step regarding the subject (back to geometry shaders?) but for now this is it. Hardware Instancing kicked Geometry Shaders in the ass.

During my research i came across this Pirelenito’s website with a great example on using FOBS+JMF, exactly what i needed. So i  downloaded it and started working on it, putting into it a bit of myself.

So here it is, a first pre-beta version of a library capable of rendering movie files (with sound) with Java/Processing without much effort.

NOTE! I’m still having problems with a couple of codecs like DivX, Xvid, H264. It crashes on me when rendering some of these, but it works just fine for Mpeg4, PhotoJPEG, Cinepak. It isn’t perfect, but if you be careful and use the right format/codec it will go smooth. Meanwhile i will be working on a better version.

Website: www.pixelnerve.com/processing/libraries/moviegl
Library Package(s) at: :  http://code.google.com/p/victamin/downloads/list

Have fun!

Credits goto Pirelenito. Thanks for sharing.

PS. If you are a 2d artist and would love to contribute for this, you know where to find me.