Gaming Your Way

May contain nuts.

Hello Pixel Bender

Played with Pixel Bender yet ? For Ionic ( My TD game which is taking an age to finish ) I wanted to use a similar effect to the transition in cronusX, the rgb split / tv interference thing. In the transition it didn't matter how long it took, but to use it in-game with God knows how many sprites along with collisions running it's got to be a lot lot quicker. Yeah you know where this is going.

ionic_rgbGrab.jpg

After firing up the Pixel Bender toolkit and swearing and looking through docs and examples for an hour or so, I kinda got it, so I thought I'd share what little I know. Just to warn you, this is going to be code heavy and no example to even look at, dry reading ahead. Maybe open up some porn in another tab and just flick between the two when your eyes start glazing over.

<languageVersion : 1.0;>

kernel RGBDistortion
< namespace : "RGB Distort";
vendor : "www.gamingyourway.com";
version : 1;
description : "Pixel Bender version of RGB distort";
>
{
  input image3 src;
  output pixel3 dst;

  parameter float rOffset
<
   minValue:float(-50.0);
   maxValue:float(50.0);
   defaultValue:float(0.0);
>;

  parameter float gOffset
<
   minValue:float(-50.0);
   maxValue:float(50.0);
   defaultValue:float(0.0);
>;

  parameter float bOffset
<
   minValue:float(-50.0);
   maxValue:float(50.0);
   defaultValue:float(0.0);
>;
  void

  evaluatePixel()
  {
    float2 outCoords=outCoord();

    float2 redPos=outCoords;
    redPos[0]+=rOffset;
    float3 inputColorR = sample(src,redPos);
    dst.r = inputColorR.r;

    float2 greenPos=outCoords;
    greenPos[0]+=gOffset;
    float3 inputColorG = sample(src,greenPos);
    dst.g = inputColorG.g;

    float2 bluePos=outCoords;
    bluePos[0]+=bOffset;
    float3 inputColorB = sample(src,bluePos);
    dst.b = inputColorB.b;
  }
}

I'll try and break it down quickly in my usual lazy not really explaining things style. At the start is simple metadata, nothing you can't figure out there.

   input image3 src;
   output pixel3 dst;


A bit more interesting. input is the image you're going to send to the kernal, image3 being the datatype ( Because we don't want to worry about the alpha channel the datatype is image3 ( RGB ), if you want alpha it would be image4 ( ARGB )), the output is the bitmap we're going to plot the result to, which is a pixel3 datatype ( Which is the same as image3 ).

  parameter float rOffset
<
   minValue:float(-50.0);
   maxValue:float(50.0);
   defaultValue:float(0.0);
>;

This how you set up an argument to pass to the kernal. It's a float and refers to the red channel offset when we jiggle things about. The min/max and default values are more for the toolkit itself. When you run it it'll give you a slider to play with, and those are the values for the slider ( It'll make sense when you copy the code into the toolkit. Also you can drop a description in there as more metadata, but this is for Flash so we don't really need it ).

   evaluatePixel()

This is our main loop. Everything in this method is run on each and every pixel in the image.

    float2 outCoords=outCoord();

The outCoord() method returns this pixels x/y as two floating point numbers, in the form of an array ( Kinda ). We just store that in a var out of habit and speed.

    float2 redPos=outCoords;
    redPos[0]+=rOffset;
    float3 inputColorR = sample(src,redPos);
    dst.r = inputColorR.r;

redPos is our x,y position. We then add the new x position to redPos.x ( redPos[0] ), with the new x position being our parameter from above.
inputColorR is quite a chunky little line. sample grabs the pixel data at ( inputSource, position ), in effect we're setting the float3 ( RGB ) variable inputColorR = sourceImage[x+our offset][y]
After doing that, we make this pixel in the r(ed channel ) in our destination this colour. Basically we're doing this:

var rgb=sourceBitmap.getPixel(x+offset,y);
destBitmap.setPixel(x,y,rgb);

( But just on the red channel ).

The rest of the kernal is just more of the same, just for the other two channels.

After that pick the Export Filter for Flash Player option and we're ready to get back to normal coding.

package Classes {
    import flash.display.Bitmap;
    import flash.display.BitmapData;
    import flash.display.DisplayObject;
    import flash.display.MovieClip;
    import flash.display.Shader;
    import flash.display.ShaderJob;
    import flash.display.ShaderParameter;
    import flash.display.Sprite;
    import flash.display.Stage;
    import flash.events.ShaderEvent;
    import flash.filters.ShaderFilter;
    
    public class Transition {
        
//---------------------------------------------------------------------------------------
// Assets
//---------------------------------------------------------------------------------------
        [Embed(source="_shaders/distortion.pbj", mimeType="application/octet-stream")]
        private var shader_distortPBJ:Class;

        [Embed("/_assets/assets.swf",symbol="staticMC")]
        private var staticMC:Class;

//---------------------------------------------------------------------------------------
// Properties
//---------------------------------------------------------------------------------------
        private var playField:Sprite;

        private var shaderBitmapData:BitmapData;
        private var grabBitmapData:BitmapData;
        private var grabBitmap:Bitmap;

        private var shader_distort:Shader;
        private var shaderFilter_distort:ShaderFilter;
        
        private var distortCnt:int;
        private var rSin:Number;

        private var staticEffect:Sprite;
        private var staticAnim:MovieClip;
                
//------------------------------------------------
// System
//------------------------------------------------
        private var main:Main;
        private var mainMovie:DisplayObject;
        private var stage:Stage;

//---------------------------------------------------------------------------------------
//Constructor
//---------------------------------------------------------------------------------------
        public function Transition(){
            main=Main.getInstance();
            mainMovie=main.getMainMovie();
            stage=main.getStage();

//Let's make our bitmaps where we're going to plot our data too
            shaderBitmapData=new BitmapData(700,380,false,0);
            
            grabBitmapData=new BitmapData(700,380,true,0);
            grabBitmap=new Bitmap(grabBitmapData);
            grabBitmap.alpha=0.4;
//Create our shader, sexy
            shader_distort=new Shader(new shader_distortPBJ());
            shader_distort.data.src.input=shaderBitmapData;

            staticEffect=new staticMC();
            staticAnim=staticEffect["anim"];
            staticAnim.gotoAndStop(1);
        }

//---------------------------------------------------------------------------------------
// Public
//---------------------------------------------------------------------------------------
        public function toString():String {
            return "Transition";
        }        

//---------------------------------------------------------------------------------------
        public function init():void{
            playField=main.getInitObj().getPlayfield().transitionPlayField;
            rSin=0;
        }
        
//---------------------------------------------------------------------------------------
        public function distortInit():void{
            if(grabBitmap.parent!=null){
                return;    
            }

            distortCnt=0;
            shaderBitmapData.draw(stage);
            playField.addChild(grabBitmap);

            staticEffect.height=int(Math.random()*100)+10;
            staticEffect.y=int(Math.random()*300)+50;
            staticAnim.gotoAndPlay(1);
            playField.addChild(staticEffect);
            
            startJob();
        }

//---------------------------------------------------------------------------------------
// Private
//---------------------------------------------------------------------------------------
        private function startJob():void{
            rSin++;
            rSin++;
            ShaderParameter(shader_distort.data.rOffset).value=[Math.sin(rSin)*12];
            ShaderParameter(shader_distort.data.gOffset).value=[Math.cos(rSin)*12];
            ShaderParameter(shader_distort.data.bOffset).value=[Math.sin(-rSin)*12];
            
            var job:ShaderJob = new ShaderJob(shader_distort,grabBitmapData);
            job.addEventListener(ShaderEvent.COMPLETE,shaderCompleted);
            job.start();
        }
        
//---------------------------------------------------------------------------------------
        private function shaderCompleted(e:ShaderEvent):void{
            if(++distortCnt==6){
                staticAnim.gotoAndStop(1);
                playField.removeChild(staticEffect);
                playField.removeChild(grabBitmap);
                return;
            }

            staticEffect.height=int(Math.random()*100)+10;
            staticEffect.y=int(Math.random()*300)+50;

            startJob();
        }

//---------------------------------------------------------------------------------------
    }
}

Sigh, that's a ton of boring code to look at, let's just pick out the fun stuff ( Equating as3 to fun, fuck me that's tragic ).

        [Embed(source="_shaders/distortion.pbj", mimeType="application/octet-stream")]
        private var shader_distortPBJ:Class;

That's how to embed the kernal.

//Let's make our bitmaps where we're going to plot our data too
            shaderBitmapData=new BitmapData(700,380,false,0);
            
            grabBitmapData=new BitmapData(700,380,true,0);
            grabBitmap=new Bitmap(grabBitmapData);
            grabBitmap.alpha=0.4;

What we're going to do is grab the screen when we want to run the effect, but we just want it faint ( alpha=0.4 ). shaderBitmapData is our source, with grabBitmapData being our dest. You can use the same bitmap for both, but Flash has to create a temp working copy, so it's slower.

//Create our shader, sexy
            shader_distort=new Shader(new shader_distortPBJ());
            shader_distort.data.src.input=shaderBitmapData;

Finally we're creating our actual shader, with the second line defining our source bitmap, which can do here as it won't change.

The distortInit() just grabs the screen, resets any vars we need and sends up the static effect ( Which has nothing to do with the pixel bender stuff, so we're going to ignore that ). Once that's all done we call the startJob() method which does the magic.

        private function startJob():void{
            rSin++;
            rSin++;
            ShaderParameter(shader_distort.data.rOffset).value=[Math.sin(rSin)*12];
            ShaderParameter(shader_distort.data.gOffset).value=[Math.cos(rSin)*12];
            ShaderParameter(shader_distort.data.bOffset).value=[Math.sin(-rSin)*12];
            
            var job:ShaderJob = new ShaderJob(shader_distort,grabBitmapData);
            job.addEventListener(ShaderEvent.COMPLETE,shaderCompleted);
            job.start();
        }

K, so we're using sin and cos to create our r,g and b position offsets, nothing new there. The ShaderParameter is how we change our parameter values in the kernal. You can hit the values directly, and they will be modified, but it doesn't actually do anything. I lost over an hour to that before finding out about ShaderParameter. Hopefully those lines should make sense.

Next up, we create a ShaderJob. We're doing this as opposed to the adding it as a filter approach, as this is the async magic that we've gone to all this effort for ( Runs on it's own thread etc. etc. Let's face it, that's the only thing we all know about PB ). The ShaderJob just wants to know which kernal we're using, and the destination bitmap.
Finally we just add a listener to that, and call the start() method, and it'll go and do it's thing and won't affect our other code. There's no waiting around here.

        private function shaderCompleted(e:ShaderEvent):void{
            if(++distortCnt==6){
                staticAnim.gotoAndStop(1);
                playField.removeChild(staticEffect);
                playField.removeChild(grabBitmap);
                return;
            }

            staticEffect.height=int(Math.random()*100)+10;
            staticEffect.y=int(Math.random()*300)+50;

            startJob();
        }

We're just been recursive bad boys here. Have we run the effect 6 times ? Yes so we're done, clear everything up, nope, so call the startJob() method again.

Hardly an indepth technical breakdown, but hopefully it's a starting point. This was a fair few hours of swearing for me, but I'm old so new things take longer to make sense, so it shouldn't take you more than a couple of hours to get PB doing cool things.

Squize.

Comments (2) -

  • Porter

    10/8/2009 2:30:06 AM |

    haha, I didn't read through all the code (the girlfriend wouldn't want me opening a tab for porn unless she was in the mood too), but I did get a good laugh at of your commentary that wasn't code. Swearing is an excellent way to relieve stress, I say "I'm fucking dumb" when I figure something out in programming all the time, it happens quite often. If I ever mess with PB, I'll definitely have to check this out.

  • Squize

    10/8/2009 11:42:51 AM |

    Thanks mate.

    It's quite hard finding a use for PB. It's like this great toy that can do so much, but then... what do I do with it ?

    I'd like to play with it for number crunching, but even then I'm at a loss. Perhaps you could run all your games Sin / Cos calls through it, but unless you were hitting that a lot or using a 3D engine, the setting up of the data would take as long as any speed advantage.

    Maybe using a triple buffer display for blitting and getting PB to clear one of those buffers for you ? As you can tell, I'm struggling to find a use for it, and it is so sexy, it's a real pity I'm not clever enough to come up with a good use.

Comments are closed