Interactive Bitmaps

Javascript and Fabric

Introduction

In this tutorial you will build on what you’ve learned in the first tutorial, working with gestures to create an image on screen that can be dragged, rotated, and scaled using multi-finger gestures. You will learn how to register touch objects with Gestureworks, register to receive gesture events, and use these events to manipulate objects on screen. For this tutorial you will need the Gestureworks Core multitouch framework; a free trial is available.

You may download Gestureworks Websocket here: gw_websocket_server.zip

A working copy of this tutorial can be downloaded here: gw_websocket_client.zip

Image
Images-screenshot.png

Requirements
A running instance of the Gestureworks Websocket Server (see Websocket: Server Setup)
Multitouch device
Fabric.js
Supported browser (built and tested with Google Chrome)
Process Overview
Getting Started with Fabric.js
Initializing Gestureworks
Registering Touches
Responding to Gestures
Manipulating the Image
Hit Testing
Process Detail
1. Getting Started with Fabric.js

Instead of using the canvas API directly, we will be using the Fabric.js library to simplify application logic. To get started, here's the boilerplate code:

<!DOCTYPE html>
GestureWorks Core Demo
When ran, this code will draw an image in the top left corner, redrawing the image with the window.requestAnimationFrame function.

2. Initializing Gestureworks

Gestureworks needs to be defined and initialized, and just like the first tutorial, we will want a global variable var gw; for Gestureworks, a var startTime global variable, and a variable to hold the name of a touch object var touchObjId=“touchObj0”;. Inside the window.onload callback, we will need to add the following:

gw = new GestureWorks(“ws:127.0.0.1:4000/”);
gw.onSocketOpen = function (e) {
console.log(“gw socket opened.”);
startTime = new Date().getTime();
gw.registerTouchObject(touchObjId);
};
gw.onSocketError = function (e) {
console.log(“gw socket error: ” + e);
};
gw.onSocketClose = function (e) {
console.log(“gw socket closed.”);
};
gw.connect();
We will also want to handle the browser window changing sizes. Inside the onSocketOpen callback, lets add an event handler for browser resizing events:

window.addEventListener(“resize”, resize);
resize();
The resize function can be defined as follows:

function resize() {
if (gw.IsConnected()) {
gw.resizeScreen(window.innerWidth, window.innerHeight);
}
canvas.width = window.innerWidth;
canvas.height = window.innerHeight;
context.setWidth(window.innerWidth);
context.setHeight(window.innerHeight);
context.calcOffset();
};
3. Registering touches

Now that Gestureworks is being initialized we can add touch input registration. in the Gestureworks onSocketOpen callback, we will need to add touch event listeners to intercept the browsers touch events:

document.addEventListener('touchstart', touchHandler, false);
document.addEventListener('touchmove', touchHandler, false);
document.addEventListener('touchend', touchHandler, false);
The touchHandler can be defined as follows:

function touchHandler(ev) {
if (gw == null || !gw.IsConnected()) { return; }
ev.preventDefault(); prevent the browser from doing its default actions with touch-gestures and prevent mouse events from firing.

      var eType = null;
      if (ev.type === 'touchmove') {
              eType = GestureWorks.TouchStatus.TOUCH_UPDATE;
      } else if (ev.type === 'touchstart') {
              eType = GestureWorks.TouchStatus.TOUCH_ADDED;
      } else if (ev.type === 'touchend') {
              eType = GestureWorks.TouchStatus.TOUCH_REMOVED;
      } else {
              //touch cancel or unknown...
      }

if (eType != null) {

              var currentTime = new Date().getTime() - startTime;
              var touches = ev.changedTouches;
              var touch;
              var i;
              //iterate through each touch and update gestureworks.
              for (i = 0; i < touches.length; i++) {
                      touch = touches[i];
                      gw.addEvent(touch.identifier, touch.clientX / window.innerWidth, touch.clientY / window.innerHeight, eType, currentTime, touch.radiusX, touch.radiusY);
              }
      }

};

4. Gestures

To enable gestures, we need to tell Gestureworks that our touch object has gestures enabled, for this example we will be using n-drag, n-scale, and n-rotate. In the onSocketOpen callback, we need to add the following:

gw.addGesture(touchObjId, “n-drag”);
gw.addGesture(touchObjId, “n-scale”);
gw.addGesture(touchObjId, “n-rotate”);
The addGesture API call tells Gestureworks that our touchObject should receive gesture events for those three gestures. For this to work however, we will need to assign the touch points to the touch object. This can be accomplished by adding some logic to the touchHandler function directly after the for loop:

if (eType === GestureWorks.TouchStatus.TOUCH_ADDED) {

      gw.forceUpdate();
      for (i = 0; i < touches.length; i++) {
              touch = touches[i];
              gw.addTouchPoint(touchObjId, touch.identifier);
      }

}
5. Manipulating the Image

The three gesture events (n-drag, n-scale, n-rotate) supply several parameters that we will apply to the touch object. Lets create an object to store all the values that our gestures can change:

var xform = { x: 0, y: 0, sx: 1, sy: 1, r: 0 };
Gestureworks provides an API callback for getting gesture events titled onConsumeGestureEvents. In the window.onloaded function, lets set the callback:

gw.onConsumeGestureEvents = consumeGestureEvents;
The consumeGestureEvents can be defined as follows:

function consumeGestureEvents(gestures) {

      var gesture;
      for (var i = 0; i < gestures.length; i++) {
              gesture = gestures[i];
              if (gesture.GestureID === "n-drag") {
                      xform.x += gesture.Values.drag_dx;
                      xform.y += gesture.Values.drag_dy;
              } else if (gesture.GestureID === "n-scale") {
                      xform.sx += gesture.Values.scale_dsx;
                      xform.sy += gesture.Values.scale_dsy;
              } else if (gesture.GestureID === "n-rotate") {
                      xform.r += gesture.Values.rotate_dtheta;
              }
      }

};
We can use the data to affect the transform of the image by adding the following to the drawCanvas function (before the call to context.renderAll):

interactive.set({

      left: xform.x,
      top: xform.y,
      angle: xform.r,
      scaleX: xform.sx,
      scaleY: xform.sy

});
6. Hit Testing

Currently, we are assigning touch points to the touch object regardless of the position of the touch point. Although the gesture recognition will still function, the experience would be improved by applying hit testing to the touch object. To keep the logic simple, we will not be using perfect hit testing. Instead, we will test the bounding box of the touch object as provided by fabric.js. The hit test function can be defined as follows:

if (isPointNear(touch.clientX, touch.clientY, xform, 400)) {

      gw.addTouchPoint(touchObjId, touch.identifier);

}
We can call this function from inside the touchHandler by replacing the call to gw.addTouchPoint with the following:

if (isPointNear(touch.clientX, touch.clientY, xform, 400)) {

      gw.addTouchPoint(touchObjId, touch.identifier);

}
Now, you should have a working demo of Gestureworks that utilizes gesture processing to make an image interactive.

« Previous Tutorial

User Tools

Site Tools

Sidebar

Interactive Bitmaps

Introduction

Page Tools