In this tutorial you will create a multitouch application that will visually display touch points and their position information in real-time. This tutorial introduces the concept of touch point events, how to consume touch point event data obtained from Gestureworks, and an example of acting on that data. For this tutorial you will need the Gestureworks Core multitouch framework; a free trial is available.

Download the code for all of the Python & Panda3D multitouch tutorials here: tutorials_python_panda3d.zip

Estimated time to complete: 30 minutes

In the first tutorial you were introduced to Gestureworks Core, the concept of “bindings,” and you created an empty gwc_python and Panda3D project in Eclipse. This tutorial assumes you have completed the steps found in Python & Panda3D: Getting Started I (Hello World), and that you have a fully configured Eclipse project ready for use. If you have have not yet done so, please follow Python & Panda3D: Getting Started I (Hello World) to prepare an Eclipse project that includes a reference to the gwc_python module before proce
eding.

1. Import required modules
2. Set screen dimensions
3. Create Panda3D app
4. Initialize GestureWorks
5. Schedule GestureWorks frame processing
6. Create a method to process and track touch points
7. Create a method to clear the screen
8. Create a method to draw touchpoint data
9. Add code to build method to register our touch window with GestureWorks

First, we will need to create a main file for our project. Add a blank Python module file to your project and call it main.py. Your Project should now look something like this:

Next, drag and drop ring_black.png from the tutorial files into your Project workspace:

We'll start by importing the GestureWorksCore class and an enumerated status value for touch points.

from gwc_python.core import GestureWorksCore
from gwc_python.GWCUtils import TOUCHREMOVED

Next, let's import everything we'll need from Panda3D:

from direct.gui.OnscreenImage import OnscreenImage
from direct.gui.OnscreenText import OnscreenText
from direct.showbase.ShowBase import ShowBase
from direct.task import Task
from pandac.PandaModules import LineSegs
from pandac.PandaModules import deg2Rad
from pandac.PandaModules import NodePath
from pandac.PandaModules import Vec3
from panda3d.core import TransparencyAttrib
from panda3d.core import TextNode

Now we're ready to create our Panda3D app.

We want to set parameters for our screen dimensions (for screen height and width)

SCREEN_WIDTH = 1920
SCREEN_HEIGHT = 1080

We will create a class for our application derived from the Panda3D Showbase class. For now, we will just create a basic constructor that takes an instance of the GestureWorksCore class we imported. We’ll also create the build method.

class MultitouchApp(ShowBase):
 
   def __init__(self, gw):
       self.gw = gw
       self.active_points = {}
       self.touch_images = {}
       self.touch_text = {}
 
       ShowBase.__init__(self)
       self.build()
 
   def build(self):
       pass

The constructor just keeps an internal reference to the GestureWorksCore object, and initializes some empty dictionaries that we’ll use to store active touch point data and corresponding images and text. It then calls the ShowBase constructor.

Now that we have a class for our Panda3D app, we need to initialize it with a GestureWorksCore instance and get it running. Add the following code to the bottom of your main.py file:

if __name__ == '__main__':
   gw = GestureWorksCore('C:\\path\\to\\GestureworksCore\\GestureworksCore32.dll')
   if not gw.loaded_dll:
       print 'Unable to load GestureWorksCore'
       exit()
   gw.initializeGestureWorks(SCREEN_WIDTH, SCREEN_HEIGHT)
   app = MultitouchApp(gw)
   app.run()

Please be sure to change the path passed to the GestureWorksCore constructor to match the path on your system to the correct DLL. If GestureWorksCore was able to load the correct DLL, we simply initialize the core with the size of our touch display, instantiate our Panda3D app and run it.

We need to add one important line to our MultitouchApp constructor to enable Gestureworks to process frames at a scheduled interval. Just after you call the build method, add the following line:

self.taskMgr.add(self.updateGestureworks, "updateGestureworksTask")

This tells the application that we want to call the method updateGestureWorks once per frame or every 1/60 seconds. We don’t have a callback method in our MultitouchApp called updateGestureWorks yet so let's create it now.

   def updateGestureworks(self, task):            
       self.gw.processFrame()
       point_events = gw.consumePointEvents()
       self.processTouchEvents(point_events)
       self.clearScreen();
       self.drawTouchPoints()
       return Task.cont

Here we tell Gestureworks to process one frame of data for us. We then consume any point events that occurred during that frame. If there were any, we’ll receive a list of PointEvent objects as defined in the GWCUtils file in the bindings. We then process the events we consumed, clear the screen, and draw desired data to the screen. We’ll create the processTouchEvents, clearScreen and drawTouchPoints methods in the following steps.

This method will also belong to the MultitouchApp class and will take as a parameter the list of touch events we received from our call to consumeTouchEvents:

   def processTouchEvents(self, touches):
       for touch in touches:
           if touch.status != TOUCHREMOVED:
               self.active_points.update({touch.point_id: touch})
           else:
               self.active_points.pop(touch.point_id)

The method simply iterates over the events in the list and checks their status. If the event does not correspond to the removal of a touch point, we update our entry in our internal dict to reference the updated value. If the event is of type TOUCHREMOVED, we remove that touch point from our dict.

This method will clear the screen of all images and text. After calling this method, we will call the method that will draw images and text to the screen.

   def clearScreen(self):
       for single_image in self.touch_images:
           self.touch_images[single_image].destroy()
       for single_text in self.touch_text:
           self.touch_text[single_text].destroy()

Our app now needs a method to draw the data we are tracking. We’ll call the method drawTouchPoints and iterate over all of the active points we are tracking.

   def drawTouchPoints(self):
       self.win
       for touch in self.active_points.values():

Since Panda3D and Gestureworks don't use the same coordinate system for tracking data, we'll need to translate the x-coordinate and y-coordinate of all of our touch points.

touch_x = float((touch.position.x - SCREEN_WIDTH/2) / SCREEN_WIDTH) * 4
touch_y = float((SCREEN_HEIGHT/2 - touch.position.y) / SCREEN_HEIGHT) * 2

Now we can draw the data on the screen:

if touch.status != TOUCHREMOVED:
    # Draw circles
    self.touch_images[touch.point_id] = OnscreenImage('ring_black.png', pos=(touch_x,0,touch_y), scale=.05)
    self.touch_images[touch.point_id].setTransparency(TransparencyAttrib.MAlpha)
 
    # Draw the touchpoint info
    label = 'ID: %d\nX: %.3f | Y: %.3f' %(touch.point_id,touch.position.x,touch.position.y)
    self.touch_text[touch.point_id] = OnscreenText(label,pos=(touch_x+0.1,touch_y), scale=.05,align=TextNode.ALeft)

First, we use an image to denote where the touchpoint is. Then we create the text to display information about the touch point, namely its ID and x and y position. This is drawn to the right of the touch point.

There is just one more small but important task we must complete. We need to register our application window by name with Gestureworks. We can do this when we call the build method. Update the build method to look like this:

   def build(self):
       if not self.gw.registerWindow('Panda'):
           print('Unable to register touch window')
           exit()

When running the application, you should see something similar to the image at the top of this page when you interact with the application window.

NOTE: Since we do not convert Panda3D window coordinates to global coordinates, touch points will only appear correctly if the application is maximized or if the upper left corner of the window is on the upper left corner of your touch screen.

In this tutorial we loaded and initialized Gestureworks and displayed information about touch events we received from Gestureworks. The loading and initialization of the Core will be performed in a similar fashion in every application so you can use the code from this tutorial as boiler plate code to get a quick start for future applications.

In the next tutorial we will introduce GML and gesture events. We will also discuss the concepts of touch objects and the methods we use to associate touch points with them.

Next tutorial: Python & Panda3D: Interactive Bitmaps
Previous tutorial: Python & Panda3D: Getting Started I (Hello World)