In this tutorial you will create a multitouch application that will visually display touch points and their position information in real-time. This tutorial introduces the concept of touch point events, how to consume touch point event data obtained from GestureWorks, and an example of acting on that data. For this tutorial you will need the Gestureworks Core multitouch framework; a free trial is available.

Download the code for all of the Python & Kivy multitouch tutorials here: tutorials_python_kivy.zip

Estimated time to complete: 30 minutes

In the first tutorial you were introduced to Gestureworks Core, the concept of “bindings,” and you created an empty gwc_python and Kivy project in Eclipse. This tutorial assumes you have completed the steps found in Python: Getting Started I (Hello World), and that you have a fully configured Eclipse project ready for use. If you have have not yet done so, please follow Python: Getting Started I (Hello World) to prepare an Eclipse project that includes a reference to the gwc_python module before proceeding.

1. Create a basic Kivy Language file
2. Import required modules
3. Create Kivy app
4. Initialize GestureWorks
5. Schedule GestureWorks frame processing
6. Create a method to track and process touch points
7. Create a method to draw touch point data
8. Register the touch window with GestureWorks

We will need a very basic Kivy Language file for this project. Right click on your project and select New → File. Name the file “miltitouch.kv” and add the following content to it:

    #:kivy 1.5.0
    #:import kivy kivy
    #:import win kivy.core.window
 
    FloatLayout:
        canvas:
            Color:
                rgb: 0, 0, 0

First, we will need to create a main file for our project. Add a blank Python module file to your project and call it “main.py”. Your Project should now look something like this:

We'll start by importing the GestureWorksCore class and an enumerated status value for touch points.

from gwc_python.core import GestureWorksCore
from gwc_python.GWCUtils import TOUCHREMOVED

Next, let's import everything we’ll need from Kivy:

from kivy.app import App
from kivy.clock import Clock
from kivy.utils import get_color_from_hex
from kivy.graphics import Color, Ellipse, Line
from kivy.core.window import Window
from kivy.uix.label import Label

Now we’re ready to create a Kivy app.

We will create a class for our application derived from the Kivy App class. For now, we will just create a basic constructor that takes an instance of the GestureWorksCore class we imported. We’ll also need to override the build method.

class MultitouchApp(App):
    def __init__(self, gw):
        self.gw = gw
        Window.clearcolor = (.64, .64, .64, 1)
        self.active_points = {}
 
        super(MultitouchApp, self).__init__()
 
    def build(self):
        pass

The constructor just keeps an internal reference to the GestureWorksCore object, sets the window’s background color and initializes an empty dict that we’ll use to store active touch point data. It then calls its parent constructor.

Now that we have a class for our Kivy app, we need to initialize it with a GestureWorksCore instance and get it running. Add the following code to the bottom of your main.py file:

if __name__ == '__main__':
    gw = GestureWorksCore('C:\\path\\to\\GestureworksCore\\GestureWorksCore32.dll')
    if not gw.loaded_dll: 
        print 'Unable to load GestureWorksCore'
        exit()
    gw.initializeGestureWorks(1920, 1080)
    app = MultitouchApp(gw)
    app.run()

Please be sure to change the path passed to the GestureWorksCore constructor to match the path on your system to the correct DLL.

If GestureWorksCore was able to load the correct DLL, we simply initialize the core with the size of our touch display, instantiate our Kivy app and run it.

We need to add one important line to our MultitouchApp constructor to enable Gestureworks to process frames at a scheduled interval. Just before you call the constructor for the app’s parent, add the following line:

Clock.schedule_interval(self.updateGestureWorks, 1.0/60)
This tells the application that we want to call the method updateGestureWorks at a rate of 60hz or every 1/60 seconds.

We don’t have a callback method in our MultitouchApp called updateGestureWorks yet so let's create it now.

    def updateGestureWorks(self, *args):
        self.root.canvas.clear()
        self.gw.processFrame()
        point_events = gw.consumePointEvents()
        self.processTouchEvents(point_events)
        self.drawTouchPoints()

Here we clear the drawing area and tell Gestureworks to process one frame of data for us. We then consume any point events that occurred during that frame. If there were any, we’ll receive a list of PointEvent objects as defined in the GWCUtils file in the bindings. We then process the events we consumed and draw desired data to the screen. We’ll create the processTouchEvents and drawTouchPoints methods in the following steps.

This method will also belong to the MultitouchApp class and will take as a parameter the list of touch events we received from our call to consumeTouchEvents:

    def processTouchEvents(self, touches):
        for touch in touches:
            if touch.status != TOUCHREMOVED:
                self.active_points.update({touch.point_id: touch})
            else:
                self.active_points.pop(touch.point_id)

The method simply iterates over the events in the list and checks their status. If the event does not correspond to the removal of a touch point, we update our entry in our internal dict to reference the updated value. If the event is of type TOUCHREMOVED, we remove that touch point from our dict.

Our app now needs a method to draw the data we are tracking. We’ll call the method drawTouchPoints and iterate over all of the active points we are tracking.

    def drawTouchPoints(self):
        for touch in self.active_points.values():

Since Kivy and Gestureworks don’t use the same coordinate system for tracking data, we’ll need to translate the y-coordinate of all of our touch points.

            touch_y = int(self.root.height - touch.position.y)
            touch_x = int(touch.position.x)

Now we can draw the data on the screen.

            with self.root.canvas:
                Color(*get_color_from_hex('ffe354'))
                Ellipse(pos=(touch_x - 20, touch_y + 20), size=(40,40))
                Line(circle=(touch_x, touch_y + 40, 30, 0, 360), width=2)
 
                label = Label(text='ID: {}\nX: {} | Y: {}'.format(touch.point_id, touch_x, touch_y))
                label.center_x = touch_x + 80
                label.center_y = touch_y + 80
                label.color = (1,1,1)

First, we draw the touch point as two concentric circles, only filling the inner circle. We do this by drawing an ellipse whose major and minor axes are the same. The position we specify for the ellipse will be the lower left corner of its bounding box so we need to shift it left and up by an amount equal to its radius.

Next we just create a label to display information about the touch point, namely its ID and x and y position. This is drawn above and to the right of the touch point.

There is just one more small but important task we must complete. We need to register our application window by name with Gestureworks. We can do this when Kivy builds our application class. Update the build method to look like this:

    def build(self):
        if not self.gw.registerWindow('Kivy'):
            print('Unable to register touch window')
            exit()

When running the application, you should see something similar to the image at the top of this page when you interact with the application window.

NOTE: Since we do not convert Kivy window coordinates to global coordinates, touch points will only appear correctly if the application is maximized or if the upper left corner of the window is on the upper left corner of your touch screen.

In this tutorial we loaded and initialized GestureWorks and displayed information about touch events we received from GestureWorks. The loading and initialization of the core will be performed in a similar fashion in every application so you can use the code from this tutorial as boiler plate code to get a quick start for future applications.

In the next tutorial we will introduce GML and gesture events. We will also discuss the concepts of touch objects and the methods we use to associate touch points with them.

Next tutorial: Python & Kivy: Interactive Bitmaps
Previous tutorial: Python & Kivy: Getting Started I (Hello World)