This tutorial will take you through the process of altering the simple Hello Multitouch to feed events to Gestureworks from a Leap Sensor Device instead of the operating system. We will briefly cover the Leap listener class(CameraListener.cpp), how to create touchpoint data from Leap data, and how to pass touch events directly to Gestureworks. For this tutorial you will need the Gestureworks Core multitouch framework; a free trial is available.

Download the code for all of the C++ & openFrameworks multitouch tutorials here: tutorials_cpp_openframeworks.zip

Estimated time to complete: 30 minutes

• Successful completion of C++ & OpenFrameWorks: Getting Started II (Hello Multitouch)
• A copy of the C++ Leap Developer SDK (see Leap.com for download)
• Basic familiarity with threading

1. Setup
   • Creating the Leap Thread
2. Listening for Leap events
   • Passing Leap events to GestureWorks Core
3. The Update and Draw Functions

Like the code itself, setup for this project is nearly identical to that for the Hello Multitouch tutorial.

You'll need to create a Visual Studio project, link in openFrameworks, and the Gestureworks bindings as before. Following that, a little setup is needed for Leap. Download the C++ Leap Developer SDK (see Leap.com) and unpack it. You'll need to link the Leap.lib file and place the Leap.dll file right next to the resulting executable.

HelloSensor.cpp looks almost identical to Multitouch.cpp except for a couple Leap listening items. The Leap listening code is placed in CameraListener.cpp.

Add the following 3 lines to the Setup function. SetFullscreen is not necessary but was used to easily see all the active points. SetUsePixels is necessary for Leap as the normalized cordinates don't translate well on the screen. The most important line is to create the Leap Thread.

linenums:65 |//HelloSenser.cpp//

ofSetFullscreen(true);
 
GestureWorks::Instance()->SetUsePixels(true);
 
CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)&LEAP_thread, 0, 0, 0);

linenums:26 |//HelloSenser.cpp//

CameraListener leap_listener;
//Thread function necessary to continuously listen for LEAP events
DWORD LEAP_thread(LPVOID lpdwThreadParam) {
    Leap::Controller leap_controller(leap_listener);
    while(1) {}
    return 0;
}

Set up CameraLister to override Leap::Listener function onFrame.

linenums:17|//CameraListener.h//

#include "Leap.h"
#include "GestureWorks.h"
#include <deque>
#include <algorithm>
 
class CameraListener : public Leap::Listener{
public:
    virtual void onFrame(const Leap::Controller&);
};

onFrame gets called once per frame and analyzes each hand in the frame. Each hand can be broken up into individual fingers. Add each finger into an object that contains all fingers to easily loop through and send each finger point data into GestureWorks Core. Save off all added finger points to leap_ids to easily remove dead points in the future. After sending all active finger points to GestureWorks Core if there are any leap_ids that were not drawn erase them from the screen. If there are no fingers, remove all existing leap_ids to clear the screen.

linenums:17|//CameraListener.cpp//

#include "CameraListener.h"
 
#define LEAP_TIMER 100
#define LEAP_ZMAX 200
#define LEAP_ZMIN -20
 
#define LEAP_LEFT -200
#define LEAP_RIGHT 200
#define LEAP_UP 300
#define LEAP_DOWN 50
 
static std::deque<gwc::touchpoint> leap_queue;
static std::vector<long> leap_ids;
 
void CameraListener::onFrame(const Leap::Controller& controller){
 
    const Leap::Frame frame = controller.frame();
    const Leap::HandList hands = frame.hands();
    std::vector<int32_t> processed_ids;
    std::vector<Leap::Finger> all_fingers;
    if(!hands.isEmpty()){
        const Leap::Hand& hand = hands[0];
	for(int h = 0; h < hands.count(); h++){
            const Leap::FingerList fingers = hand.fingers();
            for(int f = 0; f < fingers.count(); f++){
                all_fingers.push_back(fingers[f]);
            }
        }
        for(int i = 0; i < all_fingers.size(); i++){
 
            gwc::touchpoint tp;
            double x, y, z;
            Leap::Vector pos;
            pos = all_fingers[i].tipPosition();
            x = pos[0];
            y = pos[1];
            z = pos[2];
 
            double adj_x = (x - LEAP_LEFT) / abs(LEAP_RIGHT - LEAP_LEFT);
            double adj_y = (LEAP_UP + LEAP_DOWN - y) / abs(LEAP_DOWN - LEAP_UP);
 
            long id = (long)all_fingers[i].id();
            tp.init(id, adj_x, adj_y, z, 1, 1);
 
            if( (z < LEAP_ZMIN) || (z > LEAP_ZMAX) ){
                continue;
            }
 
            if(std::find(leap_ids.begin(), leap_ids.end(), id) == leap_ids.end()){
                leap_ids.push_back(id);
                tp.status = gwc::TOUCHADDED;
	    }
	    else{
                tp.status = gwc::TOUCHUPDATE;
            }
	    GestureWorks::Instance()->AddEvent( tp );
        }
 
        for(int i = 0; i < leap_ids.size(); i++){
            if(std::find(processed_ids.begin(), processed_ids.end(), leap_ids[i]) == processed_ids.end()){
                gwc::touchpoint tp;
                tp.init(leap_ids[i], 0, 0, 0, 0, 0);
                tp.status = gwc::TOUCHREMOVED;
 
                GestureWorks::Instance()->AddEvent(tp);
                leap_ids.erase(leap_ids.begin() + i);
            }
        }
    }
 
    else{
        for(int i = 0; i < leap_ids.size(); i++){
            gwc::touchpoint tp;
            tp.init(leap_ids[i], 0, 0, 0, 0, 0);
            tp.status = gwc::TOUCHREMOVED;
            GestureWorks::Instance()->AddEvent(tp);
        }
        leap_ids.clear();
    }
}

The GestureWorks Core takes all the data pushed by the Cameralistener and will return the points with a call to ConsumePointEvents in a way that HelloSensor can draw them easily.

These two functions are identical to the ones found in HelloMultitouch.cpp.

Previous tutorial: C++ & openFrameworks: Interactive Bitmaps