Getting Started II (Hello Multitouch)

In this tutorial you will create a multitouch application that will visually display touch points and their position information in real-time. This tutorial introduces the concept of touch point events, how to consume touch point event data obtained from GestureWorks, and an example of acting on that data. For this tutorial you will need the Gestureworks Core multitouch framework; a free trial is available.

Download the code for all of the Java & Java 2D multitouch tutorials here: tutorials_java_java2d.zip

In the first tutorial you were introduced to GestureWorks Core, the concept of “bindings,” and you created an empty Java Application Project in NetBeans. This tutorial assumes you have completed the steps found in Java: Getting Started I (Hello World), and that you have a fully configured NetBeans project ready for use. If you have have not yet done so, please follow Java: Getting Started I (Hello World) to prepare a NetBeans project that includes a reference to the gwc_java project before proceeding.

1. Create a custom multitouch JFrame
2. Initialize GestureWorks
3. Add abstract methods for event processing
4. Schedule GestureWorks frame processing
5. Verify GestureWorks connection
6. Create touch point graphics
7. Create multitouch frame subclass
8. Manage touch point graphics
9. Compile and run

Open the Netbeans project you created in Java Tutorial #1: Setting Up Your IDE, and add a new Java class by right clicking on the packageA package→New→Java Class.

Name the class TouchWindow and click Finish.

Extend the javax.swing.JFrame object and create a constructor with a String parameter to specify the window’s name.

  public abstract class TouchWindow extends javax.swing.JFrame {  
      public TouchWindow(String name)
      {
          super(name);
      }
  }

Set the window’s dimensions and set its visibility to true in the constructor.

  int width = 1920;
  int height = 1080;
  setSize(width, height);    
  setVisible(true);

First add an instance variable initialized with a new global GestureWorks object and import the necessary Java bindings.

  import gwc.Core;
  import gwc.GWCUtils.GestureEvent;
  import gwc.GWCUtils.PointEvent;
  ...
  protected Core gwCore;

Now perform the actual GestureWorks object initialization

Add the following to the constructor method to specify the GestureWorksDLL and GML paths, initialize the dimensions to match your window’s, and registering the window with the DLL

  gwCore = new Core("C:\\path\\to\\GestureWorksCore.dll");
  gwCore.initializeGestureWorks(width, height);
  gwCore.loadGML("resources\\basic_manipulation.gml");
  gwCore.registerWindowForTouchByName(name);

These four method calls are all that are required to initialize the GestureWorks object.

Note that there is an alternate window registration method, registerWindowForTouch(), that takes a native window handle as a parameter. The example above may also be used to register the window with GestureWorks, however, it is recommended that the window handle be used whenever possible.

Add the following methods to process point events and gesture events to the TouchWindow class. The proceeding subclasses will implement these methods as needed.

  public abstract void processPointEvents(ArrayList<PointEvent> events);
  public abstract void processGestureEvents(ArrayList<GestureEvent> events);

We need to create a method to enable GestureWorks to process frames at a scheduled interval.

Add the following method to the TouchWindow and call the method in the last line of the constructor.

  private void initializeTimer()
  {
      Timer timer = new Timer();
      timer.schedule(new UpdateGW(), 0, 16);
  }

This tells the application that we want to call the TimerTask UpdateGW every 16 milliseconds.

Now we need to add the TimerTask to process the frames.

  private class UpdateGW extends TimerTask
  {
      @Override
      public void run() {
          gwCore.processFrame();
          processPointEvents(gwCore.consumePointEvents());
          processGestureEvents(gwCore.consumeGestureEvents());
      }
  }

processFrame() tells GestureWorks to evaluate the touch point and gesture event data it has received from the operating system, performing internal pipeline processing such as point, cluster, and gesture updating as well as temporal analysis. Typically, processFrame()should be called on each frame update such as within an application’s update or draw loop; we’ll act on the updated data in the next step.

gwc_java provides GestureWorks touch point and gesture event data via its consumePointEvents() and consumeGestureEvents() methods (consumeGestureEvents() is discussed further in Java Tutorial #3: Interactive Bitmap). A “touch point” in the context of GestureWorks is a location on screen representing a single point which a user has touched, and is represented by the PointEvent object. Whenever GestureWorks receives information from the operating system that a touch point has been received, it begins tracking it as a PointEvent. PointEvents are obtained from GestureWorks Core via the ConsumePointEvents() method.

If an invalid DLL path is passed, the application will throw an UnsatisfiedLinkError. To further evaluate the success/failure of the GestureWorks initialization, assign the specified calls in the constructor to associated boolean values and print the results.

  boolean registered = gwCore.registerWindowByName(name);
  boolean gmlLoaded = gwCore.loadGML(“basic_manipulation”);
  
  System.out.println(“Window: +”name”\n\tregistered: 
  “+registered+”\n\tgmlLoaded: “+gmlLoaded);

To test the touch window, instantiate a TouchWindow in your main class (GestureWorksTutorial1) and implement the abstract methods as shown below.

  TouchWindow window = new TouchWindow("Hello World") {
  
      @Override
      public void processPointEvents(ArrayList<PointEvent> events) {
          if(!events.isEmpty())
              System.out.println(events.size());
      }
  
      @Override
      public void processGestureEvents(ArrayList<GestureEvent> events) {}
  
  };

This will output the number of active point events on our TouchWindow object.

We will ignore handling gesture events in this tutorial.

Compile and run the project.

A window named Hello World will display along with the following output:

  Window : Hello World
      registered: true
      gmlLoaded: true
    

Touching the window with various points should output the number of touch points currently interacting with the window.

In this tutorial we are going to track point events by updating a collection of PointEvent objects based on their added, updated, or removed status, then drawing active points on the screen.

First, create a new package by right clicking on Source Packages→New→Java Package. Name the package packageB. In this package, create a new Java Class, name it TouchPoint, extend java.awt.Component, and configure it based on the provided PointEvent.

  public class TouchPoint extends Component{      
      public TouchPoint(PointEvent event, Insets insets)
      {
          setSize(80, 80);
          windowInsets = insets; 
          x = event.position.x - windowInsets.left;
          y = event.position.y - windowInsets.top;
          id = event.point_id;
          outerCircle = new Ellipse2D.Double(0, 0, 60, 60);
          innerCircle = new Ellipse2D.Double(10, 10, 40, 40);
      }
  }

Override the paint method and add the following implementation to draw the graphic

  if(dispose)
      g.dispose();
  
  //adjust for window insets
  int touchX = (int)x - windowInsets.left;
  int touchY = (int)y - windowInsets.top;
  
  Graphics2D g2d = (Graphics2D)g;
  g2d.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON);
  g2d.setStroke(new BasicStroke(2));
  g2d.setColor(new Color(0x008cbf));
  g2d.translate(touchX-20, touchY-20);  //center the graphic
  g2d.draw(outerCircle);
  g2d.fill(innerCircle);
  g2d.drawString("ID: "+Integer.toString(id), 80, 0);
  g2d.drawString("X:"+Integer.toString(touchX)+"  Y:"+Integer.toString(touchY), 80, 15);

This code will produce the touch point graphic from the screen-shot in the introduction. It is positioned at the specified x and y coordinates of the associated PointEvent.

The last thing we need to add to the TouchPoint, is a way to update and destroy the graphic. Add these two methods to the TouchPoint class.

  public void update(PointEvent event)
  {
      x = event.position.x;
      y = event.position.y;
      repaint();
  }
  
  public void remove()
  {
      dispose = true;
      repaint();
  }

Create a new TouchWindow subclass called TouchPointWindow

  public TouchPointWindow extends TouchWindow
  {
  
      public TouchPointWindow(String name)
      {
          super(name);
      }
  
      @Override
      public void processPointEvents(ArrayList<PointEvent> events) {
      }
  
      @Override
      public void processGestureEvents(ArrayList<GestureEvent> events) {
      }
  }

Next, create a global hash map to store the active TouchPoint objects by id.

  private LinkedHashMap<Integer, TouchPoint> activePoints = new LinkedHashMap<>();

Setup the processPointEvent method to loop through all of the PointEvent objects. Based on the evaluation of the PointEvent status, either add the associated TouchPoint to the map and the window, update its location, or remove it from the map and the window.

  @Override
  public void processPointEvents(ArrayList<PointEvent> events) {
      for(PointEvent event: events)
      {
          switch(event.status)
          {
              case Status.TOUCHADDED:
                  addTouchPoint(event);
                  break;
              case Status.TOUCHUPDATE:
                  updateTouchPoint(event);
                  break;
              case Status.TOUCHREMOVED:
                  removeTouchPoint(event);
                  break;
              default:
                  break;
          }
      }        
  }

Now implement the add, update, and remove operations for the TouchPoint objects.

  private void addTouchPoint(PointEvent event)
  {
      TouchPoint tp = new TouchPoint(event, this.getInsets());
      activePoints.put(event.point_id, tp);
      add(tp);
      this.validate();
  }
  
  private void updateTouchPoint(PointEvent event)
  {
      activePoints.get(event.point_id).update(event);
  }
  
  private void removeTouchPoint(PointEvent event)
  {
      if(!activePoints.containsKey(event.point_id))
          return;
  
      activePoints.get(event.point_id).remove();
      activePoints.remove(event.point_id);
  }

To test your new TouchPointWindow and TouchPoint graphics, replace the previous code in the main class with the following and hit F6 to run the project.

  TouchPointWindow window = new TouchPointWindow("Hello Multitouch");

The application should display a window titled “Hello Multitouch”. Touching the window will produce the touch point graphics at the contact locations along with associated point event ids and xy coordinates.

In this tutorial we initialized GestureWorks and displayed some graphics on-screen using touch event data obtained from GestureWorks. The initialization of GestureWorks will be performed similarly for every GestureWorks-based application that you create, as are the usage of the processFrame() and consumePointEvents() methods. In the next tutorial we will introduce gesture events which are obtained via the consumeGestureEvents() method, and will discuss the concepts of touch objects and associating touch points with touch objects.

Next tutorial: Java: Interactive Bitmaps
Previous tutorial: Java: Getting Started I (Hello World)