Interactive Clock

In this lesson, we will be building on the interactive bitmap tutorial by creating a multitouch clock. We will learn how to make following items touchable in the scene:

• the bells will be touchable to create a ring sound;
• the minute hand may be adjusted with a two finger rotate;
• the entire clock will rotate, scale, and drag with the same gestures as the interactive bitmaps tutorial;
• the scene camera may be dragged and zoomed.

Download the code for all of the C# & Unity multitouch tutorials here: gestureworks-unity on GitHub

Here is a snapshot of the finished clock tutorial in game mode:

Estimated time to complete: 45 minutes

This tutorial builds upon the last where we touch enabled the Gestureworks logo. Making use of gestures is where the real power of the Core lies. If you have been following along from the first tutorial, you should have your project setup to recognize gestures.

• To successfully complete this tutorial, you will need to download and install the required items outlined in the first tutorial and start with the resources from the previous tutorial.
• The Clock.fbx will need to be copied from the tutorial assets folder inside your project’s Assets\Models folder.
• The Editor and Sounds folder will need to be copied to the Assets folder.
• A basic understanding of Object Oriented Programming is assumed.

1. Configuring the project
2. Positioning and configuring the clock
3. Adding colliders
4. Bell script
5. Clock script
6. Minute hand script
7. Second hand script (optional)
8. Camera script
9. GestureWorks configuration and main
10. Review
11. Continuing education

Duplicate the previous tutorial’s files as a starting point for this one. Update the player’s title (Edit→Project Settings→Player→Product Name) to InteractiveClock.

Make empty object in your scene hierarchy that is called Main and create a script called Main in C# script and attach it. Build and run to make sure the project is configured correctly. If all works as last time, please proceed on to setting up the GestureWorks logo.

Delete Bitmap1 and Bitmap2 from the Hierarchy panel. In the Project window expand the Models folder and delete the Unity texture that is inside the Textures folder. Delete the TouchImage script in the MyScripts folder.

Next click and drag the Clock.fbx file from the lesson start folder into your project’s Models folder. Then click and drag the Clock from the project folder into the Hierarchy folder.

Click and drag the Editor folder and Sounds folder to your Assets folder.

And finally, click and drag the my_gestures.gml file from the MyScripts folder to your Assets/MyScripts folder.

In the hierarchy panel, click on the Clock object and make sure the transform position and rotation axis (X, Y, Z) are set to 0. The Scale values should be set to 1.

The Main Camera object will need to be adjusted to bring the clock into focus. Be sure to set your camera values to the following settings as indicated below:

NOTE: be sure to make your field of view and clipping planes the same.

Go to game view (CTRL + 2) and make sure the camera is lined up to show the clock in full view.

Next, we will move our directional light and enable shadows. Make sure your directional light has the same settings as shown below:

NOTE: The tutorial uses a little bit of yellow in the directional light.

You may optionally add a spotlight with the following settings:

In the Materials panel, change the shader type to Specular for the following materials:

• Brass
• Brass1
• Gold

The ClockFace material should be changed to Self-Illumin/Bumped Diffuse with a light gray color (this lightens up the face a little).

Finally, click and drag the bell sound to the left and right bell mesh in the Hierarchy panel. Increase the minimum distance of the audio clip to 2. Uncheck Play on Awake if you don’t want to hear the bell every time the scene starts.

Now that you have configured the clock, camera, and lights, it is now time to add colliders to the scene. Like the previous tutorial, we utilize this special Unity feature for each 3D object so that we may detect touch intersections.

Select the object in the hierarchy view that you wish to add a collider to and select a box, sphere, or capsule collider in the Component > Physics menu.

The following is a sequence of settings for each clock part and collider (the green wireframe visually indicates size and placement of the collider).

Create a new C# script called TouchBell. Be sure to inherit from TouchObject like the interactive bitmap example. The final script for TouchBell is as follows:

linenums:1 |TouchBell // //

using UnityEngine;
using System.Collections;
using GestureWorksCoreNET;
using GestureWorksCoreNET.Unity;
 
public class TouchBell : TouchObject {
 
    public void Tap(GestureEvent gEvent){
        this.audio.Play();
    }
}

Click and drag TouchBell to the Bell_L and Bell_R object in the Hierarchy panel. Configure the attached script for each bell to support 1 gesture and type in Tap for Element 0. The Tap gesture ID is defined in the my_gestures.gml file and this method will get called when built.

Now is a good time to test out what you have done so far to ensure that the bells ring and that there are no further errors. If you don’t hear anything, make sure you copied the latest gml file into your MyScripts folder. Also be sure your audio connection is working correctly. And lastly, check the output log file to see if there were any errors loading files.

We will now add the clock script. Make a new C# script in the MyScripts folder and name it TouchClock. Put the following code in the file:

linenums:1 |TouchClock // //

using UnityEngine;
using System.Collections;
using GestureWorksCoreNET;
using GestureWorksCoreNET.Unity;
 
public class TouchClock : TouchObject {
 
protected GameObject AffineTransform;
    protected readonly string AffineTransformName = "GW Transform Context";
    protected Transform OriginalParentTransform;
 
    /// <summary>
    /// Starts the transform context. Call this before any value changes occur 
    /// if you want manipulations to start where the center of the gesture happens. 
    /// Important: you must use AffineTransform in the GestureUpdate as the point 
    /// of manipulation. 
    /// </summary>
    public void StartAffineTransform(Vector3 contextLocation){
        if (AffineTransform == null) {
            AffineTransform = new GameObject();
                AffineTransform.name = AffineTransformName;
                    AffineTransform.transform.position = contextLocation;   
        }       
            AffineTransform.transform.LookAt(Vector3.forward);
            OriginalParentTransform = this.transform.parent;
            this.transform.parent = null;
            AffineTransform.transform.parent = OriginalParentTransform;
            this.transform.parent = AffineTransform.transform;
 
    }
 
    /// <summary>
    /// Call this at the end of the contextual gesture manipulation.
    /// </summary>
    public void EndAffineTransform(){
 
        this.transform.parent = OriginalParentTransform;                
        AffineTransform.transform.parent = null;        
        Destroy(AffineTransform);               
    }
 
 
    public void NDrag(GestureEvent gEvent){
 
        MoveObjectInCameraPlane(gEvent);
 
        float cx = Mathf.Clamp(transform.position.x, -2.0f, 2.0f);
        float cy = Mathf.Clamp(transform.position.y, 0.0f, 2.0f);
        float cz = 0.0f;
 
        transform.position = new Vector3(cx, cy, cz);
    }
 
    public void NRotate(GestureEvent gEvent){
 
        float multiplier = 0.75f;
 
        Camera cam = Camera.main;
 
        float dTheta = gEvent.Values["rotate_dtheta"]*multiplier;
 
        float screenX = gEvent.X;
        float screenY = cam.GetScreenHeight() - gEvent.Y;
        float screenZ = cam.WorldToScreenPoint(this.transform.position).z;
 
        Vector3 touchLocation = cam.ScreenToWorldPoint(new Vector3(screenX, screenY, screenZ));
 
        StartAffineTransform(touchLocation);
 
            AffineTransform.transform.Rotate(0, 0, dTheta);
 
        EndAffineTransform();
 
    }
 
    public void NScale(GestureEvent gEvent){
 
        float multiplier = 0.5f;
 
        float scaleDX = gEvent.Values["scale_dsx"]*multiplier;
        float scaleDY = gEvent.Values["scale_dsy"]*multiplier;
 
        float cx = Mathf.Clamp(transform.localScale.x+scaleDX, 0.5f, 2f);
        float cy = Mathf.Clamp(transform.localScale.y+scaleDY, 0.5f, 2f);
        float cz = Mathf.Clamp(transform.localScale.z+scaleDY, 0.5f, 2f);
 
        transform.localScale = new Vector3(cx, cy, cz);
    }
 
    public void ThreeFingerTilt(GestureEvent gEvent){
 
        float multiplier = 1.0f;
 
        Camera cam = Camera.main;
 
        float tiltDX = gEvent.Values["tilt_dx"]*multiplier;
        float tiltDY = gEvent.Values["tilt_dy"]*multiplier;
 
        float screenX = gEvent.X;
        float screenY = cam.GetScreenHeight() - gEvent.Y;
        float screenZ = cam.WorldToScreenPoint(this.transform.position).z;
 
        Vector3 touchLocation = cam.ScreenToWorldPoint(new Vector3(screenX, screenY, screenZ));
 
            StartAffineTransform(touchLocation);
 
                AffineTransform.transform.Rotate(Vector3.up * (tiltDX));
                AffineTransform.transform.Rotate(Vector3.right * tiltDY*-1);
 
            EndAffineTransform();
    }
 
}

Note we are doing two things different here, using MoveObjectInCameraPlane to handle dragging and Affine transforms for the other gestures. MoveObjectInCameraPlane is a helper method inside of GestureWorks/Unity/TouchObject.cs that in response to touch drags the object parallel to the current camera plane. This has the effect of keeping the Clock dragging consistent in response to user touch regardless of the current camera position and orientation.

We are using Affine transforms for rotating, scale, and title. By Affine transforms we mean transforms that are centered on the gesture. When rotating the clock it will rotate around the start of the gesture, instead or rotating around the center of the clock.

Boundary checking is included in this script and is used to limit to how far the clock may be scaled and dragged. We use the clamp method to achieve this functionality.

Be sure to click and drag this script to the Clock > Clock object with the the collider box. Increase the supported gestures to 4 and type in the supported elements as follows:

• Element 0: NDrag
• Element 1: NRotate
• Element 2: NScale
• Element 3: ThreeFingerTilt

If you build and run at this point, you should be able to manipulate the clock in all the ways defined.

Keep in mind that GestureWorks clusters touchpoints with registered objects. The cluster geometry is extrapolated into an average centerpoint from which transformations occur. A two finger rotate for example, extrapolates the change in value (delta) based on the cluster’s central axis change along two points.

We will now add a script that will allow you to rotate the minute hand around the center point of the clock. This rotational gesture will require that two fingers are touching the minute hand at the same time (as defined in the gml). Create a new C# script in your MyScripts folder called TouchMinuteHand.

NOTE: the clock.fbx file was designed to contain properly registered pivot points for the clock hands. If you would like to achieve the same effect with a different model, you will need to also set registration points properly in the 3D authoring program.

linenums:1 |TouchClock: // //

using UnityEngine;
using System.Collections;
using GestureWorksCoreNET;
using GestureWorksCoreNET.Unity;
 
public class TouchMinuteHand : TouchObject {
 
    public Transform HourHand;
 
    public void NRotate(GestureEvent gEvent){
        float dTheta = gEvent.Values["rotate_dtheta"];
        transform.Rotate(0, 0, dTheta);
        HourHand.transform.Rotate(0, 0, dTheta/12);
    }
 
}

Attach this script to the MinuteHand object in the Hierarchy and increase supported gestures to 1 with NRotate as the supported gesture.

This particular script also has a public Transform property that is used in the NRotate method. The rotate method will also affect the HourHand object and rotate it based on the revolutions the minute hand makes. It is important that you click and drag the HourHand object onto the Hour Hand transform target in the script. The following is a screenshot of what it should look like when done properly:

If you would like to make the clock look like it has a good wind up (time keeping power), add this script to the SecondHand object:

linenums:1 |Creating a Unity class file // //

using UnityEngine;
using System.Collections;
 
public class SecondHand : MonoBehaviour {
 
    // Update is called once per frame
    void Update () {
        transform.Rotate(new Vector3(0,0,Time.deltaTime*2.5f));
    }
}

This script will rotate the second hand in a way that makes it look like seconds are elapsing. Note that this script does not inherit from TouchObject and is just a standard Unity script.

Whenever touchpoints occur outside of any touchable collider objects (the HitManager does not detect any hits with collider objects), we will assume that the user wants to control the camera. We will start with the script that needs to be added to the camera and then we will explain where we will modify the Main file to accommodate this hit detection change.

Create a new C# script in your MyScripts folder called TouchCamera. Add the following code to the file:

linenums:1 |TouchCamera // //

using UnityEngine;
using System.Collections;
using GestureWorksCoreNET;
using GestureWorksCoreNET.Unity;
 
public class TouchCamera : TouchObject {
 
    public void NDrag(GestureEvent gEvent){
 
        float multiplier = 0.001f;
 
        Camera cam = Camera.main;
 
        float dX = gEvent.Values["drag_dx"]*multiplier;
        float dY = gEvent.Values["drag_dy"]*multiplier;
 
        Vector3 previousPosition = cam.WorldToScreenPoint(transform.position);
        Vector3 nextPosition = new Vector3(dX, dY, 0.0f);
 
        Vector3 newPosition = previousPosition + nextPosition;
 
        float cx = Mathf.Clamp(cam.ScreenToWorldPoint(newPosition).x+dX, -2f, 2f);
        float cy = Mathf.Clamp(cam.ScreenToWorldPoint(newPosition).y+dY, 0f, 2f);
 
        transform.position = new Vector3(cx,cy,transform.position.z);
 
    }
 
    public void NScale(GestureEvent gEvent){
 
        float multiplier = 0.5f;
 
        float scaleDX = gEvent.Values["scale_dsx"]*multiplier;
        float scaleDY = gEvent.Values["scale_dsy"]*multiplier;
 
        float cz = Mathf.Clamp(transform.position.z+(scaleDX+scaleDY)*Flipped, 0.1f, 4.5f);
 
        transform.position = new Vector3(transform.position.x, transform.position.y, cz);
 
 
    }
 
}

Click and drag the script to the Main Camera object in the Hierarchy menu and increase the supported gestures to 2 with the following:

• Element 0: NDrag
• Element 1: NScale

It is important to keep in mind that Gestureworks performs gesture analysis in parallel. If one finger is touching the clock and one is outside of the clock, both will be interpreted as a drag–one for the clock and one for the camera. This feature of Gestureworks provides opportunity for a richer multi-user experience.

For this project we’ll make changes to the defaults for the GestureWorks GameObject:

All visualization of touch and mouse points are disabled. And Escape Key Exit Application is set to true to allow the user to exit the game using the escape key.

In the main script (created in step one, Configuring the project), set the following code:

linenums:1 |Main // //

using UnityEngine;
using System.Collections;
using System.Collections.Generic;
using GestureWorksCoreNET;
using GestureWorksCoreNET.Unity;
 
public class Main : MonoBehaviour {
 
    public Transform OriginalClockPosition;
    public Transform OriginalCameraPosition;
    public GameObject ClockObject;
 
    public float TimeToReset = 3.0f;
 
    // Use this for initialization
    void Start () {
    }
 
    private void ResetScene(){
 
        if(OriginalClockPosition == null || OriginalCameraPosition == null ||
            ClockObject == null)
        {
            return; 
        }
 
        ClockObject.transform.position =  new Vector3(0, 0, 0);
        ClockObject.transform.localScale = new Vector3(1, 1, 1);
        ClockObject.transform.rotation = Quaternion.identity;
 
        Camera.main.transform.position =  OriginalCameraPosition.position;
        GestureWorksUnity.Instance.ResetTimeSinceLastEvent();
    }
 
    // Update is called once per frame
    void Update () {
 
        if(GestureWorksUnity.Instance.TimeSinceLastEvent >= TimeToReset) {
            ResetScene();
        }
    }
}

This uses additional transforms setup in Unity to allow resetting of the scene. We won’t go through the detail of setting them up here, but they are illustrated below:

But what is interesting here is we can call the GestureWorksUnity singleton to see how long it has been since a touch event, and reset the scene if it is over a given limit:

linenums:1 |Calling the GestureWorksUnity singleton // //

if(GestureWorksUnity.Instance.TimeSinceLastEvent >= TimeToReset) {
        ResetScene();
}

To prevent this from being called repeatedly after timeout we can also reset the timeout timer in GestureWorksUnity:

linenums:1 |Reseting the timeout timer in GestureWorksUnity // //

GestureWorksUnity.Instance.ResetTimeSinceLastEvent();

In this tutorial we expanded on the knowledge gained in Tutorial #3 by manipulating on-screen bitmaps using gesture data obtained from GestureWorks. This tutorial covers a number of concepts that are central to the usage of GestureWorks:

• Fine tuned control over touchable objects
• Adding unique gestures to the registered touch objects
• Manipulating object instances based on data obtained from GestureWorks

It is the intention of this and the other GestureWorks Core tutorials to get the programmer started using the GestureWorks core framework, and is by no means an exhaustive explanation of all of the features of Gestureworks Core; in fact, we’ve only barely scratched the surface!

Additional tutorials for learning more about GestureWorks Unity:

• Unity Mini Tutorial: Switching Scenes
• Unity Mini Tutorial: Movement Related to Camera
• Unity Mini Tutorial: Registering Touch Objects

There is also FAQ here:

• Unity GestureWorks FAQ

For more information on GestureWorks Core, GestureML, and CreativeML, please visit the following sites:

• http://wiki.gestureworks.com
• http://gestureml.org
• http://creativeml.org

Previous tutorial: .NET & Unity: Interactive Bitmaps