How do you integrate Leap Motion Controller into a Mac app?

Question

I am trying to run my own Leap Motion project on a Mac, but I am having problems.

I would like to use Objective-C for this project, and I read that there is a Leap Motion library for this language. However, I'm not sure how to integrate Leap Motion controls with this library into a Mac application.

Something similar was asked here , only they asked using the Python Leap Motion library.

How can you add Leap Motion controls to an Objective-C Mac app?

Answer 1

I did this recently, so I can provide the steps that I used to add Leap Motion controls to a Mac application. In fact, the source code for the MyPap Molecules application is available on GitHub if you need an example to work with. All you need is a Leap SDK.

Leap Motion Objective-C headers are mostly wrappers around their base C ++ API, but you don't need to worry about it because you can only access it through Objective-C.

To add a library to your project, first install the Leap SDK somewhere on your system. From there, add links to the Leap.h , LeapMath.h , LeapObjectiveC.h and LeapObjectiveC.mm in your project. Add the libLeap.dylib library to the linked libraries.

To avoid compiler and linker errors during the beta period (which may have been resolved), I needed to go to my build settings and change the C++ Standard Library to libstdc++ .

You need to make sure that the Leap library comes with your application, so make sure it is copied to related structures during the build phase. I also needed to use the next phase of the Script build to make sure its internal path was set correctly (again, not sure if this is necessary now):

 echo TARGET_BUILD_DIR=${TARGET_BUILD_DIR} echo TARGET_NAME=${TARGET_NAME} cd "${TARGET_BUILD_DIR}/${TARGET_NAME}.app/Contents/MacOS" ls -la # then remap the loader path install_name_tool -change @loader_path/libLeap.dylib @executable_path/../Resources/libLeap.dylib "${TARGET_NAME}" 

The final caveat when setting this up is that if you isolate your Mac application, you must enable outgoing network connections or the application will not be able to connect to the Leap Motion server application running on your Mac.

Once all setup is complete, you can start entering data from the Leap Motion Controller. The central LeapController object is what provides delegate callbacks for Leap Motion events. You install one of them using the following code:

 controller = [[LeapController alloc] init]; [controller addListener:self]; 

Your delegate should comply with the LeapListener protocol, which has a bunch of additional callback methods:

 // Controller object has initialized - (void)onInit:(NSNotification *)notification // Controller has connected - (void)onConnect:(NSNotification *)notification; // Controller has disconnected - (void)onDisconnect:(NSNotification *)notification; // Exiting your LeapController object - (void)onExit:(NSNotification *)notification; // New frame data has arrived from the controller - (void)onFrame:(NSNotification *)notification; 

The connect and disconnect callbacks are obvious, although you will notice that the disconnect callback never starts when you debug your application in Xcode. You need to run the application outside the debugger so that it works when the Leap Motion controller is disabled.

You will spend most of your time on the -onFrame: , because where you get updates to your location. You get your LeapController back as a notification object from this, and you can extract the current frame data using the following:

 LeapController *aController = (LeapController *)[notification object]; LeapFrame *frame = [aController frame:0]; 

LeapFrame has all the scene data observed by the jump, including the position of the hands and fingers, as well as the overall scaling of these objects. The -hands method on the LeapFrame gives you an array of all detected LeapHand objects. In turn, you get each LeapFinger for LeapHand from your -fingers method. You can also extract palmPosition , palmNormal and general direction for the hand.

Directions are represented as LeapVectors, which are wrapping objects around three-dimensional vectors. You can extract X, Y, and Z components from them, or perform vector manipulations or comparisons between other LeapVectors.

In Molecules, I adjust the scale and orientation of my molecular structures by reading back and forth or left to right motion relative to the screen. I do this by comparing the position of an open hand from one frame to another. I save the previous frame and then do a comparison between now and the last time we saw this hand using

 LeapVector *handTranslation = [firstHand translation:previousLeapFrame]; 

You can also compare scale, rotation, etc. between frames for hands or for all objects in the frame as one group. From frame-to-frame translation, which I received in the above code, I can extract the individual components X, Y, and Z to rotate my model in response to X and Y translations and Z-based scale.

I use rough positioning for my application, but you can obviously go a lot better by tracking individual fingertips. After you connect to your application and connect to it, I recommend reading the Leap SDK header files for the Objective-C side to find out what other options are available to you. Also, experiment with different interaction modes, because you may be surprised at what does and doesn't work well in three-dimensional space.

Answer 2

@Brad Larson's answer pretty much covers everything, so consult him for more details. However, his answer shows how to use the LeapListener protocol, which works using NSNotifications.

If you either don’t like NSNotificationCenter: p, or don’t care about which thread contains your updates, here is an example of how to use LeapDelegate .

 @interface MyClass () < LeapDelegate > @property (strong, nonatomic) LeapController *controller; @end - (void)startLeapMotion { if (!_controller) { _controller = [[LeapController alloc] initWithDelegate:self]; // // Could also be... // // [_controller addDelegate:self]; // [_controller removeDelegate]; } } - (void)onInit:(LeapController *)controller { NSLog(@"Init"); } - (void)onConnect:(LeapController *)controller { NSLog(@"Connect"); // Some settings that came bundled with the Leap sample project. Use if needed // // [controller setPolicyFlags:LEAP_POLICY_DEFAULT]; // [controller enableGesture:LEAP_GESTURE_TYPE_CIRCLE enable:YES]; // [controller enableGesture:LEAP_GESTURE_TYPE_KEY_TAP enable:YES]; // [controller enableGesture:LEAP_GESTURE_TYPE_SCREEN_TAP enable:YES]; // [controller enableGesture:LEAP_GESTURE_TYPE_SWIPE enable:YES]; } - (void)onFocusGained:(LeapController *)controller { NSLog(@"Focus Gained"); } - (void)onFrame:(LeapController *)controller { // Write awesome code here!!! NSLog(@"Frame"); } - (void)onFocusLost:(LeapController *)controller { NSLog(@"Focus Lost"); } - (void)onDisconnect:(LeapController *)controller { NSLog(@"Disconnected"); } - (void)onExit:(LeapController *)controller { NSLog(@"Exited"); }