CoreLocation Header Base on Rear Camera (Augmented Reality)

Question

I would like to create an augmented reality view that points to an object in a direction. However, the CoreLocation header does not work correctly when you look up with your camera (say, a 20-story building when you are on the first floor).

It indicates the opposite direction (probably the direction pointing to the top of the phone).

I tried several methods to make it work in the direction the camera points, for example:

1, +180 degrees, when the device orientation is> 45 degrees (not accurate enough, suddenly the direction decreases by 10 ~ 20 degrees)

2, tried to calculate using CMMotionManager with the formula from the manual below. http://www.loveelectronics.co.uk/Tutorials/13/tilt-compensated-compass-arduino-tutorial .

3, tried to simulate the logic from android using ios deviceMotion.magneticField and deviceMotion.gravity.

4, use the rotation matrix (some other messages in the stack overflow, but not exactly)

double heading = M_PI + atan2(self.altitudeData.rotationMatrix.m22, self.altitudeData.rotationMatrix.m12); heading = heading*180/M_PI; 

My idea is ending, what else can I try to fix. I know that there are applications (some applications that can see the sun and the star) that do this correctly.

Answer 1

After many research and testing. I ended up using GLKit for calculation, as it saves me a lot of problems. Just leave it here for anyone who comes up with this question.

First, I started updating the movements of the CMMotionManager device with CMAttitudeReferenceFrameXTrueNorthZVertical.

 self.hasMotion = NO; CMMotionManager *cmmotionManager = [[CMMotionManager alloc] init]; [cmmotionManager startDeviceMotionUpdatesUsingReferenceFrame:CMAttitudeReferenceFrameXTrueNorthZVertical toQueue:[[NSOperationQueue alloc] init] withHandler:^ (CMDeviceMotion *motion, NSError *error) { self.hasMotion = YES; }]; self.motionManager = cmmotionManager; 

From some codes that I found on the Internet to draw an openGL world using CoreMotion rotation and mix it to get a point from the screen to the 3D world:

 float aspect = fabsf(self.view.bounds.size.width / self.view.bounds.size.height); GLKMatrix4 projectionMatrix = GLKMatrix4MakePerspective(GLKMathDegreesToRadians(45.0f), aspect, 0.1f, 100.0f); CMRotationMatrix r = self.motionManager.deviceMotion.attitude.rotationMatrix; GLKMatrix4 camFromIMU = GLKMatrix4Make(r.m11, r.m12, r.m13, 0, r.m21, r.m22, r.m23, 0, r.m31, r.m32, r.m33, 0, 0, 0, 0, 1); GLKMatrix4 viewFromCam = GLKMatrix4Translate(GLKMatrix4Identity, 0, 0, 0); GLKMatrix4 imuFromModel = GLKMatrix4Identity; GLKMatrix4 viewModel = GLKMatrix4Multiply(imuFromModel, GLKMatrix4Multiply(camFromIMU, viewFromCam)); bool isInvertible; GLKMatrix4 modelView = GLKMatrix4Invert(viewModel, &isInvertible); int viewport[4]; viewport[0] = 0.0f; viewport[1] = 0.0f; viewport[2] = self.view.frame.size.width; viewport[3] = self.view.frame.size.height; bool success; //assume center of the view GLKVector3 vector3 = GLKVector3Make(self.view.frame.size.width/2, self.view.frame.size.height/2, 1.0); GLKVector3 calculatedPoint = GLKMathUnproject(vector3, modelView, projectionMatrix, viewport, &success); if(success) { //CMAttitudeReferenceFrameXTrueNorthZVertical always point x to true north //with that, -y become east in 3D world float angleInRadian = atan2f(-calculatedPoint.y, calculatedPoint.x); return angleInRadian; } 

Answer 2

To save anyone else, here Chi answers in Swift:

 import GLKit func headingCorrectedForTilt() -> Float?{ guard let motion = self.motionManager.deviceMotion else{ return nil } let aspect = fabsf(Float(self.view.bounds.width / self.view.bounds.height)) let projectionMatrix = GLKMatrix4MakePerspective(GLKMathDegreesToRadians(45.0), aspect, 0.1, 100) let r = motion.attitude.rotationMatrix let camFromIMU = GLKMatrix4Make(Float(r.m11), Float(r.m12), Float(r.m13), 0, Float(r.m21), Float(r.m22), Float(r.m23), 0, Float(r.m31), Float(r.m32), Float(r.m33), 0, 0, 0, 0, 1) let viewFromCam = GLKMatrix4Translate(GLKMatrix4Identity, 0, 0, 0); let imuFromModel = GLKMatrix4Identity let viewModel = GLKMatrix4Multiply(imuFromModel, GLKMatrix4Multiply(camFromIMU, viewFromCam)) var isInvertible : Bool = false let modelView = GLKMatrix4Invert(viewModel, &isInvertible); var viewport = [Int32](count:4,repeatedValue: 0) viewport[0] = 0; viewport[1] = 0; viewport[2] = Int32(self.view.frame.size.width); viewport[3] = Int32(self.view.frame.size.height); var success: Bool = false let vector3 = GLKVector3Make(Float(self.view.frame.size.width)/2, Float(self.view.frame.size.height)/2, 1.0) let calculatedPoint = GLKMathUnproject(vector3, modelView, projectionMatrix, &viewport, &success) return success ? atan2f(-calculatedPoint.y, calculatedPoint.x) : nil }