Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Absolute-only DeviceOrientationEvents are bad for head tracking in VR #21

Closed
borismus opened this issue Aug 13, 2015 · 12 comments
Closed

Comments

@borismus
Copy link
Contributor

Some implementations of DeviceOrientationEvent don't work well for Virtual Reality (VR) head tracking. The main problem is drift: even when your head is stationary, your field of view will slowly rotate in some random direction. The cause of this problem is that in these implementations, DeviceOrientationEvents are absolute and fire based on the magnetometer, which is adversely affected by ever-present nearby metallic objects.

The plan to fix this in Chrome for Android is to switch the existing deviceorientation event to use Android's Sensor.TYPE_GAME_ROTATION_VECTOR and produce {absolute: false} events by default. This has the added benefit of bringing parity to the Chrome and Safari implementations. However, this is problematic for Augmented Reality and compass applications, which then have no way of getting absolute heading.

To address this, we want to add a new absolutedeviceorientation event which provides the same data format as the current 'deviceorientation' event, but will use Sensor.TYPE_ROTATION_VECTOR and provide {absolute: true} events by default. The other approach is to standardize webkitCompassHeading, or the Web Sensor API but these requires substantial standardization and implementation time and effort.

Do folks in this group have any objections to this approach?

@RByers
Copy link

RByers commented Aug 13, 2015

FWIW (from an outside perspective), I believe making deviceorientation relative to match Safari's behavior is great (probably should be part of the spec for maximum interoperability). Adding an explicitly absolute event also SGTM.

@borismus
Copy link
Contributor Author

Just wanted to note that this is a relatively small change that won't matter at all for most applications that use DeviceOrientationEvent. The only cases where it matters is for augmented reality web apps, which will need to switch to absolutedeviceorientation, and virtual reality web apps, which will automatically work much better.

There are relatively few AR apps out there. Those that exist already have a different problem, namely handling the difference between Chrome and Safari implementations.

@notthetup
Copy link

This would be great! I have been struggling with getting decent head tracking data in the browser. @borismus Is there a Chromium bug on this I could track?

@borismus
Copy link
Contributor Author

Yes, http://crbug.com/531610 is the one.

@tobie
Copy link
Member

tobie commented Sep 18, 2015

Some implementations of DeviceOrientationEvent don't work well for Virtual Reality (VR) head tracking. The main problem is drift: even when your head is stationary, your field of view will slowly rotate in some random direction. The cause of this problem is that in these implementations, DeviceOrientationEvents are absolute and fire based on the magnetometer, which is adversely affected by ever-present nearby metallic objects.

Isn't the cause of the drift the integration of the output of the gyroscope (angular velocity)? If so, how does removing the magnometer signal fix that? You actually want to use said signal to correct the drift. That's what the Oculus Rift does (see section VI. Yaw Correction of their paper on the topic).

@RByers
Copy link

RByers commented Sep 18, 2015

It also seems really strange to correct for "drift" by switching from "absolute" motion to "relative" motion events. Naively "absolute" sounds like the thing that would be better at avoiding drift :-).

How exactly this is worded in the spec seems important (to maximize the chance of interoperability and minimize developer confusion).

@borismus
Copy link
Contributor Author

Fair point about drift, let me try to clarify.

@tobie You link to a great paper, but they are using raw magnetometer values, and then doing their own calibration which is optimized for VR. This is not what we currently do for the deviceorientation event in Chrome for Android, which uses the Android sensor stack and just passes through values from TYPE_ORIENTATION into JS: http://developer.android.com/guide/topics/sensors/sensors_position.html#sensors-pos-orient. This sensor is designed for calculating magnetic north, which is very different from explicit assumptions in the paper:

Our approach is based on the following principles: 1)
True North is irrelevant because we are not trying to navigate
across the Earth; 2) the vector sum of the indoor field
and the Earth’s field provides the useful measurement; 3)
an in-use calibration procedure can eliminate most of the
offset due to the local field, and it can furthermore help
with soft iron bias; 4) never assume that the calibrated
values are completely accurate; 5) the particular field
magnitude is irrelevant, except for the requirement of a
minimum value above the noise level.

The sort of VR-specific drift correction the paper tackles is intended to solve a much more subtle drift, where for example, if you do a 360 (according to sensors), you end up tens of degrees off. This will indeed be the case when we make the switch, but this is a secondary problem to solve.

The current experience in Chrome for Android is you keep your head direction static but the screen drifts around. With raw sensor access, we could implement a robust VR-specific solution. But in the absence of this API's availability in Chrome, making this small change (which also brings us in line with Safari) yields a big win.

@tobie
Copy link
Member

tobie commented Sep 23, 2015

@borismus OK. Following the link, it seems the magnometer drift is due to the integration of the accelerometer's output which needs to be used to compensate the magnometer for the angle at which the device is held:

The orientation sensor derives its data by processing the raw sensor data from the accelerometer and the geomagnetic field sensor. Because of the heavy processing that is involved, the accuracy and precision of the orientation sensor is diminished (specifically, this sensor is only reliable when the roll component is 0).

I wasn't aware this was the case (boy are sensor interdependencies complicated). You're absolutely right. This does create drift and removing it will help alleviate the issue. It's clearly a low hanging fruit.

Regarding the rest of your comment this reinforces my thinking that we absolutely need to expose the low level sensors to the Web too (as per the Extensible Web Manifesto).

@RByers
Copy link

RByers commented Feb 2, 2016

The spec is updated for this now, right? Any further issues or can this be closed?

@timvolodine
Copy link
Contributor

yes I think the spec covers this now, i.e. by encouraging the 'deviceorientation' be relative by default.

Regarding the overall spec I believe there is one more issue (not related to this issue)
#24
which we may want to cover in the spec.

@RByers
Copy link

RByers commented Feb 2, 2016

Sorry, I meant only regarding this issue. I don't have permission to close this bug, can you (maybe reference the commit that addresses it)?

@timvolodine
Copy link
Contributor

this has been addressed in #22
closing this issue now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants