ignoring clicks on elements that have moved recently

Ojan Vafai

unread,

Mar 1, 2016, 6:56:36 PM3/1/16

to intervention-dev

Attendees: rbyers, dtapuska, esprehn, ojan

AI: rbyers will talk to vollick to see if it should be someone on his team or someone on input-dev who makes a technical design/proposal.

rbyers: Intuition is that trying to do something automatically will cause more pain than it solves. Would love to find something we can do by default. Even if we find something, it's going to be a lot of work to implement it and convince ourselves that it's a net win. Start with an opt-in mechanism the ads folks can use and then we can evaluate better whether we can do something by default.

dtapuska: Seems like ads cares mostly about opacity.

ojan: Yup, that's one of their two problems. The other is this click thing. We have a solution for that which is V2 of IntersectionObserver (IO).

rbyers: The problems they're trying to solve seem related to the same IO problems. If IO were built on primitives, they could do their own thing here.

esprehn: even getBoundingClientRect is not very cheap by itself even if layout is done.

ojan: I was confused...ads' other concern was opacity in an ancestor.

esprehn: The way this would work is all iframes get their own layer and display list is split across layers. Then the compositor can detect which frame you're hitting. Allowing origin assertions based on what you're tapping.

rbyers: Still possible to overlap iframes. Provide strict boundary for developer but not for user.

esprehn: UI attribution unsolvable. Hard to convince the user that part of the screen belongs to someone.

A couple paths forward:

1. Boolean opt-in?

2. What about only for cross-origin iframes?

3. What about start by just adding measurement?

#2 maybe we could ship by default without worrying much about compat.

How do you define movement? Should scroll do it? What about in process fling and then I tap on it as the fling is slowing down?

esprehn: Difference between programmatic scroll and gesture.

ojan: Can we look at how much the thing moved?

rbyers: How long has this thing been under this point?

esprehn: Produces deadzone on the edges.

rbyers: Yup. That's fine.

esprehn: I think shipping an UMA makes sense as a way of evaluating.

rbyers: This is going to be a large number and we have no way of detecting false from true positives. So, UMA isn't great.

esprehn: Want a sanity check that we didn't screw things up, e.g. shouldn't be 90% of pages. Really want rappor to tell if it's just one website that sucks.

rbyers: For numbers big enough, dev channel covers it.

esprehn: Not clear that it's the long tail of sites that are doing this. Might not be a general problem.

rbyers: I think this might be something we could have by default for cross-origin iframes. How would we implement? Need a log of the last 500ms of the positions of every cross-origin iframe.

rbyers: Care primarily about mobile?

esprehn: Yes, touch screens.

ojan: Yup. Mobile is the P0 here.

esprehn: Seems rare that people accidentally click on things on desktop.

ojan: That's not totally true. The malicious content hits on desktop. But, mobile is more important.

esprehn: Realistically, we're probably give all cross-origin iframes a layer anyways. Tracking at CC layer level seems fine. CC layer can keep a log of location and size.

ojan: next step?

rbyers: Build on CompositorWorker?

esprehn: I think you need a bit more knowledge there.

rbyers: Next step is to have someone look into details of the design and see if it's reasonable.

esprehn: Doesn't need to be perfect. It's OK if sometimes we allow you to tap even if there was movement.

rbyers: Yes, as long as we're conservative to avoid false positives where we disallow clicks we should let through.

Rick Byers

unread,

Mar 3, 2016, 11:23:04 AM3/3/16

to Ojan Vafai, intervention-dev, Dave Tapuska

+dtapuska

One thing I wanted to follow up on because we didn't discuss it explicitly: all of these ideas would affect ONLY the generation of mouse/click events. Some ads implement their own fastclick behaviors by listening (sometimes correctly, sometimes not) to 'touchend' events. I propose that (for now at least) we don't want to try messing with the raw touch events - that's more complicated than focusing just on tap and is probably more likely to cause problems than it is to make things better. Sound ok?

I still support the idea of an iframe sandbox option to allow navigation only in the context of 'click' (and then ads quality should enforce this - encouraging the use of "touch-action: manipulation" for fastclick if desired).

Rick

--
You received this message because you are subscribed to the Google Groups "intervention-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to intervention-d...@chromium.org.
To post to this group, send email to interven...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/intervention-dev/CANMdWTvCuOFeYZjfeK%2BDUj4APsgWaDHnivFYLKPFPHWxswqbEg%40mail.gmail.com.

Dave Tapuska

unread,

Mar 3, 2016, 11:40:33 AM3/3/16

to Rick Byers, Ojan Vafai, intervention-dev

I know the conversation got into compositor worker and hit testing with property trees. This seems much to complicated at this phase to me.

Is this not really a case of some LayoutParts (the cross-origin ones) need to be some flavor of a light-weight position observer. FrameView has a list of LayoutParts that it can iterate on and those LayoutParts could have a history of my positions in the last n milliseconds. When we goto invoke a click event if that iframe moved in the last n milliseconds the click would get suppressed.

I know that this is a slimmed down variant of the problem we were trying to solve. In that if you intended to click on the iframe and it actually moved; but the hit test is now actually the thing that was under the iframe getting hit tested; the avoidance algorithm wouldn't kick in.

But I was wondering if the solution to the less complicated problem got us most of the way there in solving the problem.

I know that we would need to iterate on a list of layout parts when scroll positions and animations changed; but we are already tracking these objects on the FrameView; so just some computational cost to iterate and find the screen coordinates of each widget. Perhaps this is where the kicker of the CPU time is.

dave.

Dru Knox

unread,

Mar 3, 2016, 3:24:23 PM3/3/16

to Dave Tapuska, Rick Byers, e...@google.com, Ojan Vafai, intervention-dev

+Emil A Eklund

Could scroll anchoring solve this problem or get us most of the way? IIUC scroll anchoring would prevent scroll shifts from elements entering above the viewport. I suppose this wouldn't help for elements loading on screen. Do we know if these misclicks are more commonly triggered by elements loading above the viewport or within it?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/intervention-dev/CAHXv1wkd8e7qJ%3DeNMfZRPATObS1rfPLtKp2mZGU5UJOvhkadkw%40mail.gmail.com.

Rick Byers

unread,

Mar 8, 2016, 6:14:53 PM3/8/16

to Dru Knox, Dave Tapuska, e...@google.com, Ojan Vafai, intervention-dev, Ian Vollick

I don't think scroll anchoring helps much. The common case here is that everything is on the screen. Presumably scroll anchoring will change scrollTop only when something moves ABOVE the current viewport.

Also regarding doing this from main vs. impl, I chatted with vollick@ a bit about this and he suggests that doing it on impl is better for a couple reasons (Ian, correct me if I get any of this wrong):

main thread jank may be common in these scenarios, impl will have much more accurate knowledge of where things are
cc knows better the screen-space position of things (probably cheaper to implement there)
If we want to eventually go further trying to tackle the malicious cases too (eg. occlusion), only CC really has the knowledge to do that well

Of course one big advantage to doing this on main is that it could work for non-composited iframes (desktop) as well. Presumably OOPIF will eliminate that difference before long.

Ojan, do you agree that we're better off exploring the impl-thread approach than the main-thread approach?

Rick Byers

unread,

Mar 9, 2016, 10:47:04 AM3/9/16

to Dru Knox, Dave Tapuska, e...@google.com, Ojan Vafai, intervention-dev, Ian Vollick, Ali Juma

+ajuma who is interested in driving this from the compositor side (assuming we agree to go that direction).

Rick Byers

unread,

Mar 16, 2016, 2:10:17 PM3/16/16

to Dru Knox, Dave Tapuska, e...@google.com, Ojan Vafai, intervention-dev, Ian Vollick, Ali Juma

Ping Ojan, still curious on your take on the main vs. impl thread choice here.

Rick Byers

unread,

Mar 30, 2016, 11:51:40 AM3/30/16

to Dru Knox, Dave Tapuska, e...@google.com, Ojan Vafai, intervention-dev, Ian Vollick, Ali Juma, Elliott Sprehn

I had a chance to sync up with Ojan and Elliott on this topic yesterday. Here's a quick summary:

The primary concern here is getting this right long term, this isn't so urgent that we should take short-term hacks that aren't in the right direction long term.
To really solve this problem fully we think the position tracking must be done in the compositor.

Thinking about this further, I'm not 100% convinced by this (at least ignoring edge cases like accelerated animations moving an iframe).
Regardless we're going to have to take the tap timestamp into account - if the event was blocked for 500ms before reaching main, we'll have to consider the iframe position for at least the previous 500ms + whatever window we'd normally use.
But big picture, only the compositor can really reason reliably about what was visible to the user when

We think it makes sense to focus on coming up with a compositor-focused long-term design

If it helps, we can assume slimming paint v2, impl-thread hit-testing, etc.
Then we can evaluate where there's a subset that make sense to ship in advance
We're OK enabling this only for composited iframes (with site isolation we expect all cross-origin iframes to become composited soon enough)
Critically when using devtools mobile emulation, you do get the same compositing behavior as on real mobile

Eg. try this page on low-dpi desktop with layer borders / paint flashing enabled with and without devtools mobile emulation - both the div and iframe get their own layer in devtools emulation, but repaint on scroll otherwise

Ali, what do you think? Are you still interested in driving the design in Q2?

Ali Juma

unread,

Mar 30, 2016, 5:46:43 PM3/30/16

to Rick Byers, Dru Knox, Dave Tapuska, e...@google.com, Ojan Vafai, intervention-dev, Ian Vollick, Elliott Sprehn

This sounds like a good plan, putting together a compositor-focused long-term design seems like the right place to start. I'm still interested in driving the design of this in Q2.

Rick Byers

unread,

Mar 30, 2016, 6:04:42 PM3/30/16

to Ali Juma, Dru Knox, Dave Tapuska, e...@google.com, Ojan Vafai, intervention-dev, Ian Vollick, Elliott Sprehn

Excellent, thanks! Please keep us up-to-date on your thinking!

Rick

Ali Juma

unread,

May 5, 2016, 11:07:02 AM5/5/16

to intervention-dev, pain...@chromium.org, site-isol...@chromium.org, Dru Knox, Dave Tapuska, e...@google.com, Ojan Vafai, Ian Vollick, Elliott Sprehn, Rick Byers

+paint-dev, +site-isolation-dev

I've put together a doc discussing a proposed approach for this, together with some preliminary implementation ideas. Please have a look:

https://docs.google.com/document/d/1AL9RMFxLRRTQ818Bi7RPjbxF5JPBhti8XWNbLg83LYE/edit?usp=sharing

Ojan Vafai

unread,

May 5, 2016, 11:35:38 PM5/5/16

to Ali Juma, intervention-dev, pain...@chromium.org, site-isol...@chromium.org, Dru Knox, Dave Tapuska, e...@google.com, Ian Vollick, Elliott Sprehn, Rick Byers

Option 3 LGTM. Would it work to add a property tree for cross-origin iframes as the implementation of getting the information to CC?

Assuming y'all agree, I think the next steps are

1. Take the non-implementation section of your doc and propose it at https://github.com/WICG/interventions/issues so we can share the discussion with other vendors. Then propose a patch to the spec.

2. Send an intent to implement.

3. Implement.

4. Ship.

1 can happen in parallel to 2/3/4 and resolving 1 does not need to block shipping IMO. But we need to have started the conversation and given other browser vendors an opportunity to chime in before we ship.

I'm very excited for this feature! I'm not sure V1 will work out, but I think we'll be heading in the right direction and can iterate on it towards the right solution. In case you haven't seen it yet, here's a video some random person made on the internet to inspire you: https://www.youtube.com/watch?v=3tUmh8OfAiY.

Ali Juma

unread,

May 6, 2016, 4:52:45 PM5/6/16

to Ojan Vafai, intervention-dev, pain...@chromium.org, site-isol...@chromium.org, Dru Knox, Dave Tapuska, e...@google.com, Ian Vollick, Elliott Sprehn, Rick Byers

On Thu, May 5, 2016 at 11:35 PM Ojan Vafai <oj...@chromium.org> wrote:

Option 3 LGTM. Would it work to add a property tree for cross-origin iframes as the implementation of getting the information to CC?

That's an interesting idea I hadn't thought of. For option 3, I was thinking in terms of an iframe being some rect/region inside of a composited layer, and in that case per-layer data rather than a property tree seems like the best way to store the info. But of course we'll still have some iframes that do get composited, potentially consisting of several composited layers, and in that case using a property tree to represent the sets of layers that need to be treated as a single unit (so that we're really checking whether a click was inside the same iframe for the past 200ms and not whether it was inside the same layer of the iframe for the past 200ms) might be good a idea. This requires some more thought.

Reply all

Reply to author

Forward