Google blocked invalid interactions but still billed me for bot installs

Google is very good at catching click fraud - but terrible at catching install fraud. On one day in September its systems blocked 873 invalid interactions on one of our ad campaigns, which had a budget of CA$40 a day. On that same day, 20 of the 21 installs Google charged us for were bots - but Google says it’s normal user behaviour. That comes down to Google’s policy: officially or unofficially, Google doesn’t really care what happens once things are outside what Google can control.

We make Dayzle, a daily puzzle app. If you read my earlier posts, this is how the refund story ends: Google’s final answer was no. I think a lot of us want to believe in a fair world, so when the evidence is put in front of the people involved, we kind of expect them to end up somewhere close to the “right” answer. That wasn’t the case here, and the rest of this post explains why it works that way.

Three players, three simple goals

Every app ad campaign has three players, and each wants something simple:

• The publisher owns an app or a website with space for ads. It wants money.
• Google fills that space with ads and takes a cut of every transaction. It wants money.
• The advertiser, in this case me, pays Google to show its ad. It wants customers.

Google’s algorithm sits in the middle. It tries your ad in lots of places, watches which ones produce whatever you told it you want, and shows more of your ad there. If something starts working, it does more of it.

Two of them can count their success. The third can’t.

The publisher and Google measure success the same way: money. They both get paid when an ad is shown, watched or clicked.

The advertiser has a harder job. In theory I want installs, but installs aren’t all worth the same. If an ad finds a bunch of people looking for Call of Duty and they end up with a Sudoku app, those installs aren’t worth much. And you can’t judge what someone wanted from ten seconds in your app.

So you move the goal further in: came back the next day, finished a game, bought something. The deeper the goal, the more surely it means a real customer. But the deeper the goal, the less often it happens, and the algorithm needs a steady stream of them to learn where to show your ad. Learning from purchases takes a budget most small companies don’t have. So small advertisers end up with shallow goals: an install, an open, a return the next day.

One thing to make clear here: I know what real traffic looks like. This isn’t a post by someone who doesn’t understand that lots of real people download an app and never open it, or open it and never do the thing it was made for. I know that, and it’s a problem every app maker deals with. But because I know what real traffic looks like, I also know what bot traffic looks like. So take my word for it, or read my previous post: when I say these are bots, they are bots.

Sep 10 was the most flagrant day, though not the only one: the same pattern turned up on other days, on both of our campaigns. Here are the 20 installs from Sep 10 that I’m calling bots, next to every new Android install in the week after, Sep 11 to 18. Most of that week’s installs came from people who read my first post. Real people are all over the place: some open the app for a few seconds and leave, some never finish a game, and one has played 466. The bots all did the same thing.

If I ran a bot farm

Bot farms are in the business of stealing advertisers’ money. They steal it by taking the actions they assume advertisers want, as many of them as possible, without getting caught. Every view earns them money, and every fake action that matches the advertiser’s goal earns them more money and more ads.

So put yourself in the farm’s position. You own some apps or sites that show ads. How do you take as many of those actions as possible without getting caught?

• Don’t click the ads. Clicks are what Google checks hardest. I know this one first-hand: Google suspended our AdMob account over 15 accidental clicks, which were caused by a bug in Google’s own ads.
• Don’t make it obvious in other ways. Thousands of views from one block of internet addresses, or a site that does nothing but play ads around the clock, would probably get flagged too, if less reliably.
• Watch the video ads instead. Google credits an install to someone who watched ten seconds of a video ad and installed the app later. No click needed.
• Install the app, but skip the Play Store. Download it once and put the saved copy on a rack of real phones run by software. It’s faster, and Play never sees it, so it can’t run any of its anti-fraud checks.
• Give the advertiser’s goal what it measures. Open the app. Let the screens load. Come back the next day.

That is exactly what our data shows. On Sep 10, 20 of our 21 installs ran a version of the app the Play Store had stopped serving, on 18 different phone models, and none of the 20 came from a click on the ad. Over 27 launches, Google Analytics recorded zero seconds with the app on screen on any of them, yet ten kept sending events for two and a half to seven minutes after launch. A person can’t use an app for minutes without it being on screen. Five came back on later days, three of them almost exactly 24 hours later, which is the “returned the next day” goal a small campaign like ours might optimize for.

The blind spot

Google’s answer was consistent across five emails: it judges the ad interaction, meaning the view or the click, and nothing that happens after it. In its words, post-install variations “do not inherently qualify as invalid ad interactions.” So whatever Google blocked, the 20 views that turned into fake installs looked like normal views, and they were billed.

That leaves the advertiser as the only one of the three players who loses money on a fake install. The publisher was paid for the view, Google took its cut, and the advertiser paid for both.

So picture rooms full of phones, driven by computers. All they do all day is “watch” ads in the apps and sites the farm works for. My guess, and I think it’s a likely one, is that this is also why our app never made it on screen: on those phones, an ad is in the foreground around the clock.

Why small advertisers get farmed hardest

This is where the algorithm makes it worse. A small advertiser’s goal is shallow (by necessity), and a shallow goal is exactly what the farm produces, reliably and cheaply. To the algorithm, the farm’s apps look like the best place to show my ad, so it sends more of my budget there. The more the farm “installs”, the better it looks, and the more ads it gets.

A large advertiser can optimize for a deeper goal, because it has enough traffic: a purchase, for example, which a bot farm would never attempt. That gets the opposite result. The farm never buys anything, so its apps look terrible and the algorithm moves that advertiser’s budget away from them. The same farm, shown the same kind of ads, ends up with a big share of a small advertiser’s budget and a tiny share of a large one’s. Large campaigns get farmed too, I’d guess. Just not for the same share of their budget.

Where I am now

Google knows which apps showed the ads behind those 20 installs. I don’t. I asked them to look. Their answer was that they audit publishers in general and can’t tell me about any one in particular. Maybe someone did look. From where I sit, Google is the only one who can fix it, and we’re the ones paying for it.

And Google could fix it easily, even if only after the fact. Once an advertiser hands over evidence like ours, Google can see which apps served the views behind the fake installs, check those apps against every other campaign they’ve shown ads for, and cut them off. It chooses not to. I can’t tell you exactly why. In theory Google should care more about keeping advertisers happy than keeping bot farms happy. My best guess is that it’s more or less known that small campaigns get farmed, and it doesn’t matter much, because some apps spend $100,000 a day on ads and that’s who Google needs to keep happy.

We tried moving the goal deeper. It’s low effort to make a script open an app and higher effort to make one solve a Sudoku, so on Sep 29 we switched our remaining campaign’s goal to “won a puzzle”. Google says it found 3 people who did. Our records show 1, and that 1 cost about CA$80, which isn’t worth it for any app. CA$40 a day is about CA$1,200 a month, and that isn’t enough installs for the algorithm to learn who actually wins a puzzle.

So we stopped it. As of Sep 30, we’re spending nothing on Google ads. If you run a small campaign with an install goal, it’s worth assuming your campaign is getting farmed. Look at the usage data - it’s the only way to see what’s really happening.