It's time to worry about online privacy again

post by Malmesbury (Elmer of Malmesbury) · 2022-12-25T21:05:30.977Z · LW · GW · 23 comments

Contents

  1. Radioactive decay
  2. The classic roast chicken scam
  3. An appointment at Times Square
  4. When to get paranoid
None
23 comments

As we all know, if you have nothing to hide, you have nothing to fear. Nobody cares about your private life. You are not an important geopolitical target. Nobody's going to spy on you to know what weird pornography you watch. 

And so, around 2015, people gave up on online privacy. Everyone stopped worrying about corporations and governments having full access to their data. In hindsight, I have to admit that things didn't go as bad as some feared. But I don't think this will last.

1. Radioactive decay

Based on real-life events: you're a biologist at the Bad Pathogen Research Institute. You receive an email from a graduate student whose name sounds vaguely familiar. She needs to measure radio-labeled samples with a scientific instrument but, unfortunately, you used it yesterday and you forgot to log out. Now it's locked with your password and she can't connect or even reboot. She's asking you to come as soon as you can to unlock it – as radioactivity decays, the signal is vanishing every minute. Sadly, you are attending a talk on the other side of the city, it's 45 minutes by bike and it's snowing. 

Obviously, you would never send your credentials by e-mail, right? Right?

This could, in principle, be phishing. Technically, a cunning spy could have stalked you, figured out your schedule, and crafted a deceptive e-mail to steal your password. But you know it's probably not the case, because nobody cares about your passwords enough to do something so complicated. So you send your credentials to the grad student using a one-time secret sharing link and everything is fine. 

I like to think that I can't be scammed because I know the ways of 1337 h4xx0rs well enough so they can't reach me. Of course, this is not true. I could totally be scammed, attackers simply don't have any interest in deploying the amount of energy it takes to scam me. 

That's why some people get phished and not others. It depends on two things:
🅰️. How much effort it takes to set up a scam so a given target falls for it
🅱️. How much effort an attacker is ready to dedicate to scamming that target

If 🅰️ is lower than 🅱️, the target gets scammed. If 🅱️ is lower than 🅰️, it's not worth it. On one end (high 🅰️, high 🅱️), you have hackers leaking e-mails from an important government official. On the other end (low 🅰️, low 🅱️), your grandfather receives an e-mail saying a hacker has caught him watching porn and he needs to send money otherwise the hacker will tell everyone. Your grandfather doesn't know much about Internet swindles, he's from a generation who's really ashamed to watch porn, and so he falls for it.

You, me, and most people are in between: too Internet-proof to fall for basic generic scams; not important enough to justify sophisticated personalized scams. Let me insist, you are safe not because hackers can't reach you, but because you are not important enough to justify the kind of attacks that would reach you.

2. The classic roast chicken scam

"Hi Alice, I hope you're having a good time at the concert. I just wanted to let you know that I'm at your apartment with a roast chicken that I bought at the farmer's market. My phone is out of battery, so I'm using my friend's phone to send you this message. Could you please send me your apartment door code so I can leave the chicken in front of your door?"

It took ChatGPT less time to write this than it took me to copy-paste it. Most of the personal context could be figured out based on localization data. Obviously, you would never let a website access your localization data unless strictly necessary, right? Right?

I don't know about you, but I'm scared. Artificial intelligence can totally automate the process of stalking someone. It can extract all the available information from all your accounts on the Internet, then a large language model can generate a perfectly realistic bait, tailored just for you. It's the Nigerian Prince all over again, except this time the Nigerian prince lives on the 5th floor and you had a beer with him last Friday. It could also be blackmail: I hope you never dared write anything politically incorrect on the Internet, because the AI will find it.

Remember 🅰️, the amount of effort required to scam normal people like you and me. What happens when AI makes it shrinks to zero? If impersonation and blackmail become a simple button push, most people will suddenly face attacks much more sophisticated than what they're used to. If unprepared, they'll think "nobody would make up such a complicated scheme just for me", and they will walk straight into the trap.

Like a puzzle, there are very strong network effects with stalking. I don't actually believe it's possible to pinpoint a person based on speech patterns alone, as film detectives do. However, if the same obscure link gets shared on Twitter and Facebook within a few minutes, and the Twitter handle is partially similar to an Instagram account that posted a picture of a monument whose location matches GPS data extracted from a random vuln guitar-tuning app, and on top of that the speech patterns are the same, you can connect the dots and draw some conclusions.

This means that, if you can scam the Median Joe automatically, then you might as well attack tens of thousands of targets in parallel, in a coordinated manner. Each message is filled with bespoke in-jokes and details about the target's whereabouts. Everyone falls for it at the same time.

Whether it's possible to push the world into chaos using social engineering depends on how much information is available about the average person. As people liberally leave more and more identifying information on tens of different platforms, a phase transition occurs, making it possible to fill all the gaps and know everything about everyone. 

3. An appointment at Times Square

Who is this attacker we're talking about? It could be something boring like lone-wolf terrorists, Russian cybersoldiers or the Vikings conspiracy. Or it could be an autonomous AI that was programmed to manufacture as many trombones as possible and is now trying to gain power so it can turn Earth into a trombone factory. 

But, unlike a lot of evil-AI-takeover stories [LW · GW], this one doesn't require any super-human intelligence. The oldfags among you might remember the 2011 involuntary flashmob, when Internet trolls lured a bunch of people into going to the same place at the same time, all believing they were going on a date. It established a new precedent for how much you can do remotely with an Internet connection. 

This didn't involve any science-fictionesque protein-based nanorobots. Instead, it took a lot of effort from many human participants to maintain conversations with the targets over several days. If AI can fully automate conversations, then something as big as the involuntary flashmob can be done with a simple python script. It certainly opens a world of trolling possibilities.

4. When to get paranoid

That leaves us with two solutions: downstream and upstream paranoia. Downstream paranoia is when it's too late – you've already given up all the information it takes to scam you, so you need to be paranoid about every single online interaction, to make sure that every message you receive is not from an impersonator. This comes with a serious erosion of Internet trust, assuming there can be a working Internet at all in these conditions.

Upstream paranoia is what Richard Stallman has been telling us to do for forty years: make sure that you don't give away enough information so that AI can't do realistic scams in the first place. Privacy is like a plunger – you should get one before you need one.

As your prototypical nerd, I used to be really into FOSS, the EFF, blob-free GNU/Linux distros, XKeyScore, Echelon, INDECT, PRISM and other names most of us have forgotten what they were. Then, like most people, I gradually stopped caring, and now I'm leaving a trail of personal data wherever I go. 

I guess it's time for me to go back to my pre-2015 technoparanoia. Don't get me wrong, at the society level, we are definitely past the phase transition point – even if you are secretive, most people are not and the information is out there. But I think it's still possible to protect yourself if you act early enough. There is no way the Internet remains the way it is now. It's hard to tell how much privacy will really be necessary on the "new" Internet, but given the pace of language models' progress, I'd rather err on the overkill side.

If you are going to engage in anonymity warfare, here are a few old-school tips for upstream paranoia (I can't promise that they are secure, let me know if you think they aren't):

• A personal favourite: TrackMeNot. This doesn't prevent Google from spying on you, it just drowns Google in a flood of fake requests. So every time you search for something, TrackMeNot also searches for all kinds of random stuff, and now Google thinks you're a hunter from Siberia. I find this approach particularly promising.
• I heard hosting your own Searx instance was pretty good. I haven't tried that yet.
• Signal or Matrix.org instead of Whatsapp, Messenger, etc. The good thing is that you will no longer sound like a terrorist or a pedophile when you ask your friends to switch, so it might actually work (just point them to this post!)
• In general, use free software (using Linux makes this much easier). Especially avoid anything that's funded by advertisement. "If it's free, you are the product."
• Remove all public information on social media. Preferably use Mastodon. Exhume your old Internet pseudonym.
• This is going to sound super extreme, but you might want to store music and films on a hard drive instead of streaming. It just sounds too easy for an AI to impersonate you when it knows what shows you watch.
• For e-mail, I currently use Protonmail. It's claimed to be fairly confidential.
• Newpipe and Invidious as front-ends for Youtube (I'm afraid Peertube is not the best thing for anonymity, as it's peer-to-peer)
• If you want to dive all the way in, try Richard Stallman's lifestyle. Also,  see Gwern on maintaining anonymity.

Let's make sure AI attacks encounter at least some resistance. This resistance starts with you (*epic music*).

Cross-posted from my blog.

23 comments

Comments sorted by top scores.