Check the online version, I often update my slides.
Looking for something on the intertubes? Just use Google, right? But what if you're looking for publicly accessible webcams, or power plants? Or leaked databases and stolen credentials? Or HTTPS certificates that often reveal long forgotten test servers? What would MacGyver do? Well, he'd sit in the first row for this talk. We'll talk about Certificate Transparency, Shodan, interesting Google queries and more.
I've delivered this talk at a Pipedrive event in Prague. I know many people in Pipedrive around the world and have many friends there from my Skype era. And they have just opened an office in Prague! It's headed by my friend, also an ex-Skyper, Tomáš Řehoř, and they're now looking for new colleagues. Bunch of nice people.
There's also a Czech version.
Date and event
#1 MacGyver is a troubleshooter from the MacGyver TV series, the first episode was aired in 1985. MacGyver was always able to find a solution to any problem, which he then solved using off-the-shelf tools. Sometimes, I also find something, using just a web browser. I'll show you how such an ubiquitous software tool, like a web browser, can be used to find problems. You'll have to find the solution yourself then, for example in some of my other talks, or at my trainings.
#2 You can use Have I Been Pwned? to check if your email address (or somebody else's email address) is in some publicly known data leak. The service is operated by Troy Hunt, and he uploads data he finds online or data people send him. The word “pwned” comes from “owned”, both meaning somebody possesses something, in this case it's your online identity.
#3 If your email (personal or work-related) is in some of the data leaks, and you haven't yet changed your password for that service, you better do it soon. Also, don't forget to set up notifications so that Troy Hunt (and Have I Been Pwned) can email you as soon as your address pops up in some of the newly imported data. You can also search whole domains, which is great for company administrators.
#4 Sometimes, you'll find interesting data by coincidence. Back in February 2014, I wanted to buy an MP3 from Soul Deep Recordings but instead, I found a database backup. The site's security was pretty bad, as a year later, in 2015, their homepage was changed to read Hacked by… You can use Google to search for directory listings with backups: search for Index of / & sql.
#5 How else do you find database “backups” (ehm, leaks)? If a database gets leaked on the Internet, there's a quite high probability that Google will index it. Imagine a company using MD5 to hash user passwords. What you can do then is to search for hashes of a few predictable passwords. Try searching for a hash of Password1 for example, add inurl:sql or filetype:sql for better results. There's still a lot of companies using MD5, unfortunately. Passwords should be hashed using algorithms like bcrypt or Argon2 and existing passwords should be rehashed when upgrading from MD5 or similar.
#6 The last slide of my Czech talk about password hashing includes a hash
a76c8ba54d7be5d57daf858987c168a458009312 but I forgot what the input was. I only knew I've used something simple so anybody could find it using Google. When writing another talk in May 2017 I tried finding it. It was a hash of a word “konec” (Czech for “the end” – at the end of the talk, ha ha, yeah, of course). But looking at the results page I've noticed something different.
#7 It was a link to customer login details dump from “Czechia” – a web hosting service by a Czech company called ZONER (here's a Czech article announcing the leak). Roughly 100k email accounts and SHA-1 passwords, some FTP credentials etc. I've reported it to the company (if you find something similar, please report it as well) and didn't publish until they have resolved it. Somebody's email password was “konec”, ZONER used SHA-1 to hash it and that's what I was looking for. Successfully, I guess.
#8 Back in 2012 I was also looking for something, and found something else. I was looking for someone, and found a source code for a bank's site and some more files instead. The thing is, someone I was looking for have worked on the system the Air Bank's site was using. They say that page 2 of Google search results is the best place to hide a dead body and I can confirm that's true.
#9 Talking about banks… their sites, and actually all sites, should be using HTTPS. For that you need HTTPS certificates issued and signed by trusted certification authorities (CAs). Though some of them are a bit messy, issue certificates without proper validation, use algorithms that should not be used anymore (see for example Symantec issues or WoSign and StartCom issues) and so on. The Certificate Transparency (CT) project has introduced the much needed insight into how CAs operate.
#10 Certificate Transparency requires all certificates to be logged into multiple CT logs. Previously, logging was optional for most of the CAs, although probably all of them were logging voluntarily anyway. Since April 2018, Chrome requires all newly issued certificates to be logged.
The browser can check the certificate has been logged without slowing down the load times for example this way: before issuing a real certificate, the CA sends a so called precertificate to the CT log, the log responds with Signed Certificate Timestamp (SCT) – the time it has seen the (pre)certificate. The CA then attaches the SCT to the certificate, and eventually issues the real certificate to whoever requested it. Then they configure their server to send the certificate to browsers connecting to it. The browser can then verify the SCT in the certificate is there (a proof it has been logged) and the timestamp is valid, and can reject the connection if not.
#11 CT logs are public and can be searched using tools like crt.sh. What's a certificate search good for? First, you can check if any trusted CA has not misissued a certificate for your domain. crt.sh also supports RSS, or Atom feed so you can automate the whole search.
Of course you can check other domains too, quite often you'll find long-forgotten services, test servers, which might not be properly safeguarded, unlike production machines. Such a list often helps with bug bounties. For example Czech T-Mobile runs a bug bounty program so here's a list of their domains (and here's another one). If subdomains are “hidden” behind a wildcard certificate, then they're not visible in the logs, of course. There's also this Domain Label Redaction draft which defines mechanisms to allow DNS domain name labels that are considered to be private to not appear in public CT logs but it's only a draft, now expired. Remember that if the domain is not in the logs it still doesn't mean nobody can find it. By the way, the crt.sh service offers direct PostgreSQL access.
There's also a CT log search by Facebook, you can even subscribe to receive notifications when a new certificates for a given domain is spotted. CertStream offers a stream of newly issued certificates, they also have a command-line client which you can use to automate scans. You'll find default and not yet secured deployments, and more fun. And of course there's a CT search by Google, too.
#12 And now for something completely different: this is the map of the Internet. It was created in 2014 by John Matherly. He's pinged all devices directly connected to the Internet and put the results on a map.
#13 If your Internet connection is fast enough, you can scan the whole Internet in 5 minutes using masscan, though to transmit 10 million packets per second is not a job for your regular Wi-Fi. In 2014, I scanned 6.5 million of Czech IP addresses for the Heartbleed vulnerability. I used the cheapest VM I could find and the scan took 2 hours.
When talking about the Internet, I mean IPv4 Internet. The IPv6 Internet has a bit more addresses so scanning all of them would not be very wise. Though it would be clever to scan only “live” addresses but how do you tell a “live” address? There's a nice trick: run a NTP server available over IPv6 and add it to the server pool. Then if a machine wants to sync its time it will connect to your server with some probability. And that's another “live” IPv6 address for you.
#15 Shodan is a search engine for everything. IoT, power plants, refrigerators, cameras, websites etc. It just tries to connect to known ports, and saves responses (both text and pictures like remote desktop screenshots) which you can then search. Searching for T-Mobile servers? Connected fridges (but why), fridges running on port 8081 (requires login), there you go. Want images from all cameras and remote desktops? You'll need an account and then head to images.shodan.io. You can also see images from just one country for example. You'll see a lot of weird things, consider yourself warned.
If you register with an academic email address (e.g. .edu, .ac.uk), you'll receive an automatic, free upgrade. If your academic email doesn't have a known academic domain you can still email Shodan support for a manual upgrade. You can also get a Shodan lifetime membership for $5, instead of $49, in a Black Friday sale, so watch out for the next one.
#16 Here are the most important search engines on one slide:
- Your email addresses in data breaches https://haveibeenpwned.com
- Certificate stream https://certstream.calidog.io
- Certificates https://developers.facebook.com/tools/ct/ and https://crt.sh
- Port scanner https://github.com/robertdavidgraham/masscan
- Search engine for anything https://shodan.io (+ https://censys.io & https://www.zoomeye.org/)
If you spot some data leaks please report them to the owner. You can also ask CSIRT for help. We have a national one in CZ, and CZ government has their CERT too. Let's make the Internet a safer place.
Running a site or an web app? Consider adding
security.txt, so people like me can report security issues easily.
I build web applications and I'm into web application security. I like to speak about secure development. My mission is to teach web developers how to build secure and fast web applications and why.