Windows 11's new AI feature makes it way too easy to steal everything you viewed or typed

Microsoft unveiled "Recall" at its special event on May 20. This Copilot+ PC-exclusive feature promises to bring "photographic memory" to your computer, allowing going back to any app or file you worked with. To combat privacy concerns, Microsoft published a page with details on how Recall works. However, security researchers strongly disagree with the company's statements.

Kevin Beaumont, a cybersecurity expert, published a detailed blog post on Medium where he dug deeper into how Recall works. The verdict is a rather harsh one: stealing everything you viewed or typed on your computer is now very easy.

Beaumont claims that the idea of Recall is an interesting feature that requires "incredibly careful communication, cybersecurity, engineering, and implementation." Sadly, Recall allegedly has none of those.

Although the data processing and encryption are indeed happening on-device only, all that info is not immune to hackers and malware. The encryption will protect your data if the attacker doesn't know your username and password, but things change when hackers get a hold of your credentials using infostealers.

Recall works by taking screenshots of everything that happens on your computer every few seconds. Then, the system OCRs (recognizes optical characters) that data and puts it into a database in the user folder. Everything is stored in plain text, and there is no need for system rights to access it.

They have tried to do a bunch of things but none of it actually works properly in the real world due to gaps you can drive a plane through.

Kevin Beaumont created a website that can process a Recall database and instantly search for anything inside it. However, he is holding the project back until Microsoft ships it or maybe does something to improve security. Kevin says, "the wider cyber community will have so much fun with this when generally available."

Things get worse when you realize what is stored in your Recall database:

Everything a user has ever seen, ordered by application. Every bit of text the user has seen, with some minor exceptions (e.g. Microsoft Edge InPrivate mode is excluded, but Google Chrome isn’t).

Every user interaction, e.g. minimizing a window. There is an API for user activity, and third party apps can plug in to enrich data and also view store data. It also stores all websites you visit, even if third party.

Customers should also know that deleting emails, messages, pictures, files, or anything else on your computer will not delete them from Recall—it stays there indefinitely or until manually deleted/overwritten.

Although Microsoft Defender is pretty good at detecting infostealers and malware, "off the shelf" malware can scrape the entire database before automated detection kicks in.

Beaumont claims Microsoft "should recall Recall" and rework the feature to address all the privacy concerns, especially in light of Satya Nadella saying engineers should prioritize security over any other priority.

You can read the full story in Kevin's Medium post. If you still want to try Recall and see how it works, check out this third-party app that makes it possible to enable the feature on existing hardware (with caveats).

It remains to be seen how Microsoft will address these revelations. For now, Recall is technically available in the Release Preview Channel of the Windows Insider program. It is expected to arrive for the general public with the first Copilot+ PCs, such as the new Surface Pro and Surface Laptop.