Microsoft's Copilot AI knows your horrible handwriting

Microsoft is preparing to enable OneNote's Copilot to read and analyze handwritten notes. The feature went into beta late last month and will allow OneNote users to use their pen to write handwritten notes, then summarise them, ask questions and even create to-do lists based on the notes. OneNote's AI-powered Copilot feature also converts your handwritten notes into text that you can easily edit and share.

Microsoft first released Copilot in OneNote in November, and the update will also be available to existing Copilot for Microsoft 365 subscribers and Copilot Pro users once it's more broadly available. I briefly tested Copilot's ability to read my handwriting and was impressed that it was even able to decipher it. The summary works well for small handwritten notes as well as larger ones. I asked Copilot to rewrite an entire paragraph of a handwritten note, and he managed to transform it into a bit more enjoyable, readable text that was faithful to the original, which is impressive given that generative AI models tend to invent things in half the time. If you use OneNote for handwritten to-do lists, this feature definitely makes it easier to convert your to-do lists to text at a later stage. Once I created my handwritten list, Copilot accurately converted it to a text list in seconds. I'm not sure how Copilot would handle worse handwriting, though: Samsung's Galaxy AI has a similar feature that automatically formats handwriting lists, but it struggles with some of the worst handwriting I've ever seen (sorry, Allison!).

If you want to test Copilot's handwritten note recognition, you need to be using the latest version of OneNote for Windows (17628.20006 or later) and be a Microsoft 365 Insider with a Copilot Pro or Copilot for Microsoft 365 subscription.