🤝 Microsoft Build 2024: Your data, your device, the future of AI is local
🤝 Microsoft Build 2024: Your data, your device, the future of AI is local
Dear curious minds,
This issue is packed with exciting news and updates from the recent Microsoft Build 2024 conference. Microsoft Build is an annual event where developers and technology enthusiasts come together to explore the latest innovations, product announcements, and advancements from Microsoft. The future Microsoft envisions and shared in this event is one where AI assists at every task we work on, with a focus on running privacy-friendly on our own devices.
In this issue:
💡 Shared Insight
Meet Your New Digital Assistant: AI That Works (and Plays) With You📰 AI Update
New Windows PCs designed for AI Outperform Apple’s MacBooks
Find Anything You've Seen on Your PC with Microsoft Recall
Microsoft’s New Phi-3 Models Power On-Device Workflows🌟 Media Recommendation
The Hidden Details of Copilot+ PCs: Alex Ziskind's Reveals Chip Variants
💡 Shared Insight
Meet Your New Digital Assistant: AI That Works (and Plays) With You
The future of PC work is evolving, and it won't be a solo journey. You will always have access to AI assistance that can see and interpret the images on your screen, hear and analyze the audio, and perform tasks based on your commands. This vision of a multi-modal AI assistant was highlighted at the Microsoft Build event, where you share your screen or session with the AI and get real-time feedback. Similar demonstrations were showcased by OpenAI and Google last week, indicating a trend towards integrated AI support in everyday computing tasks.
In the demonstration shown in the YouTube video above, Copilot assisted a user playing Minecraft, offering real-time advice and support. The AI recognized the game's interface, understood the user's requests, and responded with contextual help. This demo not only illustrated the potential of AI to assist in gaming but also highlighted the broader applications of such technology in various tasks you perform on your PC.
Imagine working on a document and needing to reference a specific image or dataset. With an AI assistant, you could simply ask for the information, and it would appear on your screen without switching to a different application. Similarly, if you are programming and have a question about an implementation, the AI will be there to assist.
The future of PC work is collaborative, with AI acting as an ever-present assistant. The demonstrations by Microsoft, OpenAI, and Google provide a glimpse into this exciting future where multi-modal AI enhances our computing experience, making it more efficient, accessible, and personalized. As these technologies continue to evolve, we can expect AI to become an integral part of our daily computing activities, transforming how we interact with our devices.
📰 AI Update
New Windows PCs designed for AI Outperform Apple’s MacBooks
Microsoft unveiled new Copilot+ PCs with AI capabilities built-in through dedicated AI chips called NPUs (Neural Processing Units). These new devices are powered by the ARM-based Snapdragon X Elite or Snapdragon X Plus processors from Qualcomm. A system on a chip (SoC) design which combines CPU, GPU and NPU.
According to the statements from Microsoft at their announcement during the Build conference, these new PCs outperform Apple’s current MacBooks in performance and battery life.
58% faster than MacBook Air M3 in sustained multithreaded performance
23% faster than MacBook Air M3 in peak multithreaded performance
20% more battery life than latest MacBook Air 15”
The first wave of Copilot+ PCs will be available on June 18. These devices will be offered at an affordable price starting at $999. There are with the Surface Pro and Surface Laptop, two devices directly from Microsoft.
Furthermore, Microsoft’s OEM partners Acer, ASUS, Dell, HP, Lenovo, and Samsung created versions of Copilot+ PCs with the same ARM-based processor. Intel and AMD Copilot+ PCs are announced to follow at a later stage. To get a better overview of the launch options, take a look at the video covered in the media recommendation below.
My take: I was impressed since their initial release by the performance of Macs powered by their ARM-based M-chips. These bring a compatible capability to classical CPUs at way more efficient power consumption. They also reduce the demand for cooling and bring with that a more silent work experience. On top, the integrated NPUs are perfectly suited to run LLMs locally, only limited by the size of the system memory. Finally, these advantages of ARM-based chips also comes to Windows and likely, in a near future, also the Linux operating system.
Find Anything You've Seen on Your PC with Microsoft Recall
Microsoft announced a new Windows feature which empowers users to find things they've seen on their PC by searching inside automatically captured screenshots.
Users can search by keywords or simply scroll through a timeline of their activity.
This feature will initially only be available on the new Copilot+ PCs. It is stated that screenshots are saved securely locally and also the AI runs on the device to ensure data privacy.
Users have control over what gets saved and can filter out specific websites or apps. Also, manual deletions of saved screenshots are possible.
My take: There is already a lot of negative press about this feature, which states that this is a privacy nightmare. As the feature is announced to run locally and the stored screenshots are encrypted, I am fine with it. However, it is unfortunate that, according to Microsoft, it will be turned on by default. If this is really the case, many people will get a perfect recording of what they have done on the PC without being aware of it. This reminds me of the location tracking in Android phones, which created a perfect history of the users movements - for many without being aware that this is the case.
Microsoft’s New Phi-3 Models Power On-Device Workflows
After the initial release of the Phi-3-mini model with 3.8B parameters, which was covered in an earlier issue, Microsoft released the following new models:
Phi-3-small with 7B parameters
Phi-3-medium with 14B parameters
Phi-3-vision with 4.2B parameters supporting text and image inputs
As stated in the corresponding blog article, the Phi-3 family of models offers strong reasoning at low compute requirements. With that, they are the models to power the new generation of Copilot + PCs, which are promoted for their capabilities to run generative AI models locally.
According to many benchmarks, the Phi-3 models outperform other models of the same size. Interestingly, the version of Phi-3-mini with only 4k tokens in context size outperforms the 128k token context size version.
My take: The development of smaller, high-performing models like the Phi-3 family is a significant step towards ensuring data privacy and control. Running these models locally on your own devices is the best way to guarantee that personal data is not used for purposes without explicit consent.
🌟 Media Recommendation
The Hidden Details of Copilot+ PCs: Alex Ziskind's Reveals Chip Variants
Alex Ziskind discussed in a YouTube video the announcement of the new Copilot+ PCs and the available models from different vendors.
The Snapdragon X Elite chip comes in multiple SKUs (models) with varying performance. It is not always stated by the manufacturer which version is used.
My take: I discovered the channel from Alex Ziskind during my research about the new Copilot+ PCs. In contrast to many other videos, he added additional information and not only repeated the highlights of the Microsoft Build keynote. Personally, I am looking forward to his announced tests of the new devices with Linux as this option will lead to an even higher privacy and maybe even better local LLM performance as it currently is the case for NVIDIA GPUs.
Disclaimer: This newsletter is written with the aid of AI. I use AI as an assistant to generate and optimize the text. However, the amount of AI used varies depending on the topic and the content. I always curate and edit the text myself to ensure quality and accuracy. The opinions and views expressed in this newsletter are my own and do not necessarily reflect those of the sources or the AI models.