🤝 Microsoft's employee monitoring system
Dear curious minds,
As tempting as a perfect digital memory sounds for me as a PKM (Personal Knowledge Management) enthusiast, there are more concerns the new possibilities bring than I initially thought. This issue will give you insights about old plans which Microsoft formulated in a patent and connects them to the recent announcements of the Recall feature. The latter wants to unlock the perfect digital memory for its users in Windows 11.
In this issue:
💡 Shared Insight
From 2007 Patent to Recall: Microsoft's Journey to Workplace Monitoring📰 AI Update
Apple Unveils Apple Intelligence: AI Assistant with a Focus on Privacy
Recall: Microsoft Addresses Flaws in AI-Powered Memory Aid for PCs
Stable Diffusion 3 Medium: Local AI Art Struggles with Human Forms🌟 Media Recommendation
Zoom CEO Envisions that AI-Powered Digital Twins Do Our Work
💡 Shared Insight
From 2007 Patent to Recall: Microsoft's Journey to Workplace Monitoring
A few days back, I was having a conversation with a friend about Microsoft’s new Recall feature. He said he was not surprised at all. He told me that many years ago, Microsoft had already filed a patent detailing how they planned to monitor every activity an office worker does on a PC.
This information made me look for the old patent and got me thinking:
What does this mean for all of us who use Microsoft Windows at work?
The Hidden Monitoring System
Microsoft holds a patent from 2007 named “Monitoring group activities” that outlines a system for monitoring everything you do on your computer at work. This system tracks your activities, logs what you do, and even analyzes your performance. The idea is that colleagues can assist you if you’re struggling with a task. Sounds helpful, right?
The New Recall Feature
Microsoft recently launched the Recall feature, which was initially covered in an earlier issue. There was a lot of negative press and the feature was reworked to some extent as described in the AI update section of this issue.
In a nutshell, the Recall feature takes screenshots of your desktop every few seconds to create a detailed log of your activities. These are stored locally and, so far, only processed locally to help you to rediscover whatever you did on your PC with a simple text search. Sounds also helpful, right?
Misuse of Recall For Monitoring You
Officially, Recall is not meant for monitoring you, but it doesn’t take much to see how it could be used that way. The concept from the old patent and this new feature combined can give your boss a clear picture of what you’re doing all day.
However, there are many downsides:
Privacy Concerns:
Imagine everything you do on your computer being watched, logged, and analyzed. This could include personal emails, private messages, or even your online shopping during lunch breaks.
It’s like having a camera over your shoulder, all the time.
Data Misuse:
This data can be used to micromanage you, reducing your freedom and making you feel like you’re under constant control.
There’s also a risk of your data being hacked, exposing sensitive information.
Increased Stress:
Knowing you’re being watched can make you anxious and stressed, leading to a less productive and enjoyable work environment.
Conclusion
While the new Recall feature can help to improve productivity, there is a risk that it could be exploited to spy on you. Besides hackers getting access to your data, there might even be a scenario where your company uses these tools to get a perfect log of your activities. It's important to be aware of these issues and advocate for transparent and fair use of monitoring tools in the workplace.
📰 AI Update
Apple Unveils Apple Intelligence: AI Assistant with a Focus on Privacy
Apple announced its new AI assistant called Apple Intelligence at WWDC 2024. It can prioritize notifications, provide writing assistance like rewriting and proofreading, and create images.
Apple Intelligence will be rolled out to iPhones 15 and 15 Pro as well as iPads, and Macs with an M1 or newer chips.
There is no concrete release date, but it was stated that it will be part of the iOS 18, iPadOS 18, and macOS Sequoia updates which are scheduled for this fall.
Apple Intelligence processes data on-device or privately in the cloud with models from OpenAI without storing user data. The latter will always ask you for permission to send your request to the OpenAI servers.
Gemini from Google may be added in the future as an alternative for cloud-based requests. This was stated in a post keynote conversation by SVP Craig Federighi.
Siri gets major upgrades with a new visual interface, more natural conversations, and awareness of on-screen content.
A new keyboard feature called Genmoji creates personalized emojis matching any moment or person.
Audio transcription and summaries powered by AI are coming to apps like Notes and Phone.
My take: I like the name Apple Intelligence, which cleverly leads to the well-known abbreviation AI. The emphasis on privacy is appreciated, and the decision to process as much data locally as possible is a step in the right direction. At the same time, the partnership with OpenAI suggests that Apple is not competing to develop the best foundation models. The announcement that the beta release won't be available until fall is disappointing. With half a year in the AI space feeling like an eternity, it remains to be seen if Apple can keep up in the AI race or if they're already too far behind.
Recall: Microsoft Addresses Flaws in AI-Powered Memory Aid for PCs
An earlier issue highlighted the announcement of the not yet released Recall feature from Microsoft. The intention is to enable users to find things they've seen on their PC by searching inside automatically captured screenshots.
The feature was publicly announced on May 21 and there was already on day one a lot of negative press about it. An article by Kevin Beaumont nicely summarizes the security flaws Microsoft Recall has and shows that the current implementation should not be used at all.
As a reaction, Microsoft has shared a statement describing changes for the Recall feature:
Opt-In Choice: Users must proactively choose to enable Recall, which is off by default.
Windows Hello Enrollment: This biometric authentication feature which was introduced with Windows 10 is now required for enabling Recall and viewing the timeline.
Just-In-Time Decryption: Enhanced security with decryption only during authentication using Windows Hello.
Sidenote: Google stated that they are working on a similar feature for their Chromebooks.
My take: The Recall feature aims to provide perfect memory for Windows users by utilizing privacy-friendly AI that leverages local processing and encryption, ensuring user control and security. While the concept sounds promising, its real-world effectiveness and safety remain to be seen. From my personal perspective, the feature is tempting. However, if I were to acquire a Copilot+ PC on its release date of June 18th, I would initially disable it until its realization is proved to be secure.
Stable Diffusion 3 Medium: Local AI Art Struggles with Human Forms
Stability AI released the weights of their Stable Diffusion 3 Medium model, or in short SD3 Medium. The announcement states that the model has only 2 billion parameters, making it ideal to run locally on consumer GPUs.
SD3 Medium boasts several standout features that set it apart from other models:
It produces images with exceptional detail, color, and lighting, resulting in photorealistic outputs as well as high-quality outputs in flexible styles.
The model comprehends long and complex prompts involving spatial reasoning, compositional elements, actions, and styles, thanks to its advanced prompt understanding capabilities.
Typography is another area where SD3 Medium excels, with fewer errors in spelling, letter forming, and spacing.
Fine-tuning capabilities allow for individualized customization using small datasets.
Besides downloading the weights and running the model on your own hardware, you can easily explore its capabilities on this HuggingFace space.
Additionally to the now openly released SD3 Medium version, there are versions which you can only run via API (pricing for usage via API as currently stated on the Stability AI Developer Platform) :
Ultra (8¢, only API)
Core (8¢, only API)
Stable Diffusion 3 Medium (3.5¢, 2B parameters)
Stable Diffusion 3 Large (6.5¢, 8B parameters, only API)
Stable Diffusion 3 Large Turbo (4¢, Turbo variant of SD3 Large, only API)
SDXL 1.0 (0.2-0.6¢, legacy model)
SD 1.6 (0.2-1.0¢, legacy model)
The comparison of different Stable Diffusion versions above is already two months old, and the model behind Stable Diffusion 3 and its Turbo variant are now renamed to Stable Diffusion 3 Large. These models are not perfect at creating human poses, and especially bad at creating realistic hands.
The now released Stable Diffusion 3 Medium, which can be used locally or via API, seems to struggle way more at generating humans. I used the same prompt “Two people doing yoga” and the model failed totally.
My take: The advancements in Stable Diffusion 3 Medium highlight the rapid progress in AI-generated art. The ability to create high-quality images on local hardware is impressive. However, SD3 Medium fails at creating human poses. Historically, open model releases have led to significant community-driven enhancements. I look forward to seeing the innovations that will emerge from this release. Hopefully, these will also fix the ability to create human poses.
🌟 Media Recommendation
Zoom CEO Envisions that AI-Powered Digital Twins Do Our Work
Zoom CEO Eric Yuan was a guest on a recent episode of the Decoder podcast, hosted by Nilay Patel.
Instead of listening to the episode, you might prefer to read the transcript or use it as input for a conversation with a chatbot.
Eric wants to revolutionize virtual meetings by allowing users to send AI-powered "digital twins" to attend meetings on their behalf, freeing up time for other tasks.
These digital twins should also be able to make decisions, respond to emails, and even handle phone calls, potentially changing the way people work and interact.
Eric aims to transform Zoom from a videoconferencing to a complete workplace platform. The goal is to leverage to compete with tech giants like Microsoft and Google. The recent release of Zoom workplace is the first step in this direction.
This vision involves significant investment in AI research and development, and Eric stated that Zoom plans to invest heavily in natural language processing (NLP) and computer vision technologies to enhance its existing products and explore new opportunities.
My take: Big vision, but can we really trust that our digital twins always decide in the same way as we would? What happens if there are multiple copies of digital twins acting on our behalves at the same time - how will they stay in sync? These and other questions arise while thinking about this topic. They are to some extent already covered in the podcast episode, but I am not convinced that this will work out as planned.
Disclaimer: This newsletter is written with the aid of AI. I use AI as an assistant to generate and optimize the text. However, the amount of AI used varies depending on the topic and the content. I always curate and edit the text myself to ensure quality and accuracy. The opinions and views expressed in this newsletter are my own and do not necessarily reflect those of the sources or the AI models.