Customize Consent Preferences

We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.

The cookies that are categorized as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... 

Always Active

Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

No cookies to display.

Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

No cookies to display.

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

No cookies to display.

Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.

No cookies to display.

Advertisement cookies are used to provide visitors with customized advertisements based on the pages you visited previously and to analyze the effectiveness of the ad campaigns.

No cookies to display.

Microsoft Rolling Out Copilot Vision in Preview, Can Understand User’s Browsing Activity

Share

Microsoft introduced a new artificial intelligence (AI) capability for Copilot on Thursday. Dubbed Copilot Vision, it now enables the AI chatbot to see and understand the context of what a user does online. The Redmond-based tech giant said that Copilot can access the Microsoft Edge browser. The vision-enabled chatbot can now monitor which websites and web pages the user browses, and then can perform certain commands when prompted. This capability is currently available in preview to a select number of Copilot Pro subscribers in the US.

Microsoft Copilot Vision

The Windows maker has been trying to bring its AI-powered chatbot Copilot to the forefront of the PC experience. Earlier, it unveiled purpose-built AI agents to complete tasks on the PC, and now, it has added another capability to Copilot that will let it answer user queries based on their browser-related activity, as per the blog post. Notably, Copilot Vision is not an agentic function, so the AI cannot take actions on behalf of the user.

Copilot Vision can understand the full context of what a user sees on the Edge browser. It can see the images and elements, read the text, and even comprehend the navigation buttons. The vision-enabled chatbot will be present at the bottom of the screen and users can either type or speak the prompt.

Coming to what the AI can do, it can answer questions, summarise and translate text, and spotlight certain parts of the page when prompted to do so. Microsoft states that Copilot Vision can also act as a game assistant and help a user understand how to play games like GeoGuessr, and offer tips while playing a game of Chess online.

Highlighting examples, the tech giant said Copilot Vision can help users plan an outing at the museum by pointing out the important information a user should know. It can also help users with shopping by recommending the right products based on the user’s needs and preferences.

On privacy and security, Microsoft stated that Copilot Vision will be entirely opt-in and users can decide when to turn it on and when to keep it turned off. The AI will not be able to see something that the user does not want to. Additionally, any conversation data with the AI will be deleted at the end of each session. However, the Copilot’s responses are logged and they will be used to improve the company’s safety systems.

Microsoft further added that Copilot Vision does not capture, store, or use any data from publishers to train its AI models. It is also working with a group of external testers (red teamers) to improve the safety of the new feature.

Copilot Vision is currently rolling out to a limited number of Copilot Pro subscribers in the US and it can be accessed via Copilot Labs. Notably, Vision will currently work only on a select set of websites, and the functionality will be expanded gradually.

Affiliate links may be automatically generated – see our ethics statement for details.