Devin, a generative artificial intelligence (AI) model that can function as a software engineer, was introduced by the AI startup Cognition Labs. The company has claimed that Devin has successfully passed practical engineering interviews from AI companies and has even completed real jobs on Upwork. The AI tool comes with its shell, a code editor, and a browser to perform complex engineering tasks such as completing end-to-end coding projects, building and deploying websites and apps, and even training and fine-tuning its own AI models.
Cognition Labs unveiled the AI model in a post on X (formerly Twitter) and hailed it as the “first software engineer”. Making the announcement, the startup said, “Devin is the new state-of-the-art on the SWE-Bench coding benchmark, has successfully passed practical engineering interviews from leading AI companies, and has even completed real jobs on Upwork.”
The AI model comes equipped with its shell or interface, an inbuilt code editor to write and deploy codes, and a browser within a sandboxed computing environment that enables it to perform complex engineering tasks. In a blog post, the company delved deeper into its capabilities. As per the post and multiple video demonstrations, Devin can learn to use unfamiliar technologies, build and deploy apps end-to-end, autonomously find and fix bugs in codebases, address bugs and feature requests in open-source repositories, contribute to mature production repositories, and even train and fine-tune its own AI models.
Additionally, Devin AI also scored 13.86 percent on the SWE-bench coding benchmark. Not only did it massively outperform other major AI models such as Claude 2 which scored 4.80 percent and GPT-4 which scored 1.74 percent, but the company claims it was able to resolve issues unassisted. Notably, all other AI models were assisted and were told exactly which files needed to be edited.
While Cognition has made tall claims, they cannot be verified at the moment as the platform is not available in the public domain. The startup has also not released a detailed technical report about the AI model, although it stated that it will be released soon. However, if the claims are true, Devin the AI model has created a new standard in the AI-powered code generation space. So far, all coding-centric models are assistive in nature and can only perform tasks based on the prompts and in limited capacity. Devin, however, can not only work autonomously but also handle end-to-end projects. The pressing question is whether it can replace a human software engineer or not.
Devin is currently in early access, but the developers have said that people looking to hire the AI model for engineering work can reach out to them.
