Watch the Low-Code/No-Code Summit on-demand sessions to learn how to successfully innovate and achieve efficiencies by upskilling and scaling citizen developers. Watch now.
One of the most widely used machine learning (ML) technologies today is the open-source PyTorch framework.
PyTorch started in 2016 at Facebook (now known as Meta) with release 1.0 debuting in 2018. In September 2022, Meta moved the PyTorch project to the new PyTorch Foundation, which is managed by the Linux Foundation. Today, PyTorch developers took the next big step forward for PyTorch by announcing the first experimental release of PyTorch 2.0. The new release promises to help accelerate ML training and development while maintaining backward compatibility with existing PyTorch application code.
“We’ve added an extra function called ‘torch.compile’ that users need to re-insert into their codebases,” said Soumith Chintala, lead maintainer, PyTorch. told VentureBeat. “We’re calling it 2.0 because we think users will find it an important new addition to the experience.”
Contents
The new compiler in PyTorch that makes a difference for ML
There have been discussions in the past about when the PyTorch project should invoke a new release 2.0.
For example, in 2021 there was a brief discussion of whether PyTorch 1.10 should be labeled as release 2.0. Chintala said PyTorch 1.10 didn’t have enough fundamental changes from 1.9 to warrant a large number of upgrades to 2.0.
The most recent generally available release of PyTorch is version 1.13, which came out at the end of October. A key feature in that release came from an IBM code contribution that allowed the machine learning framework to work more effectively with standard Ethernet-based networks for large-scale workloads.
Chintala stressed that now is the right time for PyTorch 2.0, as the project introduces an additional new paradigm to the PyTorch user experience, called torch.compile, that delivers solid speedups for users that were not possible in PyTorch 1.0’s default enthusiast mode.
He explained that on about 160 open-source models on which the PyTorch project validated early builds of 2.0, there was a 43% speed-up and they worked reliably with the addition of one line to the codebase.
“We expect people with PyTorch 2 to change the way they use PyTorch on a daily basis,” said Chintala.
He said that with PyTorch 2.0, developers will start their experiments with eager mode and, once they have trained their models for long periods of time, will activate compiled mode for additional performance.
“Data scientists will be able to do the same things with PyTorch 2.x as they did with 1.x, but they can do them faster and at a larger scale,” Chintala said. “If your model was training for 5 days and with 2.x compiled mode it now trains in 2.5 days, you can iterate more ideas in this extra time or build a larger model that will train in the same 5 days .”
More Python coming to PyTorch 2.x
PyTorch gets the first part of its name (Py) from the open-source Python programming language widely used in data science.
However, modern PyTorch releases are not written entirely in Python, as parts of the framework are now written in the C++ programming language.
“Over the years we have moved many parts of the building torch.nn from Python to C++ to squeeze that last-mile performance,” Chintala said.
Chintala said the PyTorch project expects to move code related to torch.nn back to Python in the later 2.x series (but not 2.0). He noted that C++ is usually faster than Python, but the new compiler (torch.compile) is ultimately faster than running the equivalent code in C++.
“Moving these parts to Python improves hackability and lowers the threshold for code contributions,” said Chintala.
Work on Python 2.0 will be ongoing over the next few months, with general availability not expected until March 2023. In addition to the development effort, PyTorch is transitioning from being controlled and operated by Meta to being its own independent effort.
“It’s early days for the PyTorch Foundation, and you’ll be hearing more on a longer time horizon,” Chintala said. “The foundation is in the process of making several transfers and establishing goals.”
VentureBeat’s mission is to become a digital city plaza where tech decision makers can learn about transformative business technology and execute transactions. Discover our Briefings.
Janice has been with businesskinda for 5 years, writing copy for client websites, blog posts, EDMs and other mediums to engage readers and encourage action. By collaborating with clients, our SEO manager and the wider businesskinda team, Janice seeks to understand an audience before creating memorable, persuasive copy.