PyTorch is a machine learning library based on the Torch library, used for applications such as computer vision and natural language processing. It is recognized as one of the two most popular machine learning libraries alongside TensorFlow, offering free and open-source software released under the modified BSD license.

An open source machine learning framework that accelerates the path from research prototyping to production deployment. PyTorch is an open source project at the Linux Foundation.

PyTorch

Upcoming webinar: Dinosaur Bone Hunting with Intel AI! 🦖🔎 Join us for a live, hands on workshop on February 6.

Details + registration: https://hubs.la/Q02jwmrk0

Check out what's new in PyTorch Documentation 👀 Featuring 3 new tutorials and PyTorch 2.2 documentation + tutorials!

Details 👉  https://hubs.la/Q02jwrbr0

PyTorch 2.2 is here 🎉

Featuring: 
- SDPA support of FlashAttention-2
- New ahead-of-time extension of TorchInductor
- device_mesh, a new abstraction for initializing and representing ProcessGroups 
- A standardized, configurable logging mechanism called TORCH_LOGS

More announcement details in our latest blog: https://hubs.la/Q02jc6kB0

With this release, we've also added new libraries. 📚 These updates demonstrate our focus on developing common and extensible APIs across all domains to make it easier for our community to build ecosystem projects on PyTorch. 🔥

Check out the library updates here: https://hubs.la/Q02jc4gH0

Colossal-AI has achieved remarkable results on 7B LLaMA-2 model by utilizing only 0.0085 trillion tokens of data, investing 15 hours, and incurring a few hundred training costs! 💯🔥

Read more: https://hubs.la/Q02j0Chq0

Check it out on GitHub: https://hubs.la/Q02j0wkY0 

Happy to share an early version of float8_experimental, a library to accelerate training with float8 + torch.compile written in native PyTorch: https://lnkd.in/ghsfYSJr 🔥 

NVIDIA H100s support float8, with theoretical advantages of 2x in throughput and memory usage over 16-bit formats. We aim to make it easy to use float8 in PyTorch.

Per-tensor scaling is commonly used to maintain accuracy, introducing significant overhead in the absence of fused kernels.

We use torch.compile to generate fast kernels for per-tensor scaling, achieving speedups of up to 1.20x on pre-training LLaMa 7B benchmarks on one H100-SXM GPU.

What about distributed? FSDP with 16-bit all-gather is supported today. 

Support for tensor parallel, sequence parallel and FSDP with 8-bit all-gather is planned. 

Learn more here 👉  https://lnkd.in/gp_McA_S

ORG:PYTORCH

PyTorch

Analysis Not Available