Efficiently train models with large sequence lengths using Amazon SageMaker model parallel
CAT
1 min read
In this post, we demonstrate how the Amazon SageMaker model parallel library (SMP) addresses this need through support for new features such as 8-bit floating point (FP8) mixed-precision training for accelerated training performance and context parallelism for processing large input sequence lengths, expanding the list of its existing features.
Originally appeared here:
Efficiently train models with large sequence lengths using Amazon SageMaker model parallel
