triton layer norm EU AI Act Compliance Profile

Triton layer normalization kernels.

This kernel implements layers normalization using Triton. This kernel is from the flash-attention project.

Functions

Function layer_norm

(x: torch.Tensor, weight: torch.Tensor, bias: torch.Tensor, residual: Optional[torch.Tensor] = None, x1: Optional[torch.Tensor] = None, weight1: Optional[torch.Tensor] = None, bias1: Optional[torch.Tensor] = None, eps: float = 1e-06, dropout_p: float = 0.0, rowscale=None, prenorm: bool = False, residual_in_fp32: bool = False, is_rms_norm: bool = False, return_dropout_mask: bool = False, out: Optional[torch.Tensor] = None, residual_out: Optional[torch.Tensor] = None)

Apply layer normalization to the input tensor with Triton acceleration.

Parameters

• x (torch.Tensor) -- Input tensor to normalize.
• weight (torch.Tensor) -- Scale parameter for normalization.
• bias (torch.Tensor) -- Shift parameter for normalization.
• residual (torch.Tensor, optional) -- Optional residual tensor to add to the input before normalization.
• x1 (torch.Tensor, optional) -- Optional second input tensor to combine with x. When provided, the function first adds x1 to x and then applies normalization.
• weight1 (torch.Tensor, optional) -- Scale parameter for the second normalization.
• bias1 (torch.Tensor, optional) -- Shift parameter for the second normalization.
• eps (float, optional, defaults to 1e-6) -- Small constant added for numerical stability in normalization.
• dropout_p (float, optional, defaults to 0.0) -- Dropout probability. If greater than 0, applies dropout to the input before normalization and residual addition.
• rowscale (torch.Tensor, optional) -- Optional scaling factor applied to each row of the input tensor. Not compatible with the use of x1.
• prenorm (bool, optional, defaults to False) -- If True, returns both

Your obligations as a triton layer norm deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library