einops : A new flavour of deep learning operations. ( https://github.com/arogozhnikov/einops )
exllamav2 : Inference library for running local LLMs on consumer hardware. ( https://github.com/turboderp/exllamav2 )
exllamav3 : Inference library for running local LLMs on consumer hardware. ( https://github.com/turboderp/exllamav3 )
flash-attn : Flash Attention: Fast and Memory-Efficient Exact Attention (Python component). ( https://github.com/Dao-AILab/flash-attention )
Add an ebuild in portage :
The ebuild is now in the portage tree.
You can also use layman : emerge layman then layman -a tatsh-overlay
For Paludis use this rsync : rsync://gentoo.zugaina.org/tatsh-overlay-portage
If you have a problem : ycarus(-at-)zugaina.org