k2-fsa/OmniVoice
Text-to-Speech • Updated • 2.21M • 837
Mert Bozkurt
mertbozkurt
AI & ML interests
None yet
Recent Activity
liked a model 3 days ago
k2-fsa/OmniVoice liked a model 6 days ago
pthinc/Asena_ESP32 liked a model 7 days ago
cihatyldz/sifahane-bert-turkish-medicalOrganizations
liked a
model 3 days ago
liked a
model 6 days ago
liked 2
models 7 days ago
upvoted an article 17 days ago
Article
Darwin-TTS: We Gave a TTS Model 3% of an LLM's Brain — It Started Showing Emotion
•
13
liked a
model 27 days ago
liked a
model about 2 months ago
updated a
dataset 3 months ago
published a
dataset 3 months ago
updated a
model 3 months ago
published a
model 3 months ago
updated a
model 3 months ago
published a
model 3 months ago
reacted to ZennyKenny's post with 🚀 4 months ago
Post
3253
😎 My new personal website is live! Check out https://kennethhamilton.me to chat with an LLM about my professional skills and personal projects.
🙈 Think of it like a really, really vain version of ChatGPT.
🙈 Think of it like a really, really vain version of ChatGPT.
- 6 replies
liked 2
datasets 4 months ago
liked a
model 5 months ago
reacted to flozi00's post with ❤️ 5 months ago
Post
3588
When models get too large for a single GPU, simply stacking layers vertically (Pipeline Parallelism) isn't always the answer. Sometimes, you need to slice the matrices themselves.
My latest guide breaks down the hardware mechanics of Tensor Parallelism (TP). We look at how to shard individual operations across devices to make a cluster function as one massive accelerator.
This isn't high-level theory—it is a look at the bare metal implementation.
Here is what is covered in the deep dive:
The Strategies: Column vs. Row Parallelism
We analyze how to split weight matrices (W) and inputs (X).
Column-Linear: Splits weights by columns. Requires an All-Gather to reconstruct the output.
Row-Linear: Splits weights by rows. Requires an All-Reduce to sum partial results.
The "Megatron-LM" Optimization
Efficiency comes from minimizing communication. By sandwiching the non-linearity (GeLU) between a Column-Parallel layer and a Row-Parallel layer, we can skip synchronization entirely during the activation phase. This cuts communication events by 50% per block.
The Hardware Reality: The Bandwidth Wall
In TP, the dist.all_reduce operation sits on the critical path. The CUDA cores effectively stall while waiting for the ring-reduce to finish.
Intra-Node: Works well because NVLink provides enough bandwidth to hide this latency.
Inter-Node: Fails at scale. Standard networking (Ethernet/InfiniBand) is too slow for the high-frequency syncs required by TP.
The article includes a raw PyTorch implementation using torch.distributed primitives to show exactly where the data moves and where the bottlenecks sit.
Read the full hardware-centric guide here:
https://flozi.net/en/guides/ai/scaling/tensor_parallel
My latest guide breaks down the hardware mechanics of Tensor Parallelism (TP). We look at how to shard individual operations across devices to make a cluster function as one massive accelerator.
This isn't high-level theory—it is a look at the bare metal implementation.
Here is what is covered in the deep dive:
The Strategies: Column vs. Row Parallelism
We analyze how to split weight matrices (W) and inputs (X).
Column-Linear: Splits weights by columns. Requires an All-Gather to reconstruct the output.
Row-Linear: Splits weights by rows. Requires an All-Reduce to sum partial results.
The "Megatron-LM" Optimization
Efficiency comes from minimizing communication. By sandwiching the non-linearity (GeLU) between a Column-Parallel layer and a Row-Parallel layer, we can skip synchronization entirely during the activation phase. This cuts communication events by 50% per block.
The Hardware Reality: The Bandwidth Wall
In TP, the dist.all_reduce operation sits on the critical path. The CUDA cores effectively stall while waiting for the ring-reduce to finish.
Intra-Node: Works well because NVLink provides enough bandwidth to hide this latency.
Inter-Node: Fails at scale. Standard networking (Ethernet/InfiniBand) is too slow for the high-frequency syncs required by TP.
The article includes a raw PyTorch implementation using torch.distributed primitives to show exactly where the data moves and where the bottlenecks sit.
Read the full hardware-centric guide here:
https://flozi.net/en/guides/ai/scaling/tensor_parallel
upvoted an article 6 months ago
Article
Continuous batching from first principles
- +1
•
378
reacted to flozi00's post with 👍 6 months ago
Post
2775
Running large language models efficiently is more than just raw GPU power. The latest guide breaks down the essential math to determine if your LLM workload is compute-bound or memory-bound.
We apply these principles to a real-world example: Qwen's 32B parameter model on the new NVIDIA RTX PRO 6000 Blackwell Edition.
In this guide, you will learn how to:
Calculate your GPU's operational intensity (Ops:Byte Ratio)
Determine your model's arithmetic intensity
Identify whether your workload is memory-bound or compute-bound
Read the full guide here: https://flozi.net/en/guides/ai/llm-inference-math
We apply these principles to a real-world example: Qwen's 32B parameter model on the new NVIDIA RTX PRO 6000 Blackwell Edition.
In this guide, you will learn how to:
Calculate your GPU's operational intensity (Ops:Byte Ratio)
Determine your model's arithmetic intensity
Identify whether your workload is memory-bound or compute-bound
Read the full guide here: https://flozi.net/en/guides/ai/llm-inference-math