README.md

---
license: mit
library_name: pytorch
tags:
  - speech
  - semantic-communication
  - joint-source-channel-coding
  - audio
  - reproduction
datasets:
  - JacobLinCool/VoiceBank-DEMAND-16k
pipeline_tag: audio-to-audio
---

# DeepSC-S — Semantic Communication for Speech (reproduction)

PyTorch reproduction of **DeepSC-S** (Weng, Qin & Li, 2021 —
[arXiv:2012.05369](https://arxiv.org/abs/2012.05369)), a neural joint source-channel
coding system for speech: instead of separate compression + error-correction + modulation,
a single network learns to encode speech directly into channel symbols and reconstruct it
at the receiver, trained end-to-end through a simulated noisy channel.

- **Code:** https://github.com/prashantrajbista/semantic-communication
- **Write-up (all 5 stages, with results):** https://prashantrajbista.github.io/semantic-communication/
- **Paper:** [arXiv:2012.05369](https://arxiv.org/abs/2012.05369)
- **License:** MIT

## Checkpoints

Three final-weights checkpoints, each trained with a different simulated channel during
training (`ChannelLayer` in `deepscs/channel.py`):

| file | trained on |
|---|---|
| `deepsc_s_awgn_final.pt` | AWGN |
| `deepsc_s_rayleigh_final.pt` | Rayleigh fading |
| `deepsc_s_rician_final.pt` | Rician fading (K=1) |

All three share the same architecture (`DeepSC_S` in `deepscs/model.py`):

- SemanticEncoder / SemanticDecoder: 4 SE-ResNet blocks, feature depth D=32, 5×5 kernels,
  cardinality 4, SE reduction ratio r=4
- ChannelEncoder / ChannelDecoder: compression knob `depth=8` (8 real channel symbols per
  frame-column position)
- Trained with Adam (lr=1e-3), MSE loss, SNR sampled uniformly in [0, 20] dB per batch

Where the paper underspecifies (kernel size, r, D, Rician K, optimizer), the official
[TensorFlow repo](https://github.com/Zhenzi-Weng/DeepSC-S) was used as ground truth
instead of guessing — see `docs/initial_plan.md` in the code repo for the full list of
flagged choices.

## Training data & scale

Trained on a 2,000-clip subset of the Edinburgh DataShare / VoiceBank-DEMAND clean
28-speaker set (~10k clips available), 40 epochs. This is a deliberate scope choice for a
tractable single-session reproduction, not the paper's full-scale run — expect lower
absolute quality (PESQ ~1.0–1.35) than the paper reports. SDR/PESQ curves and the
matched-vs-mismatched-channel robustness results are in the write-up linked above.

## Usage

```python
import torch
from deepscs.model import DeepSC_S  # from github.com/prashantrajbista/semantic-communication

model = DeepSC_S(depth=8, n_blocks=4, cardinality=4, r=4)
model.load_state_dict(torch.load("deepsc_s_awgn_final.pt", map_location="cpu"))
model.eval()

# x: (B, 1, F, L) framed audio — see deepscs/audio.py for the framing/deframing utilities
# and deepscs/channel.py for ChannelLayer to simulate the transmission channel.
```

## Scope

**In:** neural transceiver, end-to-end MSE training, AWGN + Rayleigh + Rician, SDR
(+ PESQ), cross-channel robustness evaluation.

**Out:** traditional PCM+Turbo/64-QAM baseline, exact numeric paper reproduction, real RF
hardware, estimated-CSI channels, full-scale (~10k clip) training.