remove example pt file

725159424.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:eeb05bde43937f346fae4d7cf6152021187dc7dcaef6471506b255e0fd5ef647
-size 1610891985

README.md

@@ -37,6 +37,37 @@ Training logs are available [via wandb](https://wandb.ai/lewington/ViT-L-14-laio
 
 ## Usage
 
+```python
+import PIL
+from clipscope import ConfiguredViT, TopKSAE
+
+device='cpu'
+filename_in_hf_repo = "725159424.pt"
+sae = TopKSAE.from_pretrained(repo_id="lewington/CLIP-ViT-L-scope", filename=filename_in_hf_repo, device=device)
+
+transformer_name='laion/CLIP-ViT-L-14-laion2B-s32B-b82K'
+locations = [(22, 'resid')]
+transformer = ConfiguredViT(locations, transformer_name, device=device)
+
+input = PIL.Image.new("RGB", (224, 224), (0, 0, 0)) # black image for testing
+
+activations = transformer.all_activations(input)[locations[0]] # (1, 257, 1024)
+assert activations.shape == (1, 257, 1024)
+
+activations = activations[:, 0] # just the cls token
+# alternatively flatten the activations
+# activations = activations.flatten(1)
+
+print('activations shape', activations.shape)
+
+output = sae.forward_verbose(activations)
+
+print('output keys', output.keys())
+
+print('latent shape', output['latent'].shape) # (1, 65536)
+print('reconstruction shape', output['reconstruction'].shape) # (1, 1024)
+```
+
 ## Error Formulae
 
 We calculate MSE as `(batch - reconstruction).pow(2).sum(dim=-1).mean()` i.e. The MSE between the batch and the un-normalized reconstruction, summed across features. We use batch norm to bring all activations into a similar range.