cdpearlman commited on
Commit
67f5be3
·
1 Parent(s): e248349

Added numbers to BertViz visualization, easier to map attention head numbers

Browse files
Files changed (2) hide show
  1. plans.md +2 -2
  2. utils/model_patterns.py +25 -0
plans.md CHANGED
@@ -1,5 +1,4 @@
1
  To Do:
2
- - generating with 1 output token doesn't show the "selected" beam, so users can't see the output from the model
3
  - add head index numbers to color swatches in BertViz visualization graph
4
  - clicking send in the chatbot doesn't immediately show the purple text bubble from the user and the typing animation, there are a couple seconds of nothing. It should immediately populate the text bubble and start the typing animation
5
 
@@ -13,4 +12,5 @@ Done:
13
  - shorter, concise responses in system prompt
14
  - add video links to glossary
15
  - three blue one brown
16
- - specs on what each attention head does
 
 
1
  To Do:
 
2
  - add head index numbers to color swatches in BertViz visualization graph
3
  - clicking send in the chatbot doesn't immediately show the purple text bubble from the user and the typing animation, there are a couple seconds of nothing. It should immediately populate the text bubble and start the typing animation
4
 
 
12
  - shorter, concise responses in system prompt
13
  - add video links to glossary
14
  - three blue one brown
15
+ - specs on what each attention head does
16
+ - generating with 1 output token doesn't show the "selected" beam, so users can't see the output from the model
utils/model_patterns.py CHANGED
@@ -1383,6 +1383,31 @@ def generate_bertviz_html(activation_data: Dict[str, Any], layer_index: int, vie
1383
  'headColors = d3.scale.category10();',
1384
  _patch
1385
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1386
  return html_str
1387
 
1388
  except Exception as e:
 
1383
  'headColors = d3.scale.category10();',
1384
  _patch
1385
  )
1386
+
1387
+ # Inject head-index labels inside the checkbox swatches.
1388
+ # Target the first `updateCheckboxes();` call inside drawCheckboxes
1389
+ # and prepend D3 code that appends <text> elements over each rect.
1390
+ _label_js = (
1391
+ 'checkboxContainer.selectAll("text")\n'
1392
+ ' .data(config.headVis)\n'
1393
+ ' .enter()\n'
1394
+ ' .append("text")\n'
1395
+ ' .text((d, i) => i)\n'
1396
+ ' .attr("x", (d, i) => i * CHECKBOX_SIZE + CHECKBOX_SIZE / 2)\n'
1397
+ ' .attr("y", top + CHECKBOX_SIZE / 2)\n'
1398
+ ' .attr("text-anchor", "middle")\n'
1399
+ ' .attr("dominant-baseline", "central")\n'
1400
+ ' .attr("font-size", "10px")\n'
1401
+ ' .attr("font-weight", "bold")\n'
1402
+ ' .attr("fill", "white")\n'
1403
+ ' .attr("pointer-events", "none");\n'
1404
+ ' updateCheckboxes();'
1405
+ )
1406
+ html_str = html_str.replace(
1407
+ 'updateCheckboxes();\n\n checkbox.on',
1408
+ _label_js + '\n\n checkbox.on',
1409
+ 1, # replace only the first occurrence
1410
+ )
1411
  return html_str
1412
 
1413
  except Exception as e: