opendawg commited on
Commit
2151f97
·
verified ·
1 Parent(s): 79de50a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -14
README.md CHANGED
@@ -14,27 +14,25 @@ pinned: false
14
 
15
  [OddRam is a *space* created for collecting [OpenDawg](http://opendawg.com)'s research on *Guardrails*, *Safety*, *Ethics* & *Morals* for **LLM**s.]
16
 
17
- Although LLMs were initially issued without particular attention to safety, the advent of widely used apps without particular attention paid to age verification or jurisdiction, has led to a need for some degree of model safety.
18
 
19
- e.g. Even notes around fairly recent releases of LLama models suggest it is up to the developer/user to add guardrails themselves, such as LLamaGuard[1].
20
 
21
- Whilst "Instruct" variations of LLMs generally have some additional training and tuning to encourage the model to act as a helpful assistant, rather than as an auto-complete or 'Raw' model, LLMs were perceived as not sufficiently trained to shut down many queries about unsafe, unsavoury or illegal subjects.
22
 
23
- This appears to have led to different approaches from the model releasing community:
24
  1. **Fix The Training data**.
25
  New models may have extensive "cleaning" of training data sets to remove unwanted inputs. This may be done manually or synthetically, probably mostly the latter. Essentially trying to mitigate the 'garbage in garbage out' syndrome.
26
  2. **Guards**.
27
- LLama Guard makes reference to Anthropic's safety training research[2], using their 'Red book'[3]\* & other data to fine-tune train their own LLM to help warn application developers of 'unsafe' conversations. This had the issue of requiring multiple LLMs to be running, though newer models could add specific Guard layers in a similar way that they add the image or audio functionality layers. Also, some issues may not have been addressed around multi-language safety as early data sets were predominantly in English. See [CultureGuard](https://arxiv.org/abs/2508.01710v4)[4] for an Nvidia researched partial solution.
28
  3. **Programmatic Solutions**.
29
- Other traditional programmatic methods of cleaning language and 'hot' words, but not using AI. e.g. This might include 'clean lists' of approved words.
30
- 4. **Fine Tuning**.
31
- Make variations of core models that have had extensive tuning against moderated data sets in specialist areas. You have to do this each time a base model gets an update, but e.g. LoRA is generally much quicker and far less compute intensive than training a new model from scratch because of initial training data issues.
32
- 5. **Context injections**.
33
- As maximum context window sizes expand and the major AI players start to monetize 'token burn', injecting few shot style training in the context window can both rapidly direct conversations away from hot topics and reduce unwanted output whilst increasing the bottom line $$$. It is certainly easier than running fine-tuning training repeatedly each time a model gets updated. This can be seen as a 'Few Shot' approach, but that term could also apply to a very limited LoRA training session.
34
- 6. **Zero shot**.
35
- If your user query includes specific instructions and examples, you may be able to guide the conversation response, although useless for corporate responsibility for user safety it may still be helpful for when you want answers with specific morals & ethics considered.
36
-
37
- If you need an extra 'Guardrail' it probably depends on the level of built-in training for a model. Helpfully, research resulting in tools like [SweEval](https://arxiv.org/abs/2505.17332v1)[5] helps you evaluate a model's built in performance on dealing with swear words.
38
 
39
  > \*Disclaimer: The data (especially the harmlessness preference data and the red team data) contain content that may be offensive or upsetting. Topics include, but are not limited to, discriminatory language and discussions of abuse, violence, self-harm, exploitation, and other potentially upsetting subject matter. Please only engage with the data in accordance with your own personal risk tolerance.
40
 
 
14
 
15
  [OddRam is a *space* created for collecting [OpenDawg](http://opendawg.com)'s research on *Guardrails*, *Safety*, *Ethics* & *Morals* for **LLM**s.]
16
 
17
+ Although LLMs were initially issued without particular attention to safety, the advent of widely used apps has led to a need for some degree of model safety.
18
 
19
+ e.g. Even notes around fairly recent releases of LLama models suggest it is up to the LLM app developer to add guardrails themselves, such as LLamaGuard[1].
20
 
21
+ "Instruct" variations of LLMs generally have some additional training and tuning to encourage the model to act as a helpful assistant, rather than simply as an auto-complete bot or 'Raw' model but these LLMs were perceived as not sufficiently trained to shut down many queries about unsafe, unsavoury or illegal subjects.
22
 
23
+ Whilst researching we have different severasl approaches to addressing these issues from the model releasing community:
24
  1. **Fix The Training data**.
25
  New models may have extensive "cleaning" of training data sets to remove unwanted inputs. This may be done manually or synthetically, probably mostly the latter. Essentially trying to mitigate the 'garbage in garbage out' syndrome.
26
  2. **Guards**.
27
+ The LLamaGuard[1] paper makes reference to Anthropic's safety training research[2], and using their 'Red book'[3]\* & other data the researchers fine-tune trained their own LLM to help warn LLM users (and App developers) of 'unsafe' conversations. This approach has the issue of requiring multiple LLMs to be running, though newer models could add specific Guard layers in a similar way that they add the image or audio functionality layers. Also, some issues may not have been addressed around multi-language safety as early data sets were predominantly in English. See [CultureGuard](https://arxiv.org/abs/2508.01710v4)[4] for an Nvidia researched partial solution.
28
  3. **Programmatic Solutions**.
29
+ Other traditional programmatic methods of cleaning language and 'hot' words, but not using AI. e.g. This might include 'clean lists' of approved words or 'black lists' of language to be scrubbed.
30
+ 4. **Context injections**.
31
+ As maximum context window sizes expand and the major AI players start to monetize 'token burn', injecting 'few shot' style training within the context window can both rapidly direct conversations away from hot topics and reduce unwanted output whilst increasing the bottom line token burn $$$. It is certainly easier than running fine-tuning training repeatedly each time a model gets updated. This can be seen as a 'Few Shot' approach, but that term could also apply to a very limited LoRA training session.
32
+ 5. **Zero shot**.
33
+ If your user query includes specific instructions and examples, you may be able to guide the conversation response, although useless for serious corporate responsibility, this user 'self-safety' approach may still be helpful when you want answers with specific morals & ethics considered.
34
+
35
+ Needing an extra 'Guardrail' will probably depend on the level of built-in 'safety' training. e.g. [SweEval](https://arxiv.org/abs/2505.17332v1)[5] helps you evaluate a model's built in performance on dealing with swear words and thus may assist in identifying guardrail needs.
 
 
36
 
37
  > \*Disclaimer: The data (especially the harmlessness preference data and the red team data) contain content that may be offensive or upsetting. Topics include, but are not limited to, discriminatory language and discussions of abuse, violence, self-harm, exploitation, and other potentially upsetting subject matter. Please only engage with the data in accordance with your own personal risk tolerance.
38