Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat] Request response refusal validator #1189

Open
railsstudent opened this issue Dec 9, 2024 · 1 comment
Open

[feat] Request response refusal validator #1189

railsstudent opened this issue Dec 9, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@railsstudent
Copy link

Description
Request a validator that determines whether or not a LLM refuses a prompt and generates an output that starts with texts such as "I cannot", "I can't", and "It is illegal"

Why is this needed
If the response is refused, it should not be returned to the client application to display. The validator should throw a validation error that the application should handle appropriately.

Implementation details
I suppose Huggingface provides models for response refusal checking.

End result
After a LLM generate a text, the validator is used to validate whether the response is refused by having but not limited to texts such as "I can not", "I can't", "It is not legal", etc.

@railsstudent railsstudent added the enhancement New feature or request label Dec 9, 2024
@JosephCatrambone
Copy link
Contributor

Ooh. That's a neat idea. It's not on the current sprint listing but I'd like to take a swing at it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants