At GroundedAI, we believe in the responsible development of artificial intelligence (AI) through rigorous evaluation frameworks. Our mission is to push the boundaries of generative AI while ensuring it remains grounded in ethical principles and transparency.
This repository houses a collection of Jupyter Notebooks that demonstrate how to leverage our small language models as judges for evaluating large language model (LLM) applications. By using these lightweight models, we aim to provide a scalable and cost-effective solution for assessing the quality, safety, and adherence to ethical standards of LLM outputs.
- Evaluation Notebooks: Step-by-step guides on using small language models for evaluating LLM applications, covering tasks such as toxicity detection, rag-relevance-etc.
- Pretrained Models: Access to pretrained small language models optimized for critical benchmarks, ready for fine-tuning or direct use in evaluation pipelines.
- Customization: Flexibility to adapt the evaluation models and techniques to your specific domain or use case.
- Open Source: Transparent and collaborative development, enabling the global AI community to contribute and build upon our work.
To get started with GroundedAI's evaluation notebooks, follow these steps:
- Clone the repository:
git clone https://github.com/grounded-ai/slm-evaluator-cookbook.git- Install dependencies:
pip install transformers accelerate peft flash-attn- Explore the notebooks:
Open the Jupyter Notebook of your choice. Each notebook provides detailed prompt instructions and code examples for getting started evaluating LLM applications using small language models.
- Documentation: Keep up with our progress at https://groundedai.tech/
- Discord: Join our Discord server to engage with the GroundedAI community, get support, and stay updated on the latest developments.
This project is licensed under the MIT License.
At GroundedAI, we are committed to fostering a vibrant ecosystem for responsible AI development. By providing accessible evaluation tools and promoting transparency, we aim to shape the future of generative AI, grounded in ethical principles and fueled by collaborative efforts with the global AI community.