YiVal is an open-source project designed to revolutionize the way developers and researchers evaluate and refine AI models.
Project description
YiVal
Table of Contents
About
YiVal is an adaptable AI development framework, designed to provide a tailored experimentation experience. Whether you're a hands-on developer or leaning into automation, YiVal is equipped for both:
- Data Input: Choose between manual data input or let the framework handle auto-generation.
- Variations: Manually set parameter and prompt variations or utilize the automated capabilities for optimal settings.
- Evaluation: Engage with manual evaluators or leverage the built-in automated evaluators for efficient results.
On the horizon is the YiVal Agent, an ambitious addition aimed at autonomously driving the entire experimentation process. With its blend of manual and automated features, YiVal stands as a comprehensive solution for AI experimentation, ensuring flexibility and efficiency every step of the way.
Screenshots
Best Parameter Combination
Data Analysis
Test Cases Side by Side
Roadmap
Qian (The Creative, Heaven) 🌤️ (乾):
- Setup the framework for wrappers that can be used directly
in the production code.
- Set up the BaseWrapper
- Set up the StringWrapper
- Setup the config framework
- Setup the experiment main function
- Setup the evaluator framework to do evaluations
- One auto-evaluator
- Ground truth matching
- Human evaluator
- Interactive evaluator
- Reader framework that be able to process different data
- One reader from csv
- Output parser - Capture detailed information
- Documents
- Git setup
- Cotribution guide
- End2End Examples
- Release
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
