Unitxt 1.15.6 - Chat Inference

Main changes

• Added support for generating output in ChatAPI format (user/assistant turns) and for inference engines to process ChatAPI input.
  See details in blog.

• Improved catalog browsing experience with cleaner formatting of catalog assets, and clickable hyper links between catalog assets and between catalog assets and code. See for example.

New Features

Inference Engines that support ChatApi interface

• Add target_prefix erasing post processor by default by @elronbandel in #1361
• Add multi api inference engine by @elronbandel in #1343
• Add chat api format with standard open ai chat format by @elronbandel in #1314
• Add option selecting huggingface inference engine by @elronbandel in #1357

Improved multi model support

• Add seed bench dataset and support for videos by @elronbandel in #1309
• Add LMMSEvalInferenceEngine by @elronbandel in #1301
• Vision robustness blog by @elronbandel in #1318

New Asserts

• added QTSUMM taskcard for query-focused table summarization task by @csrajmohan in #1304
• Add OptionSelectingByLogProbsInferenceEngine by @martinscooper in #1317
• Replace 20 newsgroup with a shorter version in bluebench by @perlitz in #1347
• Bluebench Update by @perlitz in #1342
• Update Blue Bench description by @elronbandel in #1354
• Batched multi class classification by @yoavkatz in #1340
• move rag binary llmaj under rag metrics by @lilacheden in #1338
• adding generic inference binary+idk judges by @Roni-Friedman in #1316
• Add table augmentors by @elronbandel in #1328
• Align augmenters with task and types mechanisms by @elronbandel in #1356
• add serializers to catalog + new table operators by @ShirApp in #1365

Performance

• Add loaders cache by @elronbandel in #1333

Usuability

• Allow turning single stream to dataset by @elronbandel in #1335

Documentation

• Add ability to load_dataset without a template for simpler usage for beginners by @elronbandel in #1350
• add score name prefix for judge_raw_output/input in llmaj metric by @OfirArviv in #1323
• Add link to source in catalog assets by @elronbandel in #1362
• Fix docs compilation and links from docs to github by @elronbandel in #1359
• Fix website docs-code links by @elronbandel in #1360
• Update error checking and documentation of processors by @yoavkatz in #1325
• Unified catalog terminology by @yoavkatz in #1355
• Improved documentation formatting by @dafnapension in #1334
• Fix catalog links by @elronbandel in #1348
• Print catalog entries as yamls by @dafnapension in #1351

CI/CD

• a more elaborated message from performace-test-summary, and doc-string of card_profiler by @dafnapension in #1307
• Make package requirements compatible with requirements.txt like format by @elronbandel in #1310
• Make inference engine tests run only when inference.py has changed by @elronbandel in #1311
• Seperate examples tests by @elronbandel in #1322
• Fix pyproject.toml to be standalone and comply with modern standards by @elronbandel in #1324
• Fix GitHub Actions concurrence execution by @elronbandel in #1349
• Make tests faster and clearer by @dafnapension in #1345

New Contributors

• @martinscooper made their first contribution in #1317

Full Changelog: 1.14.1...1.15.6