Unitxt 1.4.2

Enhancements

• Added ability to calculate confidence interval for additional scores beyond the main_score by @assaftibm in #431
• Improved error messaging in test cards and added option to disable via environment variable by @yoavkatz in #440
• Added support for Japanese language in sacrebleu by @yoavkatz in #433
• Introduced binary recall, precision metrics, and advanced filtering operators by @lilacheden in #424
• Added text completion task and modified lm harness template by @perlitz in #429

Internal code changes

• Converted instructions to operators by @elronbandel in #450

Bug fixes

• Fixed split definition in debater datasets by @ilyashnil in #443
• Added new requirements for sacrebleu by @gitMichal in #448

Documentation

• Documented release notes process by @matanor in #444
• Added guidelines for updating Unitxt documentation by @matanor in #446

Non backward compatible changes

• Implemented SystemFormat instance operator and updated BasicRecipe to use all operators, removing renderers and ICLFormat by @dafnapension in #423
  (this change can break any code using the deprecated ICLFormat)
• Additional verification introduced, aiming for uniquely determined results by @dafnapension in #435

New Contributors

• @lilacheden made their first contribution in #424

Full Changelog: 1.4.1...1.4.2

Unitxt 1.4.1

(Same as 1.4.0 - rereleasing due to release process error)

Enhancements

• New random generation mechanism to remove dependency between different random generators @matanor in #414
• New MultipleChoiceTemplate which changes all QA datasets by @elronbandel in #405
• New MAP, MRR, and Retrieval@K metrics by @assaftibm in #422
• New LoadFromKaggle loader which allows direct loading of datasets from Kaggle by @ilyashnil in #413
• New StringContainment metric that if one of the references is contained in the prediction by @ellarabi in #394
• New ConvertToBoolean post processor that changes the predict to either TRUE or FALSE by @ellarabi in #394
• 15 new open source classification datasets by @ilyashnil in #410, #418
• Documentation is now automatically generate in each release (#384)

Bug fixes

• Fixed and improved error checking in multi label F1 by @yoavkatz in #390
• Changed translations to use normalized_sacrebleu by @gitMichal in #426

Non backward compatible changes

• MultipleChoice QA datasets need to move to new MultipleChoiceTemplates
• Translation blue metric is now between 0-1 and not 0-100
• New randomization mechanism may change selects selection of demos , randomized text augmentation, or any other random choice.

What's Changed

• metric and processor related to robustness evaluation by @ellarabi in #394
• Removed variable set in debugging by @yoavkatz in #409
• Adding simple datasets by @ilyashnil in #410
• Add multiple choice template and fix mmlu by @elronbandel in #405
• Fixed error check in multi label F1 by @yoavkatz in #390
• FIX multiple choice template by @perlitz in #416
• example of kaggle loader by @ilyashnil in #413
• Medical abstract by @ilyashnil in #418
• Multilabel and jsons by @ilyashnil in #419
• Improve dataset addition tutorial + Fix auto uploads of the catalog to the web by @elronbandel in #384
• Improve docs by @elronbandel in #420
• Fix docs and docs compilation tests by @elronbandel in #421
• Hard code classes names by @elronbandel in #425
• Retrieval metrics by @assaftibm in #422
• modify to different bleu impl. by @gitMichal in #426
• Improve docs by @elronbandel in #427
• Independent random generators by @matanor in #414

New Contributors

• @ellarabi made their first contribution in #394

Full Changelog: 1.3.1...1.4.0

Uniitxt 1.4.0

Enhancements

• New random generation mechanism to remove dependency between different random generators @matanor in #414
• New MultipleChoiceTemplate which changes all QA datasets by @elronbandel in #405
• New MAP, MRR, and Retrieval@K metrics by @assaftibm in #422
• New LoadFromKaggle loader which allows direct loading of datasets from Kaggle by @ilyashnil in #413
• New StringContainment metric that if one of the references is contained in the prediction by @ellarabi in #394
• New ConvertToBoolean post processor that changes the predict to either TRUE or FALSE by @ellarabi in #394
• 15 new open source classification datasets by @ilyashnil in #410, #418
• Documentation is now automatically generate in each release (#384)

Bug fixes

• Fixed and improved error checking in multi label F1 by @yoavkatz in #390
• Changed translations to use normalized_sacrebleu by @gitMichal in #426

Non backward compatible changes

• MultipleChoice QA datasets need to move to new MultipleChoiceTemplates
• Translation blue metric is now between 0-1 and not 0-100
• New randomization mechanism may change selects selection of demos , randomized text augmentation, or any other random choice.

What's Changed

• metric and processor related to robustness evaluation by @ellarabi in #394
• Removed variable set in debugging by @yoavkatz in #409
• Adding simple datasets by @ilyashnil in #410
• Add multiple choice template and fix mmlu by @elronbandel in #405
• Fixed error check in multi label F1 by @yoavkatz in #390
• FIX multiple choice template by @perlitz in #416
• example of kaggle loader by @ilyashnil in #413
• Medical abstract by @ilyashnil in #418
• Multilabel and jsons by @ilyashnil in #419
• Improve dataset addition tutorial + Fix auto uploads of the catalog to the web by @elronbandel in #384
• Improve docs by @elronbandel in #420
• Fix docs and docs compilation tests by @elronbandel in #421
• Hard code classes names by @elronbandel in #425
• Retrieval metrics by @assaftibm in #422
• modify to different bleu impl. by @gitMichal in #426
• Improve docs by @elronbandel in #427
• Independent random generators by @matanor in #414

New Contributors

• @ellarabi made their first contribution in #394

Full Changelog: 1.3.1...1.4.0

Unitxt 1.3.2

Enhancements

• Add classification datasets by @ilyashnil in #410
• Added StringContainment metric and convert_to_boolean post processor that normalizes "yes/no/true/false" by @ellarabi in #394

Bug fixes

• Fix for AugmentPrefixSuffix that ignored labels field by @yoavkatz in #409

What's Changed

• metric and processor related to robustness evaluation by @ellarabi in #394
• Removed variable set in debugging by @yoavkatz in #409
• Adding simple datasets by @ilyashnil in #410

New Contributors

• @ellarabi made their first contribution in #394

Full Changelog: 1.3.1...1.3.2

Unitxt 1.3.1

Enhancements:

• FilterByValue can filter out instances that contain a given value (@elronbandel) #402
• added ag_news by (@ilyashnil) in #400

Fixes:

• DiverseLabelSampler used in balancing demonstrations in NER , binary and multi-label classification receives input field to balance on.(@yoavkatz) #399
• Fix to allow override empty_label of multi_label template used in non-english templates. (@yoavkatz) #403
• Fix to AugmentorPrefixSuffix not return same prefix/suffix repeated (@yoavkatz) in #407

Possible changes

• NER results will improve due to better balancing(@yoavkatz) #399

Documentation

• expand code coverage and documentation of refiner classes by (@dafnapension) #396

What's Changed

• added ag_news by @ilyashnil in #400
• peek at the first instance by @dafnapension in #401
• DiverseLabelsSampler fix by @yoavkatz in #399
• Enhance FilterByValue to have disallowed_values by @elronbandel in #402
• expand code coverage and documentation of refiner classes by @dafnapension in #396
• Also added more tests and documentation to DiverseLabelsSampler by @yoavkatz in #404
• Fix to allow override empty_label of multi_label template. by @yoavkatz in #403
• Fix suffix prefix not return same prefix/suffix repeated by @yoavkatz in #407

New Contributors

• @ilyashnil made their first contribution in #400

Full Changelog: 1.3.0...1.3.1

Unitxt 1.3.0

Release Notes for Version 1.3.0

New Features and Enhancements:

• Caching in LoadFromIBMCos: Improved data retrieval efficiency with caching in LoadFromIBMCos (@yoavkatz) - PR #377.
• Additional Inputs for HuggingFace Metrics: Enhanced metrics with the ability to include additional inputs (@yoavkatz) - PR #383.
• Support for Independent Random Generators: Enhanced data diversity with support for independent random generators (@matanor) - PR #374.
• JSON Dumps with Non-standard ASCII Support: Updated json.dumps calls for non-standard ASCII support (@yoavkatz) - PR #386.
• Improved API and Coverage of CastFields: Overhauled CastFields API and fixed issues with casting list fields (@dafnapension) - PR #372.
• Expand Augment Suffix to Prefix Suffix: Expanded augment suffix functionality to include prefix suffix, increasing versatility (@dafnapension) - PR #375.

Fixes:

• Circular Reference Error in FieldOperators: Addressed circular reference errors in FieldOperators (@dafnapension) - PR #376.
• Llama Fix: Removed <s> from llama format. (@gitMichal) - PR #392.
• Improvements and Fixes in Type Utils: Improved reliability and performance in type utilities (@elronbandel) - PR #395.
• Deep Copy in Extract Field Values: Ensured data integrity with deep copy in extract_field_values (@dafnapension) - PR #379.

Potential for results changes

• Llama Fix: Removed <s> from llama format. (@gitMichal) May changes llama model results- PR #392.
• Support for Independent Random Generators: Enhanced data diversity with support for independent random generators (@matanor) May change random selection of demonstrations - PR #374.

Improvements:

• Documentation and Code Coverage Expansion: Expanded documentation and coverage of additional operators (@dafnapension) - PR #397.
• Backward Compatibility in Exposing Types Dictionary: Maintained backward compatibility by not exposing the types dictionary (@dafnapension) - PR #393.

Full Changelog: 1.1.4...1.3.0

What's Changed

• Add caching to LoadFromIBMCos by @yoavkatz in PR #377
• Allow adding additional inputs to HF metrics by @yoavkatz in PR #383
• Support random generators independent of previous randomizations by @matanor in PR #374
• Change calls to json.dumps to allow storing in non standard ascii by @yoavkatz in PR #386
• Improve API and coverage of CastFields and fix problem with casting list fields by @dafnapension in PR #372
• Expand augment suffix to become augment prefix suffix by @dafnapension in PR #375
• Generate error when there are circular references between from and to fields in FieldOperators by @dafnapension in PR #376
• Llama fix by @gitMichal in PR #392
• Improve and fix type utils by @elronbandel in PR #395
• Deep copy at extract_field_values, in order to allow the same input m… by @dafnapension in PR #379
• Cover and document recursive DivideAllFieldsBy, and also ApplyOperatorsField by @dafnapension in PR #397
• Not exposing types dictionary. Back to backward compatibility by @dafnapension

Unitxt 1.2.0

Release Notes for Version 1.2.0

New Features and Enhancements:

• Cache Usage Optimization: Optimized cache usage with a new default setting to enhance performance by not storing intermediate results in HuggingFace cache (@elronbandel) - PR #360.

Fixes

• Fixed bug in Mapping Full List in MapInstanceValues: MapInstanceValues did not correctly map lists (@yoavkatz) - PR #367.

Improvements

• Update Logging: Unitxt now uses standard logger for better clarity and troubleshooting (@elronbandel) - PR #363.

• Better error checking in FilterByListsOfValues Operator: FilterByListsOfValues now throws and error when filtering all elements of a list, unless explicitly requested otherwise (@elronbandel) - PR #362.

• Templates Converted to Operators: Transformed templates into operators for improved functionality and integration (@elronbandel) - PR #357.

Documentation and Code Coverage:

• Expand Code Coverage and Documentation: Improved documentations and test coverages of operators (@dafnapension) - PR #361 PR #366, PR #370, PR #371.

For more detailed information, please refer to the linked pull requests in our repository.

What's Changed

• ethos and post processors by @naamaz in #358
• Improve filtering operator by @elronbandel in #362
• Update logging by @elronbandel in #363
• first draft for expand code coverage and documentation by @dafnapension in #361
• Add support for mapping full list in MapInstanceValues by @yoavkatz in #367
• Allow using cache set default to not by @elronbandel in #360
• Expand code coverage and document operators by @dafnapension in #366
• cover additional operators by @dafnapension in #370
• cover take by @dafnapension in #371
• Turn templates to operators by @elronbandel in #357
• Update README.md by @elronbandel in #368

Full Changelog: 1.1.4...1.2.0

v1.0.1 pre release

Full Changelog: https://github.com/IBM/unitxt/commits/1.0.0