Version 1.0.8

Amended some incorrect casts within the image token calculations (Rookie mistake)

Version 1.0.7

Added in image to token calculator for the following model:

• gpt-4-vision-preview

Based on the OpenAI API documentation for Vision enabled models (as of 04/12/2023), to calculate the token count of an image, you need to consider the size of the image and the detail option on each image_url block.

• If the detail option is set to "low", the token cost is a fixed 85 tokens, regardless of the size of the image.
• If the detail option is set to "high", the process is a bit more complex:
  • The image is first scaled to fit within a 2048 x 2048 square, maintaining their aspect ratio. Then, it is scaled such that the shortest side of the image is 768px long. The image is divided into 512px squares. Each of those squares costs 170 tokens. Finally, an additional 85 tokens are always added to the final total.

Version 1.0.5

Support for .NET6, .Net Standard 2.0, and 2.1 Added. Moved assets to embedded resources and improved read speed capabilities. Added in unsafe capabilities for token counting for non-production use and parallelism for managed functions.

Version 1.0.4

The URL fetch seemed unnecessary, removed these in favour of pre-packaged tiktoken files.