python/notebooks/07-hugging-face-for-plugins.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "68e1c158",
   "metadata": {},
   "source": [
    "# Using Hugging Face With Plugins\n",
    "\n",
    "In this notebook, we demonstrate using Hugging Face models for Plugins using both SemanticMemory and text completions. \n",
    "\n",
    "SK supports downloading models from the Hugging Face that can perform the following tasks: text-generation, text2text-generation, summarization, and sentence-similarity. You can search for models by task at https://huggingface.co/models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a77bdf89",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -m pip install semantic-kernel==0.4.6.dev0\n",
    "\n",
    "# Note that additional dependencies are required for the Hugging Face connectors:\n",
    "!python -m pip install torch==2.0.0\n",
    "!python -m pip install transformers==^4.28.1\n",
    "!python -m pip install sentence-transformers==^2.2.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "508ad44f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import semantic_kernel as sk\n",
    "import semantic_kernel.connectors.ai.hugging_face as sk_hf"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d8ddffc1",
   "metadata": {},
   "source": [
    "First, we will create a kernel and add both text completion and embedding services. \n",
    "\n",
    "For text completion, we are choosing GPT2. This is a text-generation model. (Note: text-generation will repeat the input in the output, text2text-generation will not.)\n",
    "For embeddings, we are using sentence-transformers/all-MiniLM-L6-v2. Vectors generated for this model are of length 384 (compared to a length of 1536 from OpenAI ADA).\n",
    "\n",
    "The following step may take a few minutes when run for the first time as the models will be downloaded to your local machine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f8dcbc6",
   "metadata": {},
   "outputs": [],
   "source": [
    "kernel = sk.Kernel()\n",
    "\n",
    "# Configure LLM service\n",
    "kernel.add_text_completion_service(\n",
    "    service_id=\"gpt2\",\n",
    "    service=sk_hf.HuggingFaceTextCompletion(ai_model_id=\"gpt2\", task=\"text-generation\"),\n",
    ")\n",
    "kernel.add_text_embedding_generation_service(\n",
    "    service_id=\"sentence-transformers/all-MiniLM-L6-v2\",\n",
    "    service=sk_hf.HuggingFaceTextEmbedding(ai_model_id=\"sentence-transformers/all-MiniLM-L6-v2\"),\n",
    ")\n",
    "kernel.register_memory_store(memory_store=sk.memory.VolatileMemoryStore())\n",
    "kernel.import_plugin(sk.core_plugins.TextMemoryPlugin())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2a7e7ca4",
   "metadata": {},
   "source": [
    "### Add Memories and Define a plugin to use them\n",
    "\n",
    "Most models available on huggingface.co are not as powerful as OpenAI GPT-3+. Your plugins will likely need to be simpler to accommodate this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d096504c",
   "metadata": {},
   "outputs": [],
   "source": [
    "await kernel.memory.save_information_async(collection=\"animal-facts\", id=\"info1\", text=\"Sharks are fish.\")\n",
    "await kernel.memory.save_information_async(collection=\"animal-facts\", id=\"info2\", text=\"Whales are mammals.\")\n",
    "await kernel.memory.save_information_async(collection=\"animal-facts\", id=\"info3\", text=\"Penguins are birds.\")\n",
    "await kernel.memory.save_information_async(collection=\"animal-facts\", id=\"info4\", text=\"Dolphins are mammals.\")\n",
    "await kernel.memory.save_information_async(collection=\"animal-facts\", id=\"info5\", text=\"Flies are insects.\")\n",
    "\n",
    "# Define semantic function using SK prompt template language\n",
    "my_prompt = \"\"\"I know these animal facts: {{recall $query1}} {{recall $query2}} {{recall $query3}} and \"\"\"\n",
    "\n",
    "# Create the semantic function\n",
    "my_function = kernel.create_semantic_function(prompt_template=my_prompt, max_tokens=45, temperature=0.5, top_p=0.5)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2calf857",
   "metadata": {},
   "source": [
    "Let's now see what the completion looks like! Remember, \"gpt2\" is nowhere near as large as ChatGPT, so expect a much simpler answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "628c843e",
   "metadata": {},
   "outputs": [],
   "source": [
    "context = kernel.create_new_context()\n",
    "context[sk.core_plugins.TextMemoryPlugin.COLLECTION_PARAM] = \"animal-facts\"\n",
    "context[sk.core_plugins.TextMemoryPlugin.RELEVANCE_PARAM] = \"0.3\"\n",
    "\n",
    "context[\"query1\"] = \"animal that swims\"\n",
    "context[\"query2\"] = \"animal that flies\"\n",
    "context[\"query3\"] = \"penguins are?\"\n",
    "output = await kernel.run_async(my_function, input_vars=context.variables)\n",
    "\n",
    "output = str(output).strip()\n",
    "\n",
    "query_result1 = await kernel.memory.search_async(\n",
    "    collection=\"animal-facts\", query=context[\"query1\"], limit=1, min_relevance_score=0.3\n",
    ")\n",
    "query_result2 = await kernel.memory.search_async(\n",
    "    collection=\"animal-facts\", query=context[\"query2\"], limit=1, min_relevance_score=0.3\n",
    ")\n",
    "query_result3 = await kernel.memory.search_async(\n",
    "    collection=\"animal-facts\", query=context[\"query3\"], limit=1, min_relevance_score=0.3\n",
    ")\n",
    "\n",
    "print(f\"gpt2 completed prompt with: '{output}'\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}