Implement HuggingFaceVisualQuestionAnsweringRemoteInferencePromptSchema #1396
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rholinshead
requested review from
saqadri,
suyoglastmileai,
Ankush-lastmile,
jonathanlastmileai and
rossdanlm
as code owners
Ankush-lastmile
approved these changes
Mar 4, 2024
| type: "string", | ||
| description: `Hugging Face model to use. Can be a model ID hosted on the Hugging Face Hub or a URL | ||
| to a deployed Inference Endpoint`, | ||
| default: "dandelin/vilt-b32-finetuned-vqa", |
There was a problem hiding this comment.
does this model work fine on the free inference endpoint?
Ya, I think we can improve it a bit. Probably makes more sense to have attachment first since it seems to be the main focus? |
# Implement HuggingFaceVisualQuestionAnsweringRemoteInferencePromptSchema For #1392 This will add the prompt schema so that visual question answering prompts have the nice UI for input and settings
rholinshead
added a commit
that referenced
this pull request
Mar 4, 2024
# Render Attachment before Data When Both Exist in Input See comment in #1396 (review) When we have a Prompt Input with both attachment and data, most likely the attachment will be the main focus (e.g. Visual Question Answering) so we can show it first in the prompt input:   Was initially thinking we could order it based on the properties order in the schema, but just doing this for now since property order in schema may not be guaranteed when we define it in python and then serialize it to to client (will just be passing JSON object in response) --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/lastmile-ai/aiconfig/pull/1397). * __->__ #1397 * #1396
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement HuggingFaceVisualQuestionAnsweringRemoteInferencePromptSchema
Implement HuggingFaceVisualQuestionAnsweringRemoteInferencePromptSchema
For #1392
This will add the prompt schema so that visual question answering prompts have the nice UI for input and settings
Stack created with Sapling. Best reviewed with ReviewStack.