v0.9.15rc1

What's Changed

• YOLO-World Inference Pipeline by @paulguerrie in #282
• QR code workflow step by @sberan in #286
• Add structured API logger by @PawelPeczek-Roboflow in #287
• Feature/yolov9 by @probicheaux in #290

Full Changelog: v0.9.14...v0.9.15rc1

v0.9.14

🚀 Added

LMMs (GPT-4V and CogVLM) 🤝 workflows

Now, with Roboflow workflows LMMs integration is made easy 💪 . Just look at the demo! 🤯

lmms_in_workflows.mp4

As always, we encourage you to visit workflows docs 📖 and examples.

This is how to create a multi-functional app with workflows and LMMs:

inference server start

from inference_sdk import InferenceHTTPClient

LOCAL_CLIENT = InferenceHTTPClient(
    api_url="http://127.0.0.1:9001", 
    api_key=ROBOFLOW_API_KEY,
)
FLEXIBLE_SPECIFICATION = {
    "version": "1.0",
    "inputs": [
        { "type": "InferenceImage", "name": "image" },
        { "type": "InferenceParameter", "name": "open_ai_key" },
        { "type": "InferenceParameter", "name": "lmm_type" },
        { "type": "InferenceParameter", "name": "prompt" },
        { "type": "InferenceParameter", "name": "expected_output" },
    ],
    "steps": [     
        {
            "type": "LMM",
            "name": "step_1",
            "image": "$inputs.image",
            "lmm_type": "$inputs.lmm_type",
            "prompt": "$inputs.prompt",
            "json_output": "$inputs.expected_output",
            "remote_api_key": "$inputs.open_ai_key",
        },
    ],
    "outputs": [
        { "type": "JsonField", "name": "structured_output", "selector": "$steps.step_1.structured_output" },
        { "type": "JsonField", "name": "llm_output", "selector": "$steps.step_1.*" },
    ]   
}

response_gpt = LOCAL_CLIENT.infer_from_workflow(
    specification=FLEXIBLE_SPECIFICATION,
    images={
        "image": cars_image,
    },
    parameters={
        "open_ai_key": OPEN_AI_KEY,
        "lmm_type": "gpt_4v",
        "prompt": "You are supposed to act as object counting expert. Please provide number of **CARS** visible in the image",
        "expected_output": {
            "objects_count": "Integer value with number of objects",
        }
    }
)

🌱 Changed

• developer friendly theming aka dark theme by @onuralpszr in #270 (thanks for contribution 🥇 )
• YoloWorld docs @capjamesg in #276
• @ryanjball made their first contribution in #271 with his cookbook for RGB anomaly detection

🔨 Fixed

• turn off instant page for load to cookbook page properly by @onuralpszr in #275 (thanks for contribution 🥇 )
• bug in workflows that made cropping in multi-detection set-up

Full Changelog: v0.9.13...v0.9.14

v0.9.13

🚀 Added

YOLO World 🤝 workflows

We've introduced Yolo World model into workflows making it trivially easy to use the model as any other object-detection model ☺️

To try this out, install dependencies first:

pip install inference-sdk inference-cli

Start the server:

inference server start

And run the script:

from inference_sdk import InferenceHTTPClient

CLIENT = InferenceHTTPClient(api_url="http://127.0.0.1:9001", api_key="YOUR_API_KEY")

YOLO_WORLD = {
    "specification": {
        "version": "1.0",
        "inputs": [
            { "type": "InferenceImage", "name": "image" },
            { "type": "InferenceParameter", "name": "classes" },
            { "type": "InferenceParameter", "name": "confidence", "default_value": 0.003 },
        ],
        "steps": [
            {
                "type": "YoloWorld",
                "name": "step_1",
                "image": "$inputs.image",
                "class_names": "$inputs.classes",
                "confidence": "$inputs.confidence",
            },
        ],
        "outputs": [
            { "type": "JsonField", "name": "predictions", "selector": "$steps.step_1.predictions" },
        ]   
    }
}

response = CLIENT.infer_from_workflow(
    specification=YOLO_WORLD["specification"],
    images={
        "image": frame,
    },
    parameters={
        "classes": ["yellow filling", "black hole"]  # each time you may specify different classes!
    }
)

Check details in documentation 📖 and discover usage examples.

🏆 Contributors

@PawelPeczek-Roboflow (Paweł Pęczek)

Full Changelog: v0.9.12...v0.9.13

v0.9.12

🚀 Added

inference cookbook

Visit our cookbook 🧑‍🍳

🔨 Fixed

In this release, we are fixing issues spotted in YoloWorld model released in v0.9.11, in particular:

• bug with hashing of YOLO World classes making it impossible in some cases to run inference due to improper caching of CLIP embeddings
• bug with YOLO World pre-processing of colour channels causing model misunderstanding of prompted colours

🏆 Contributors

@capjamesg (James Gallagher), @PawelPeczek-Roboflow (Paweł Pęczek)

Full Changelog: v0.9.11...v0.9.12

v0.9.12rc3

Fixed embeddings hashing

v0.9.12rc2

Fixed hashing of text embeddings

v0.9.12rc1

Release candidate with fix to Yolo-World pre-processing

v0.9.11

🚀 Added

YOLO World in the inference

Have you heard about YOLO World model? 🤔 If not - you would probably be interested to learn something about it! Our blog post 📰 may be a good starting point❗

Great news is that YOLO World is already integrated with inference. Model is capable to perform zero-shot detections of classes specified in inference parameter. Thanks to that, you may start making videos like that just now 🚀

yellow-filling-output-1280x720.mp4

Simply install dependencies.

pip install inference-sdk inference-cli

Start the server

inference server start

And run inference against our HTTP server:

from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(api_url="http://127.0.0.1:9001")
result = client.infer_from_yolo_world(
    inference_input=YOUR_IMAGE,
    class_names=["dog", "cat"],
)

Active Learning 🤝 workflows

Active Learning data collection made simple with workflows 🔥 Now, with just a little bit of configuration you can start data collection to improve your model over time. Just take look how easy it is:

active_learning_in_workflows.mp4

Key features:

• works for all models supported at Roboflow platform, including the ones from Roboflow Universe - making it trivial to use off-the-shelf model during project kick-off stage to collect dataset while serving meaningful predictions
• combines well with multiple workflows blocks - including DetectionsConsensus - making it possible to sample based on predictions of models ensemble 💥
• Active Learning block may use project-level config of Active Learning or define Active Learning strategy directly in the block definition (refer to Active Learning documentation 📖 for details on how to configure data collection)

See documentation 📖 of new ActiveLearningDataCollector to find detailed info.

🌱 Changed

InferencePipeline now works with all models supported at Roboflow platform 🎆

For a long time - InferencePipeline worked only with object-detection models. This is no longer the case - from now on, other type of models supported at Roboflow platform (including stubs - like my-project/0) work under InferencePipeline. No changes are required in existing code. Just put model_id of your model and the pipeline should work. Sinks suited for detection-only models were adjusted to ignore non-compliant formats of predictions and produce warnings notifying about incompatibility.

🔨 Fixed

• Bug in yolact model in #266

🏆 Contributors

@paulguerrie (Paul Guerrie), @probicheaux (Peter Robicheaux), @PawelPeczek-Roboflow (Paweł Pęczek)

Full Changelog: v0.9.10...v0.9.11

v0.9.10

🚀 Added

inference Benchmarking 🏃‍♂️

A new command has been added to the inference-cli for benchmarking performance. Now you can test inference in different environments with different configurations and measure its performance. Look at us testing speed and scalability of hosted inference at Roboflow platform 🤯

scaling_of_hosted_roboflow_platform.mov

Run your own benchmark with a simple command:

inference benchmark python-package-speed -m coco/3 

See the docs for more details.

🌱 Changed

• Improved serialisation logic of requests and responses that helps Roboflow platform to improve model monitoring

🔨 Fixed

• bug #260 causing inference API instability in multiple-workers setup and in case of shuffling large amount of models - from now on, API container should not raise strange HTTP 5xx errors due to model management
• faulty logic for getting request_id causing errors in parallel-http container

🏆 Contributors

@paulguerrie (Paul Guerrie), @SolomonLake (Solomon Lake ), @robiscoding (Rob Miller) @PawelPeczek-Roboflow (Paweł Pęczek)

Full Changelog: v0.9.9...v0.9.10

v0.9.10rc3

This is a pre-release version that mainly addresses some instabilities in the model manager.

What's Changed

• Add source to cache serializer by @SolomonLake in #242
• Parse request/response before caching by @robiscoding in #227
• Inference benchmarking by @PawelPeczek-Roboflow in #250

Full Changelog: v0.9.9...v0.9.10rc3