Part 1: Vision Pipeline

Goal: Get a computer vision pipeline working.

Skills: Connect a machine to Viam, configure components with the Viam CLI, use fragments to add preconfigured services.

Time: ~10 min

Prerequisites

Before starting this tutorial, you need the can inspection simulation running. Follow the Gazebo Simulation Setup Guide to:

  1. Build the Docker image with Gazebo Harmonic
  2. Create a machine in Viam and get credentials
  3. Start the container with your Viam credentials

Once you see “Can Inspection Simulation Running!” in the container logs and your machine shows Live in the Viam app, return here to continue.

You also need the Viam CLI installed and authenticated. See Viam CLI overview for installation and authentication instructions.

1.1 Verify Your Machine is Online

If you followed the setup guide, your machine should already be online.

Run:

viam machines status --machine=<machine-id>

Look for status: live in the output. If your machine isn’t live yet, check that the Docker container is running and that the logs show “Can Inspection Simulation Running!”.

Ordinarily, after creating a machine in Viam, you would download and install viam-server together with the cloud credentials for your machine. For this tutorial, we’ve already installed viam-server and launched it in the simulation Docker container.

1.2 Get Your Part ID

Your machine is online but empty. To configure your machine, you will add components and services to your machine part. Your machine part is the compute hardware — in this tutorial, a virtual machine running Linux in the Docker container.

Get the part ID for inspection-station-1-main:

viam machines part list --machine=<machine-id>

Copy the part ID from the output. You’ll use it in every command that follows.

1.3 Configure the Camera

You’ll now add the camera as a component.

What's a component?

In Viam, a component is any piece of hardware: cameras, motors, arms, sensors, grippers. You configure components by declaring what they are, and Viam handles the drivers and communication.

The power of Viam’s component model: All cameras expose the same API—USB webcams, Raspberry Pi camera modules, IP cameras, simulated cameras. Your application code uses the same GetImages() method regardless of the underlying hardware. Swap hardware by changing configuration, not code.

Add a camera component

Add the gz-camera module’s rgb-camera model and name it inspection-cam:

viam machines part add-resource \
  --part=<part-id> \
  --name=inspection-cam \
  --model-name=gz-camera:gz-camera:rgb-camera
Why does the model triplet have three parts?
The model triplet format is namespace:module-name:model-name. Here, gz-camera is both the namespace and module name, and rgb-camera is the specific model within that module. The module (gz-camera) is what implements the Viam camera API for this simulated hardware. When you add a component that comes from a registry module, viam-server downloads and runs the module automatically.

Configure the camera

Set the camera ID so the module knows which Gazebo camera to connect to:

viam resource update \
  --part=<part-id> \
  --resource-name=inspection-cam \
  --config '{"id": "/inspection_camera"}'

1.4 Test the Camera

The easiest way to verify the camera is working is to use the Viam app’s built-in test panel, which shows a live video feed.

  1. Open app.viam.com and navigate to your machine
  2. Click the CONFIGURE tab and select inspection-cam
  3. Expand the TEST section at the bottom of the camera’s configuration panel

You should see a live overhead view of the conveyor/staging area.

1.5 Add a vision pipeline with a fragment

Now you’ll add machine learning to run inference on your camera feed. You need two services:

  1. ML model service that loads a trained model for the inference task
  2. Vision service that connects the camera to the model and returns detections
Components versus services
  • Components are hardware: cameras, motors, arms
  • Services are capabilities: vision (ML inference), motion (arm kinematics), custom control logic

Services often use components. A vision service takes images from a camera, runs them through an ML model, and returns structured results, detections with bounding boxes and labels, or classifications with confidence scores.

The ML model service loads a trained model (TensorFlow, ONNX, or PyTorch) and exposes an Infer() method. The vision service handles the rest: converting camera images to tensors, calling the model, and interpreting outputs into usable detections.

Instead of adding each service manually, you’ll use a fragment. A fragment is a reusable block of configuration that can include components, services, modules, and ML models. Fragments let you share tested configurations across machines and teams.

The try-vision-pipeline fragment includes an ML model service loaded with a can defect detection model and a vision service wired to that model. The fragment accepts a camera_name variable so it works with any camera.

Add the fragment

Apply the try-vision-pipeline fragment to your machine part:

viam machines part fragments add \
  --part=<part-id> \
  --fragment=<try-vision-pipeline-fragment-id>
How do I find the fragment ID?
You can find the fragment ID in the Viam registry by searching for try-vision-pipeline and copying the ID from the fragment’s detail page.

Set the camera variable

The fragment uses a camera_name variable to wire the vision service to your specific camera. Set it in the machine’s JSON configuration:

viam machines part run \
  --part=<part-id> \
  --data='{"name": "inspection-station-1-main"}' \
  viam.app.v1.RobotService.GetRobotPart

Then update the fragment’s variable override. Open the machine’s JSON config and add the fragment_mods entry:

{
  "fragment_mods": [
    {
      "fragment_id": "<try-vision-pipeline-fragment-id>",
      "mods": [
        {
          "$ set": {
            "camera_name": "inspection-cam"
          }
        }
      ]
    }
  ]
}

Save this as fragment-vars.json and apply it with the app API, or set the variable in the Viam app if you prefer.

Test the vision service

  1. Open app.viam.com and navigate to your machine
  2. Click the CONFIGURE tab and find vision-service
  3. Expand the TEST section, select inspection-cam as the camera source, and set Detections/Classifications to Live

Continue to Part 2: Data Capture →