Kimi K3 Vision Tutorial: Image, Video & Screenshot-to-Code

One of the most practical upgrades in Kimi K3 is that vision is now a native capability — no bolted-on OCR, no separate vision model. You drop images and video straight into messages and let the model “look at the picture” to write code, read charts, and understand interfaces. But there are a few details in building a multimodal request that you have to get exactly right, or it fails outright. This guide lays out the correct approach.

Two hard rules first

Kimi K3’s vision input has two strict requirements — commit these to memory before anything else:

1. The image-and-text content must be an “object array,” never flattened into a serialized string. In other words, content has to be a list like [{type:"text",...}, {type:"image_url",...}].
2. Public image URLs are not supported. Inline local images as base64, and for video, upload through the files API and reference it as ms://<file-id>. Passing https://example.com/a.png directly will fail.

Keep these two straight and you sidestep most “vision request errors.”

Image input: inline base64

import base64, os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MOONSHOT_API_KEY"],
    base_url="https://api.moonshot.cn/v1",
)

with open("design.png", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()

resp = client.chat.completions.create(
    model="kimi-k3",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Which UI components are in this image? Give me a list."},
            {"type": "image_url",
             "image_url": {"url": f"data:image/png;base64,{b64}"}},
        ],
    }],
)
print(resp.choices[0].message.content)

Note that image_url.url uses a data URI (data:image/png;base64,...). This is “inline base64,” not a “public link” — and inline base64 is exactly what K3 supports.

Video input: files API + ms:// reference

Upload the video file through the files API first, then reference it in the message with ms://<file-id> once you have the file id:

# 1) Upload the video and get a file id
f = client.files.create(file=open("demo.mp4", "rb"), purpose="assistants")

# 2) Reference it in the message with ms://<file-id>
resp = client.chat.completions.create(
    model="kimi-k3",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "In this screen recording, which steps did the user complete?"},
            {"type": "image_url", "image_url": {"url": f"ms://{f.id}"}},
        ],
    }],
)

Parameters for the upload endpoint, such as purpose, follow the official documentation — verify the current version before you integrate.

Hands-on: turning a design mockup into a frontend

K3 is built for scenarios that “combine software engineering with visual analysis,” and the most typical of those is writing a frontend straight from a design mockup:

with open("mockup.png", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()

resp = client.chat.completions.create(
    model="kimi-k3",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text":
                "Write the corresponding HTML + Tailwind CSS strictly from this mockup. "
                "Match the layout, spacing, and colors as closely as you can, and output a complete, runnable single file."},
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
        ],
    }],
)
print(resp.choices[0].message.content)

The same approach works for reading chart data for analysis, locating problems from an error screenshot, or editing code from a game/CAD screenshot. Because K3 has a 1M context, you can even stuff “the mockup plus your existing codebase” in together and have it make edits within the full project context instead of generating from scratch (for long-context usage, see the 1M context guide).

Common errors

• Vision request fails outright: usually you passed a public URL, or you flattened content into a string. Switch to base64 / ms:// plus an object array.
• Image too large / over budget: images and video both count as tokens, and a big image or long video eats a lot of context. Compress the dimensions or sample frames when needed.
• State lost across turns: in a multi-turn conversation with images, remember to send the full assistant message back verbatim — don’t keep only content.

Wrap-up

K3’s native vision turns “look at an image + write code” into something a single model can do in a single request. Keep the two hard rules in mind (object array, no public URLs) and you basically won’t get tripped up. For full integration, pricing, and migration, see the Kimi K3 API tutorial. To call K3 alongside other multimodal models with a single key and compare results per task, grab a key from the GetModel dashboard to get started.