Creative Technology · Build your own AI mini app
How to Start
Run a pretrained captioner, understand its output, and build one useful improvement.
1. Explore the demo
- Choose one of the example images or upload a PNG, JPEG, or WebP image.
- Start with ViT-GPT2 and select Run AI Captioning.
- Allow time for the first model download. Subsequent runs can reuse cached files. Use Stop this run if needed.
- Read the caption and check its objects, actions, and relationships against the image.
- Keep the same image and try Florence-2 or SmolVLM. Compare the actual sentences in the log.
2. Change one thing
Florence-2 supports short and detailed captioning tasks. SmolVLM accepts a description instruction. ViT-GPT2 produces a short caption without an instruction. The output-token limit is not a target word count.
Choose one experiment: compare models, change the requested detail, or crop an image. Record both successes and failures. Download the comparison log before reloading the page.
3. Make your own Hugging Face Space
- Sign in to Hugging Face and open this project's Space page.
- Use the Space menu to duplicate it into your own account. Alternatively, create a new Static Space and upload this folder's files with their subfolders.
- Keep
sdk: staticandapp_file: index.htmlin the README metadata. - Wait for the Space to build, open its App tab, and test one example and one uploaded image.
This project needs no API key, paid inference endpoint, or training job. Internet access is required to obtain the library and model files.
4. Find the code that matters
| File | What to change |
|---|---|
| index.html | The task explanation and page structure. |
| app.js | Image selection, result display, and comparison records. |
| models.js | Model IDs, pinned revisions, and task settings. |
| captioner.js | Load the model, prepare inputs, generate tokens, decode the caption. |
| caption-worker.js | Run inference separately so the page stays responsive. |
| examples/ | Teaching images, manifest, and source records. |
When changing a model, check its Transformers.js support and input format. A Python model ID cannot always be substituted directly.
5. Explain and submit
- A working public Space URL and a short explanation of the app's purpose.
- The model ID and an explanation of the image → features → text process.
- At least one successful example and one difficult or incorrect description.
- One controlled comparison with its images, settings, and unchanged generated captions.
- Image sources and a note on limitations. Training a model is optional.
If something does not work
Model download fails: check access to Hugging Face and jsDelivr, then retry. The first run loads roughly a few hundred MB per model.
Generation is slow: use a recent desktop browser, close memory-heavy tabs, choose a shorter output limit, or stop the run. Loading time is shown separately from generation time.
The sentence is wrong: save the actual output and identify the unsupported detail. Try another model or a clearer image; a fluent sentence is not a guarantee of correctness.
Local files do not load: serve the folder over HTTP instead of opening index.html with file://.
python -m http.server 7860Open http://localhost:7860. Uploaded images are processed in your browser. They are not sent to an inference server.