Camera stream

Streams live MJPEG video from an ESP32 camera module into a p5.js canvas over WiFi.

Video is served over a separate HTTP connection — it does not flow through the WebSocket control channel, so camera streaming and Arduino control can run simultaneously without interfering with each other.

What you need

• An ESP32 board with a camera module and PSRAM (see supported boards below)
• The board flashed with a sketch that #includes <PardaloteCamera.h> (the bundled camera example fits the bill)
• A web browser on the same WiFi network

Files

Arduino setup

1. Open the library example

In Arduino IDE: File → Examples → Pardalote → camera-stream. The sketch looks like:

// Uncomment ONE:
#define CAMERA_MODEL_WROVER_KIT               // Freenove ESP32-WROVER-DEV
// #define CAMERA_MODEL_AI_THINKER            // AI-Thinker ESP32-CAM
// #define CAMERA_MODEL_XIAO_ESP32S3          // Seeed XIAO ESP32S3 Sense
// #define CAMERA_MODEL_ESP32S3_EYE           // Espressif ESP32-S3-EYE
// #define CAMERA_MODEL_M5STACK_PSRAM         // M5Stack PSRAM
// #define CAMERA_MODEL_M5STACK_WIDE          // M5Stack Wide
// ... (see PardaloteCamera.h for all supported models)

#include <Pardalote.h>
#include <PardaloteCamera.h>

void setup() { Pardalote.begin(); }
void loop()  { Pardalote.run();   }

Pick the CAMERA_MODEL_* define that matches your hardware before the include.

2. Upload

Flash the sketch to your board. Once connected to WiFi, the IP address is printed in the Serial Monitor (115200 baud).

Browser setup

1. Connect on the page

Open index.html, then in the Board row enter the Arduino's IP (or switch to USB) and press Connect — no code editing needed, and the setting is remembered per browser. The camera's stream port is fixed by the firmware (82), so there's nothing else to set.

2. Open the example

Serve the folder from a local web server and open index.html in a browser. You can use any static server — for example:

npx serve .

If you are using a code editor you can run the web server there.

Why a server? Browsers block WebSocket connections from file:// pages on some platforms. A local server avoids this.

How it works

Arduino                          Browser
────────────────────────────────────────────────────────
WebSocket :81  ←──── control ────→  pardalote.js (Camera extension)
HTTP      :82  ──── MJPEG stream ──→  <img src="http://ip:82/stream">

1. arduino.connect() opens a WebSocket on port 81.
2. On ready, arduino.cam.attach(CAMERA_PORT) sends a CMD_CAMERA_INIT frame to the Arduino over the WebSocket. The Arduino initialises the camera hardware and starts an HTTP server on the requested port.
3. The Arduino echoes the confirmed port back. The Camera extension sets an <img> element's src to http://<ip>:<port>/stream and emits the stream event.
4. The browser fetches the MJPEG stream directly over HTTP — the WebSocket is only used for the initial handshake and any subsequent control commands (resolution, quality).
5. In draw(), image(camEl, 0, 0, width, height) renders the current frame to the p5.js canvas. Just before it, the sketch copies the frame's real size into camEl.width / camEl.height: p5 records an image's size only when its first frame loads, so without this a resolution change leaves a black band at the bottom.

Adjusting the stream

// Resolution — call before or after attach()
arduino.cam.setResolution(FRAMESIZE_QVGA);   // 320×240  ← default
arduino.cam.setResolution(FRAMESIZE_VGA);    // 640×480
arduino.cam.setResolution(FRAMESIZE_HD);     // 1280×720

// JPEG quality: 0 = best image / highest bandwidth
//              63 = worst image / lowest bandwidth
arduino.cam.setQuality(12);  // default

Higher resolutions require more bandwidth and may reduce frame rate. QVGA is a good starting point for most WiFi environments. Anything above QQVGA also needs PSRAM enabled, and the largest sizes don't work on every sensor — see Troubleshooting below.

Snapshots

Capture a single JPEG still without interrupting the stream:

const url = await arduino.cam.snapshot();
// url is a blob: URL — use as an <img> src or save it
// revoke when done to free memory:
URL.revokeObjectURL(url);

The snapshot is served from http://<ip>:<port+1>/snapshot — its own small web server on the next port (83 for this example), so it answers even while the stream is running.

Pixel manipulation

Because the camera HTTP server sends Access-Control-Allow-Origin: *, p5.js can read pixel data from the stream:

arduino.cam.on('stream', ({ url }) => {
    camEl = createImg(url, '');
    camEl.hide();
});

function draw() {
    if (camEl) {
        camEl.width  = camEl.elt.naturalWidth;   // keep p5's idea of the size current
        camEl.height = camEl.elt.naturalHeight;
        image(camEl, 0, 0, width, height);
        loadPixels();
        // pixels[] is now available — RGBA values for every canvas pixel
        for (let i = 0; i < pixels.length; i += 4) {
            pixels[i] = 255 - pixels[i];     // invert red
            pixels[i+1] = 255 - pixels[i+1]; // invert green
            pixels[i+2] = 255 - pixels[i+2]; // invert blue
        }
        updatePixels();
    }
}

Note: loadPixels() on every frame is CPU-intensive. For heavy processing, consider drawing to an off-screen createGraphics() buffer and only calling loadPixels() on that.

Troubleshooting

Stream doesn't appear

• Check that the IP in the Board field is correct and you pressed Connect
• Open http://<ip>:<port>/stream directly in a browser tab — if you see video there, the Arduino is working and the issue is in the JS
• Check the browser console for errors

"Camera Init failed: 0x106 (ESP_ERR_NOT_SUPPORTED)"

• Wrong board define — the camera sensor PID didn't match. Select the correct CAMERA_MODEL_* for your hardware and re-flash.

Low frame rate

• Reduce resolution with setResolution(FRAMESIZE_QVGA) or increase quality number (lower quality) with setQuality(20)
• Move the Arduino closer to the WiFi access point
• Disable WiFi modem sleep — Pardalote does this automatically with WiFi.setSleep(false)

Image is stuck at 160×120 (QQVGA) and setResolution() won't change it

• Anything above QQVGA needs the board's PSRAM enabled and working. Without it the camera falls back to a single QQVGA (160×120) frame buffer in internal RAM, and any setResolution() to a larger size is refused — you'll see [Camera] No PSRAM — using DRAM, forced to QQVGA in the Serial Monitor. Fix PSRAM and the higher resolutions come back.
• Seeed XIAO ESP32S3: it has 8 MB of octal PSRAM, but you must select it — Tools → PSRAM → OPI PSRAM (not "QSPI PSRAM", not "Disabled"). Re-flash; the "No PSRAM" line should be gone.
• ESP32 WROVER: PSRAM is enabled automatically by the "ESP32 Wrover Module" board, so there's no PSRAM menu to set. If the Serial Monitor shows esp_psram: SPI SRAM memory test fail … writes failed, PSRAM is enabled but the module's chip is failing its power-on test — usually marginal power or a faulty module. Try a full power-cycle with a different USB cable/port, drop Tools → Flash Frequency → 40 MHz, and if it persists use a different WROVER (that PSRAM chip is likely bad).

FRAMESIZE_HD stutters or drops the stream (cam_hal: FB-OVF / net::ERR_INCOMPLETE_CHUNKED_ENCODING)

• FB-OVF means a frame was bigger than the buffer the camera driver set aside for it, so it was dropped (the browser console shows net::ERR_INCOMPLETE_CHUNKED_ENCODING). Older Pardalote versions left too little room for HD frames. Update the Pardalote Arduino library: it now reserves room for frames up to 1600×1200 on boards with PSRAM, and HD streams on the XIAO ESP32S3. If it still happens, check PSRAM is enabled (above), then step down a size (e.g. FRAMESIZE_SVGA, 800×600).

XIAO ESP32S3 won't connect to WiFi

• The Sense board has a u.FL connector for an external antenna. If the antenna switch is set to external and nothing is plugged in, the radio has no antenna and cannot connect. Either attach an external antenna or move the antenna switch to the internal position.

Canvas size doesn't match stream

• The canvas size and stream resolution are independent. image(camEl, 0, 0, width, height) scales the stream to fill the canvas. Adjust createCanvas() dimensions to match your desired aspect ratio.

Black band at the bottom after changing resolution

• p5's createImg() records the image's size once, from the stream's first frame. When a new resolution arrives with a different shape, image() still uses the old size and the picture no longer fills the canvas. Copy the real size in before drawing: camEl.width = camEl.elt.naturalWidth; camEl.height = camEl.elt.naturalHeight; (the example does this).

Only one browser can receive the stream at a time

• This is a fundamental limitation of the ESP32 camera driver. The camera's esp_camera_fb_get() function is single-consumer by design — only one HTTP client can receive MJPEG frames at a time. A second browser connecting will see a black screen and will only get the feed once the first disconnects. This behaviour is identical to Espressif's own CameraWebServer example. Multi-client streaming requires a dedicated FreeRTOS task that owns the camera, copies each frame to a shared PSRAM buffer, and fans it out to all connected clients — this is not currently implemented in Pardalote.