Custom sources

Custom sources allow you to publish media that doesn't come straight from the camera, microphone or screen share, for example a game render loop or an ML-processed camera feed. Any content you can produce as video frames or audio samples can be published.

important

If you only wish to send plain camera, microphone or screen share output through Fishjam, then you most likely should refer to Streaming media, Managing devices and Screen sharing instead of this section. To blur the background or draw effects into your camera, see Camera effects.

Every custom source follows the same contract: you register a MediaStream under a stable source ID with the useCustomSource hook (Web · React Native), and Fishjam publishes its video track as a customVideo track and its audio track as a customAudio track. Other peers receive them like any other track.

Choose a guide

Guide	Platform	Use it when
Web	🌐 Web	You have a MediaStream from a canvas, a media element, Web Audio, or a library like Smelter
React Native	📱 Mobile	You have a React Native MediaStream to publish, or want to publish app-generated audio
Low-level frame API	📱 Mobile	You produce video frames yourself, e.g. from a native pipeline or your own renderer
WebGPU effects	🌐 Web	You render your own shaders into a canvas and want to publish it

Platform support

On the web you can publish both video and audio from any MediaStream. In React Native, custom video comes from the low-level frame API or the VisionCamera integration, and custom audio from the useCustomAudioSource hook, which publishes PCM samples your app pushes.

Publishing from VisionCamera?

If your app captures the camera with VisionCamera, publish that feed as a custom source with the VisionCamera integration.

Receiving custom tracks

Custom tracks published by other peers show up in the usePeers hook (Web · React Native), bucketed into the customVideoTracks and customAudioTracks arrays of each peer.

React (Web)

import { usePeers } from "@fishjam-cloud/react-client";

export function RemoteCustomVideos() {
  const { remotePeers } = usePeers();

  return (
    <ul>
      {remotePeers.map((peer) => (
        <li key={peer.id}>
          {peer.customVideoTracks.map((track) => (
            <VideoRenderer key={track.trackId} stream={track.stream} />
          ))}
        </li>
      ))}
    </ul>
  );
}

Remote customAudioTracks are played by attaching each track's stream to an <audio> element, the same way as microphone tracks.

React Native (Mobile)

import React from "react";
import { View } from "react-native";
import { usePeers, RTCView } from "@fishjam-cloud/react-native-client";

export function RemoteCustomVideos() {
  const { remotePeers } = usePeers();

  return (
    <View>
      {remotePeers.map((peer) =>
        peer.customVideoTracks.map(
          (track) =>
            track.stream && (
              <RTCView
                key={track.trackId}
                mediaStream={track.stream}
                style={{ height: 200, width: 200 }}
                objectFit="cover"
              />
            ),
        ),
      )}
    </View>
  );
}

Incoming customAudioTracks are played back automatically; you don't render anything for them.

Guides in this section

📄️Web 🌐

Stream non-standard video or audio sources (e.g. WebGL, WebGPU, Three.js) through Fishjam in web apps.

📄️React Native 📱

Publish any React Native MediaStream or app-generated audio to a Fishjam room with the useCustomSource and useCustomAudioSource hooks.

📄️Low-level frame API 📱

Publish video frames from any source, such as a native ML pipeline or your own renderer, with the generic custom video source layer.

Further reading

• How custom sources work: concepts behind the pipeline
• API reference: useCustomSource (Web), useCustomSource (React Native), useCustomAudioSource (React Native), Custom Video Source package