Custom sources

Custom sources allow you to publish media that doesn't come straight from the camera, microphone or screen share, for example a game render loop or an ML-processed camera feed. Any content you can produce as video frames or audio samples can be published.

important

If you only wish to send plain camera, microphone or screen share output through Fishjam, then you most likely should refer to Streaming media, Managing devices and Screen sharing instead of this section.

Every custom source follows the same contract: you register a MediaStream under a stable source ID with the useCustomSource hook (Web · React Native), and Fishjam publishes its video track as a customVideo track and its audio track as a customAudio track. Other peers receive them like any other track.

Choose a guide

Guide	Platform	Use it when
Web	🌐 Web	You have a MediaStream from a canvas, a media element, Web Audio, or a library like Smelter
React Native	📱 Mobile	You have a React Native MediaStream to publish, or want to publish app-generated audio
Vision Camera	📱 Mobile	You want to publish a VisionCamera feed, optionally running frame processors on it
WebGPU effects	🌐 Web · 📱 Mobile	You want to draw your own shaders, overlays or effects into the published video
Low-level frame API	📱 Mobile	You produce video frames yourself, e.g. from a native pipeline or your own renderer

Platform support

On the web you can publish both video and audio from any MediaStream. In React Native, custom video comes from the video guides in this section, and custom audio from the useCustomAudioSource hook, which publishes PCM samples your app pushes.

Receiving custom tracks

Custom tracks published by other peers show up in the usePeers hook (Web · React Native), bucketed into the customVideoTracks and customAudioTracks arrays of each peer.

React (Web)

import { usePeers } from "@fishjam-cloud/react-client";

export function RemoteCustomVideos() {
  const { remotePeers } = usePeers();

  return (
    <ul>
      {remotePeers.map((peer) => (
        <li key={peer.id}>
          {peer.customVideoTracks.map((track) => (
            <VideoRenderer key={track.trackId} stream={track.stream} />
          ))}
        </li>
      ))}
    </ul>
  );
}

Remote customAudioTracks are played by attaching each track's stream to an <audio> element, the same way as microphone tracks.

React Native (Mobile)

import React from "react";
import { View } from "react-native";
import { usePeers, RTCView } from "@fishjam-cloud/react-native-client";

export function RemoteCustomVideos() {
  const { remotePeers } = usePeers();

  return (
    <View>
      {remotePeers.map((peer) =>
        peer.customVideoTracks.map(
          (track) =>
            track.stream && (
              <RTCView
                key={track.trackId}
                mediaStream={track.stream}
                style={{ height: 200, width: 200 }}
                objectFit="cover"
              />
            ),
        ),
      )}
    </View>
  );
}

Incoming customAudioTracks are played back automatically; you don't render anything for them.

Guides in this section

📄️Web 🌐

Stream non-standard video or audio sources (e.g. WebGL, WebGPU, Three.js) through Fishjam in web apps.

📄️React Native 📱

Publish any React Native MediaStream or app-generated audio to a Fishjam room with the useCustomSource and useCustomAudioSource hooks.

📄️Vision Camera 📱

Publish a react-native-vision-camera feed to Fishjam, optionally running frame-processor plugins on the published frames.

📄️WebGPU effects

Draw your own shaders, overlays and effects into the video you publish to Fishjam, with WebGPU on the web and in React Native.

📄️Low-level frame API 📱

Publish video frames from any source, such as a native ML pipeline or your own renderer, with the generic custom video source layer.

Further reading

• How custom sources work: concepts behind the pipeline
• API reference: useCustomSource (Web), useCustomSource (React Native), useCustomAudioSource (React Native), Vision Camera Source package, Custom Video Source package