Project overview
"); ``` ### Ownership Like other CoValues, you can specify ownership when creating CoTexts. ```ts // Create with shared ownership const teamGroup = Group.create(); teamGroup.addMember(colleagueAccount, "writer"); const teamNote = co.plainText().create("Team updates", { owner: teamGroup }); ``` See [Groups as permission scopes](/docs/permissions-and-sharing/overview) for more information on how to use groups to control access to CoText values. ## Reading Text CoText values work similarly to JavaScript strings: ```ts // Get the text content console.log(note.toString()); // "Meeting notes" console.log(`${note}`); // "Meeting notes" // Check the text length console.log(note.length); // 14 ``` ## Making Edits Insert and delete text with intuitive methods: ```ts // Insert text at a specific position note.insertBefore(8, "weekly "); // "Meeting weekly notes" // Insert after a position note.insertAfter(21, " for Monday"); // "Meeting weekly notes for Monday" // Delete a range of text note.deleteRange({ from: 8, to: 15 }); // "Meeting notes for Monday" // Apply a diff to update the entire text note.$jazz.applyDiff("Team meeting notes for Tuesday"); ``` ### Applying Diffs Use `applyDiff` to efficiently update text with minimal changes: ```ts // Original text: "Team status update" const minutes = co.plainText().create("Team status update"); // Replace the entire text with a new version minutes.$jazz.applyDiff("Weekly team status update for Project X"); // Make partial changes let text = minutes.toString(); text = text.replace("Weekly", "Monday"); minutes.$jazz.applyDiff(text); // Efficiently updates only what changed ``` Perfect for handling user input in form controls: ## Using Rich Text with ProseMirror Jazz provides a dedicated plugin for integrating `co.richText()` with the popular ProseMirror editor that enables bidirectional synchronization between your co.richText() instances and ProseMirror editors. ### ProseMirror Plugin Features * **Bidirectional Sync**: Changes in the editor automatically update the `co.richText()` and vice versa * **Real-time Collaboration**: Multiple users can edit the same document simultaneously * **HTML Conversion**: Automatically converts between HTML (used by `co.richText()`) and ProseMirror's document model ### Installation ```bash pnpm add prosemirror-view \ prosemirror-state \ prosemirror-schema-basic ``` ### Integration We don't currently have a React Native-specific example, but you need help you can [request one](https://github.com/garden-co/jazz/issues/new), or ask on [Discord](https://discord.gg/utDMjHYg42). For use with React: ```tsx function RichTextEditor() { const me = useAccount(JazzAccount, { resolve: { profile: { bio: true } } }); const editorRef = useRefInitial content
"); const editorContainer = document.getElementById("editor") as HTMLDivElement; const cleanup = setupRichTextEditor(doc, editorContainer); // Later when done with the editor cleanup(); ``` ### FileStreams # FileStreams FileStreams handle binary data in Jazz applications - think documents, audio files, and other non-text content. They're essentially collaborative versions of `Blob`s that sync automatically across devices. Use FileStreams when you need to: * Distribute documents across devices * Store audio or video files * Sync any binary data between users **Note:** For images specifically, Jazz provides the higher-level `ImageDefinition` abstraction which manages multiple image resolutions - see the [ImageDefinition documentation](/docs/core-concepts/covalues/imagedef) for details. FileStreams provide automatic chunking when using the `createFromBlob` method, track upload progress, and handle MIME types and metadata. In your schema, reference FileStreams like any other CoValue: **File name: schema.ts** ```ts import { co, z } from "jazz-tools"; const Document = co.map({ title: z.string(), file: co.fileStream(), // Store a document file }); ``` ## Creating FileStreams There are two main ways to create FileStreams: creating empty ones for manual data population or creating directly from existing files or blobs. ### Creating from Blobs and Files For files from input elements or drag-and-drop interfaces, use `createFromBlob`: ```ts // From a file input const fileInput = document.querySelector( 'input[type="file"]', ) as HTMLInputElement; fileInput.addEventListener("change", async () => { const file = fileInput.files?.[0]; if (!file) return; // Create FileStream from user-selected file const fileStream = await co .fileStream() .createFromBlob(file, { owner: myGroup }); // Or with progress tracking for better UX const fileWithProgress = await co.fileStream().createFromBlob(file, { onProgress: (progress) => { // progress is a value between 0 and 1 const percent = Math.round(progress * 100); console.log(`Upload progress: ${percent}%`); progressBar.style.width = `${percent}%`; }, owner: myGroup, }); }); ``` ### Creating Empty FileStreams Create an empty FileStream when you want to manually [add binary data in chunks](#writing-to-filestreams): ```ts const fileStream = co.fileStream().create({ owner: myGroup }); ``` ### Ownership Like other CoValues, you can specify ownership when creating FileStreams. ```ts // Create a team group const teamGroup = Group.create(); teamGroup.addMember(colleagueAccount, "writer"); // Create a FileStream with shared ownership const teamFileStream = co.fileStream().create({ owner: teamGroup }); ``` See [Groups as permission scopes](/docs/permissions-and-sharing/overview) for more information on how to use groups to control access to FileStreams. ## Reading from FileStreams `FileStream`s provide several ways to access their binary content, from raw chunks to convenient Blob objects. ### Getting Raw Data Chunks To access the raw binary data and metadata: ```ts // Get all chunks and metadata const fileData = fileStream.getChunks(); if (fileData) { console.log(`MIME type: ${fileData.mimeType}`); console.log(`Total size: ${fileData.totalSizeBytes} bytes`); console.log(`File name: ${fileData.fileName}`); console.log(`Is complete: ${fileData.finished}`); // Access raw binary chunks for (const chunk of fileData.chunks) { // Each chunk is a Uint8Array console.log(`Chunk size: ${chunk.length} bytes`); } } ``` By default, `getChunks()` only returns data for completely synced `FileStream`s. To start using chunks from a `FileStream` that's currently still being synced use the `allowUnfinished` option: ```ts // Get data even if the stream isn't complete const partialData = fileStream.getChunks({ allowUnfinished: true }); ``` ### Converting to Blobs For easier integration with web APIs, convert to a `Blob`: ```ts // Convert to a Blob const blob = fileStream.toBlob(); // Get the filename from the metadata const filename = fileStream.getChunks()?.fileName; if (blob) { // Use with URL.createObjectURL const url = URL.createObjectURL(blob); // Create a download link const link = document.createElement("a"); link.href = url; link.download = filename || "document.pdf"; link.click(); // Clean up when done URL.revokeObjectURL(url); } ``` ### Loading FileStreams as Blobs You can directly load a `FileStream` as a `Blob` when you only have its ID: ```ts // Load directly as a Blob when you have an ID const blobFromID = await co.fileStream().loadAsBlob(fileStreamId); // By default, waits for complete uploads // For in-progress uploads: const partialBlob = await co.fileStream().loadAsBlob(fileStreamId, { allowUnfinished: true, }); ``` ### Checking Completion Status Check if a `FileStream` is fully synced: ```ts if (fileStream.isBinaryStreamEnded()) { console.log("File is completely synced"); } else { console.log("File upload is still in progress"); } ``` ## Writing to FileStreams When creating a `FileStream` manually (not using `createFromBlob`), you need to manage the upload process yourself. This gives you more control over chunking and progress tracking. ### The Upload Lifecycle `FileStream` uploads follow a three-stage process: 1. **Start** \- Initialize with metadata 2. **Push** \- Send one or more chunks of data 3. **End** \- Mark the stream as complete ### Starting a `FileStream` Begin by providing metadata about the file: ```ts // Create an empty FileStream const manualFileStream = co.fileStream().create({ owner: myGroup }); // Initialize with metadata manualFileStream.start({ mimeType: "application/pdf", // MIME type (required) totalSizeBytes: 1024 * 1024 * 2, // Size in bytes (if known) fileName: "document.pdf", // Original filename (optional) }); ``` ### Pushing Data Add binary data in chunks - this helps with large files and progress tracking: ```ts const data = new Uint8Array(arrayBuffer); // For large files, break into chunks (e.g., 100KB each) const chunkSize = 1024 * 100; for (let i = 0; i < data.length; i += chunkSize) { // Create a slice of the data const chunk = data.slice(i, i + chunkSize); // Push chunk to the FileStream fileStream.push(chunk); // Track progress const progress = Math.min( 100, Math.round(((i + chunk.length) * 100) / data.length), ); console.log(`Upload progress: ${progress}%`); } // Finalise the upload fileStream.end(); console.log("Upload complete!"); ``` ### Completing the Upload Once all chunks are pushed, mark the `FileStream` as complete: ```ts // Finalise the upload fileStream.end(); console.log("Upload complete!"); ``` ## Subscribing to `FileStream`s Like other CoValues, you can subscribe to `FileStream`s to get notified of changes as they happen. This is especially useful for tracking upload progress when someone else is uploading a file. ### Loading by ID Load a `FileStream` when you have its ID: ```ts const fileStreamFromId = await co.fileStream().load(fileStreamId); if (fileStream.$isLoaded) { console.log("FileStream loaded successfully"); // Check if it's complete if (fileStream.isBinaryStreamEnded()) { // Process the completed file const blob = fileStream.toBlob(); } } ``` ### Subscribing to Changes Subscribe to a `FileStream` to be notified when chunks are added or when the upload is complete: ```ts const unsubscribe = co .fileStream() .subscribe(fileStreamId, (fileStream: FileStream) => { // Called whenever the FileStream changes console.log("FileStream updated"); // Get current status const chunks = fileStream.getChunks({ allowUnfinished: true }); if (chunks) { const uploadedBytes = chunks.chunks.reduce( (sum: number, chunk: Uint8Array) => sum + chunk.length, 0, ); const totalBytes = chunks.totalSizeBytes || 1; const progress = Math.min( 100, Math.round((uploadedBytes * 100) / totalBytes), ); console.log(`Upload progress: ${progress}%`); if (fileStream.isBinaryStreamEnded()) { console.log("Upload complete!"); // Now safe to use the file const blob = fileStream.toBlob(); // Clean up the subscription if we're done unsubscribe(); } } }); ``` ### Waiting for Upload Completion If you need to wait for a `FileStream` to be fully synchronized across devices: ```ts // Wait for the FileStream to be fully synced await fileStream.$jazz.waitForSync({ timeout: 5000, // Optional timeout in ms }); console.log("FileStream is now synced to all connected devices"); ``` This is useful when you need to ensure that a file is available to other users before proceeding with an operation. ### CoVectors # CoVectors CoVectors let you store and query highβdimensional vectors directly in Jazz apps. They are ideal for semantic search, or personalization features that work offline, sync across devices, and remain endβtoβend encrypted. The [Journal example](https://github.com/garden-co/jazz/tree/main/examples/vector-search) demonstrates semantic search using of CoVector. CoVectors are defined using `co.vector()`, and are often used as fields in a CoMap within a CoList (making it easy to perform vector search across list items). ```ts import { co, z } from "jazz-tools"; const Embedding = co.vector(384); // Define 384-dimensional embedding const Document = co.map({ content: z.string(), embedding: Embedding, }); export const DocumentsList = co.list(Document); ``` The number of dimensions matches the embedding model used in your app. Many small sentence transformers produce 384βdim vectors; others use 512, 768, 1024 or more. ## Creating CoVectors You can create vectors in your Jazz application from an array of numbers, or Float32Array instance. ```ts // Generate embeddings (bring your own embeddings model) const vectorData = await createEmbedding("Text"); const newDocument = Document.create({ content: "Text", embedding: Embedding.create(vectorData), }); documents.$jazz.push(newDocument); ``` ### Ownership Like other CoValues, you can specify ownership when creating CoVectors. ```ts // Create with shared ownership const teamGroup = Group.create(); teamGroup.addMember(colleagueAccount, "writer"); const teamList = co.vector(384).create(vector, { owner: teamGroup }); ``` See [Groups as permission scopes](/docs/permissions-and-sharing/overview) for more information on how to use groups to control access to CoVectors. ### Immutability CoVectors cannot be changed after creation. Instead, create a new CoVector with the updated values and replace the previous one. ## Semantic Search Semantic search lets you find data based on meaning, not just keywords. In Jazz, you can easily sort results by how similar they are to your search query. ```tsx import { useCoState } from "jazz-tools/react"; const { queryEmbedding } = useCreateEmbedding(); const foundDocuments = useCoState(DocumentsList, documentsListId, { resolve: { $each: { embedding: true }, }, select(documents) { if (!documents.$isLoaded) return; // If no query embedding, return all entries if (!queryEmbedding) return documents.map((value) => ({ value })); return documents .map((value) => ({ value, similarity: value.embedding.$jazz.cosineSimilarity(queryEmbedding), // [!code ++] })) .sort((a, b) => b.similarity - a.similarity) .filter((result) => result.similarity > 0.5); }, }); ``` Wrapping each item with its similarity score makes it easy to sort, filter, and display the most relevant results. This approach is widely used in vector search and recommendation systems, since it keeps both the data and its relevance together for further processing or display. ### Cosine Similarity To compare how similar two vectors are, we use their [cosine similarity](https://en.wikipedia.org/wiki/Cosine%5Fsimilarity). This returns a value between `-1` and `1`, describing how similar the vectors are: * `1` means the vectors are identical * `0` means the vectors are orthogonal (i.e. no similarity) * `-1` means the vectors are opposite direction (perfectly dissimilar). If you sort items by their cosine similarity, the ones which are most similar will appear at the top of the list. Jazz provides a built-in `$jazz.cosineSimilarity` method to calculate this for you. ## Embedding Models CoVectors handles storage and search, you provide the vectors. Generate embeddings with any model you prefer (Hugging Face, OpenAI, custom, etc). **Recommended:** Run models locally for privacy and offline support using [Transformers.js](https://huggingface.co/docs/transformers.js). Check our [Journal app example](https://github.com/garden-co/jazz/tree/main/examples/vector-search) to see how to do this. The following models offer a good balance between accuracy and performance: * [Xenova/all-MiniLM-L6-v2](https://huggingface.co/Xenova/all-MiniLM-L6-v2) β 384 dimensions, \~23 MB * [Xenova/paraphrase-multilingual-mpnet-base-v2](https://huggingface.co/Xenova/paraphrase-multilingual-mpnet-base-v2) β 768 dimensions, \~279 MB * [mixedbread-ai/mxbai-embed-large-v1](https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1) β 1024 dimensions, \~337 MB * [Browse more models β](https://huggingface.co/models?pipeline%5Ftag=feature-extraction&library=transformers.js) Alternatively, you can generate embeddings using server-side or commercial APIs (such as OpenAI or Anthropic). ## Best Practices ### Changing embedding models **Always use the same embedding model for all vectors you intend to compare.**Mixing vectors from different models (or even different versions of the same model) will result in meaningless similarity scores, as the vector spaces are not compatible. If you need to switch models, consider storing the model identifier alongside each vector, and re-embedding your data as needed. ### ImageDefinitions # ImageDefinition `ImageDefinition` is a specialized CoValue designed specifically for managing images in Jazz applications. It extends beyond basic file storage by supporting a blurry placeholder, built-in resizing, and progressive loading patterns. Beyond `ImageDefinition`, Jazz offers higher-level functions and components that make it easier to use images: * [createImage()](#creating-images) \- function to create an `ImageDefinition` from a file * [loadImage, loadImageBySize, highestResAvailable](#displaying-images) \- functions to load and display images * [Image](#displaying-images) \- Component to display an image ## Installation \[!framework=react-native\] Jazz's images implementation is based on `@bam.tech/react-native-image-resizer`. Check the [installation guide](/docs/react-native/project-setup#install-dependencies) for more details. ## Creating Images The easiest way to create and use images in your Jazz application is with the `createImage()` function: ```ts import { createImage } from "jazz-tools/media"; import { launchImageLibrary } from "react-native-image-picker"; async function handleImagePicker() { // Use your favorite image picker library to get the image URI const result = await launchImageLibrary({ mediaType: "photo", quality: 1, }); if ( !result.didCancel && result.assets && result.assets.length > 0 && me.profile.$isLoaded ) { // Creates ImageDefinition with a blurry placeholder, limited to 1024px on the longest side, and multiple resolutions automatically. // See the options below for more details. const image = await createImage(result.assets[0].uri ?? "", { owner: me.$jazz.owner, maxSize: 1024, placeholder: "blur", progressive: true, }); // Store the image me.profile.$jazz.set("image", image); } } ``` The `createImage()` function: * Creates an `ImageDefinition` with the right properties * Optionally generates a small placeholder for immediate display * Creates multiple resolution variants of your image * Returns the created `ImageDefinition` ### Configuration Options ```ts declare function createImage( image: Blob | File | string, options?: { owner?: Group | Account; placeholder?: false | "blur"; maxSize?: number; progressive?: boolean; }, ): Promise