feat: image support (#4011)

Co-authored-by: Emil Atanasov <heitara@gmail.com>
Co-authored-by: Aakansha Doshi <aakansha1216@gmail.com>

src/packages/excalidraw/CHANGELOG.md

@@ -13,6 +13,30 @@ Please add the latest change on the top under the correct section.
 
 ## Unreleased
 
+- Image support.
+
+  NOTE: the unreleased API is highly unstable and may change significantly before the next stable release. As such it's largely undocumented at this point. You are encouraged to read through the [PR](https://github.com/excalidraw/excalidraw/pull/4011) description if you want to know more about the internals.
+
+  General notes:
+
+  - File data are encoded as DataURLs (base64) for portability reasons.
+
+  [ExcalidrawAPI](https://github.com/excalidraw/excalidraw/blob/master/src/packages/excalidraw/README.md#onLibraryChange):
+
+  - added `getFiles()` to get current `BinaryFiles` (`Record<FileId, BinaryFileData>`). It may contain files that aren't referenced by any element, so if you're persisting the files to a storage, you should compare them against stored elements.
+
+  Excalidraw app props:
+
+  - added `generateIdForFile(file: File)` optional prop so you can generate your own ids for added files.
+  - `onChange(elements, appState, files)` prop callback is now passed `BinaryFiles` as third argument.
+  - `onPaste(data, event)` data prop should contain `data.files` (`BinaryFiles`) if the elements pasted are referencing new files.
+  - `initialData` object now supports additional `files` (`BinaryFiles`) attribute.
+
+  Other notes:
+
+  - `.excalidraw` files may now contain top-level `files` key in format of `Record<FileId, BinaryFileData>` when exporting any (image) elements.
+  - Changes were made to various export utilityies exported from the package so that they take `files`. For now, TypeScript should help you figure the changes out.
+
 ## Excalidraw API
 
 ### Features
@@ -380,6 +404,7 @@ Please add the latest change on the top under the correct section.
 - #### BREAKING CHANGE
   Use `location.hash` when importing libraries to fix installation issues. This will require host apps to add a `hashchange` listener and call the newly exposed `excalidrawAPI.importLibrary(url)` API when applicable [#3320](https://github.com/excalidraw/excalidraw/pull/3320). Check the [readme](https://github.com/excalidraw/excalidraw/blob/master/src/packages/excalidraw/README.md#importlibrary) for more details.
 - Append `location.pathname` to `libraryReturnUrl` default url [#3325](https://github.com/excalidraw/excalidraw/pull/3325).
+- Support image elements [#3424](https://github.com/excalidraw/excalidraw/pull/3424).
 
 ### Build
 

src/packages/excalidraw/README_NEXT.md

@@ -379,6 +379,7 @@ To view the full example visit :point_down:
 | [`handleKeyboardGlobally`](#handleKeyboardGlobally) | boolean | false | Indicates whether to bind the keyboard events to document. |
 | [`onLibraryChange`](#onLibraryChange) | <pre>(items: <a href="https://github.com/excalidraw/excalidraw/blob/master/src/types.ts#L200">LibraryItems</a>) => void &#124; Promise&lt;any&gt; </pre> |  | The callback if supplied is triggered when the library is updated and receives the library items. |
 | [`autoFocus`](#autoFocus) | boolean | false | Implies whether to focus the Excalidraw component on page load |
+| [`generateIdForFile`](#generateIdForFile) | `(file: File) => string | Promise<string>` | Allows you to override `id` generation for files added on canvas |
 
 ### Dimensions of Excalidraw
 
@@ -448,7 +449,8 @@ You can pass a `ref` when you want to access some excalidraw APIs. We expose the
 | --- | --- | --- |
 | ready | `boolean` | This is set to true once Excalidraw is rendered |
 | readyPromise | [resolvablePromise](https://github.com/excalidraw/excalidraw/blob/master/src/utils.ts#L317) | This promise will be resolved with the api once excalidraw has rendered. This will be helpful when you want do some action on the host app once this promise resolves. For this to work you will have to pass ref as shown [here](#readyPromise) |
-| [updateScene](#updateScene) | <pre>(<a href="https://github.com/excalidraw/excalidraw/blob/master/src/types.ts#L207">sceneData</a>)) => void </pre> | updates the scene with the sceneData |
+| [updateScene](#updateScene) | <pre>(scene: <a href="https://github.com/excalidraw/excalidraw/blob/master/src/types.ts#L207">sceneData</a>) => void </pre> | updates the scene with the sceneData |
+| [addFiles](#addFiles) | <pre>(files: <a href="https://github.com/excalidraw/excalidraw/blob/master/src/types.ts">BinaryFileData</a>) => void </pre> | add files data to the appState |
 | resetScene | `({ resetLoadingState: boolean }) => void` | Resets the scene. If `resetLoadingState` is passed as true then it will also force set the loading state to false. |
 | getSceneElementsIncludingDeleted | <pre> () => <a href="https://github.com/excalidraw/excalidraw/blob/master/src/element/types.ts#L78">ExcalidrawElement[]</a></pre> | Returns all the elements including the deleted in the scene |
 | getSceneElements | <pre> () => <a href="https://github.com/excalidraw/excalidraw/blob/master/src/element/types.ts#L78">ExcalidrawElement[]</a></pre> | Returns all the elements excluding the deleted in the scene |
@@ -471,7 +473,7 @@ Since plain object is passed as a `ref`, the `readyPromise` is resolved as soon
 ### `updateScene`
 
 <pre>
-(<a href="https://github.com/excalidraw/excalidraw/blob/master/src/types.ts#L207">sceneData</a>)) => void
+(scene: <a href="https://github.com/excalidraw/excalidraw/blob/master/src/types.ts#L207">sceneData</a>) => void
 </pre>
 
 You can use this function to update the scene with the sceneData. It accepts the below attributes.
@@ -483,6 +485,12 @@ You can use this function to update the scene with the sceneData. It accepts the
 | `collaborators` | <pre>Map<string, <a href="https://github.com/excalidraw/excalidraw/blob/master/src/types.ts#L29">Collaborator></a></pre> | The list of collaborators to be updated in the scene. |
 | `commitToHistory` | `boolean` | Implies if the `history (undo/redo)` should be recorded. Defaults to `false`. |
 
+### `addFiles`
+
+<pre>(files: <a href="https://github.com/excalidraw/excalidraw/blob/master/src/types.ts">BinaryFileData</a>) => void </pre>
+
+Adds supplied files data to the `appState.files` cache, on top of existing files present in the cache.
+
 #### `onCollabButtonClick`
 
 This callback is triggered when clicked on the collab button in excalidraw. If not supplied, the collab dialog button is not rendered.
@@ -662,6 +670,14 @@ The unique id of the excalidraw component. This can be used to identify the exca
 
 This prop implies whether to focus the Excalidraw component on page load. Defaults to false.
 
+#### `generateIdForFile`
+
+Allows you to override `id` generation for files added on canvas (images). By default, an SHA-1 digest of the file is used.
+
+```
+(file: File) => string | Promise<string>
+```
+
 ### Does it support collaboration ?
 
 No, Excalidraw package doesn't come with collaboration built in, since the implementation is specific to each host app. We expose APIs which you can use to communicate with Excalidraw which you can use to implement it. You can check our own implementation [here](https://github.com/excalidraw/excalidraw/blob/master/src/excalidraw-app/index.tsx).

src/packages/excalidraw/index.tsx

@@ -34,6 +34,7 @@ const Excalidraw = (props: ExcalidrawProps) => {
     handleKeyboardGlobally = false,
     onLibraryChange,
     autoFocus = false,
+    generateIdForFile,
   } = props;
 
   const canvasActions = props.UIOptions?.canvasActions;
@@ -94,6 +95,7 @@ const Excalidraw = (props: ExcalidrawProps) => {
         handleKeyboardGlobally={handleKeyboardGlobally}
         onLibraryChange={onLibraryChange}
         autoFocus={autoFocus}
+        generateIdForFile={generateIdForFile}
       />
     </InitializeApp>
   );
@@ -187,3 +189,9 @@ export {
 export { isLinearElement } from "../../element/typeChecks";
 
 export { FONT_FAMILY, THEME } from "../../constants";
+
+export {
+  mutateElement,
+  newElementWith,
+  bumpVersion,
+} from "../../element/mutateElement";

src/packages/utils.ts

@@ -3,14 +3,16 @@ import {
   exportToSvg as _exportToSvg,
 } from "../scene/export";
 import { getDefaultAppState } from "../appState";
-import { AppState } from "../types";
+import { AppState, BinaryFiles } from "../types";
 import { ExcalidrawElement } from "../element/types";
 import { getNonDeletedElements } from "../element";
 import { restore } from "../data/restore";
+import { MIME_TYPES } from "../constants";
 
 type ExportOpts = {
   elements: readonly ExcalidrawElement[];
   appState?: Partial<Omit<AppState, "offsetTop" | "offsetLeft">>;
+  files: BinaryFiles | null;
   getDimensions?: (
     width: number,
     height: number,
@@ -20,6 +22,7 @@ type ExportOpts = {
 export const exportToCanvas = ({
   elements,
   appState,
+  files,
   getDimensions = (width, height) => ({ width, height, scale: 1 }),
 }: ExportOpts) => {
   const { elements: restoredElements, appState: restoredAppState } = restore(
@@ -31,6 +34,7 @@ export const exportToCanvas = ({
   return _exportToCanvas(
     getNonDeletedElements(restoredElements),
     { ...restoredAppState, offsetTop: 0, offsetLeft: 0, width: 0, height: 0 },
+    files || {},
     { exportBackground, viewBackgroundColor },
     (width: number, height: number) => {
       const canvas = document.createElement("canvas");
@@ -44,22 +48,23 @@ export const exportToCanvas = ({
   );
 };
 
-export const exportToBlob = (
+export const exportToBlob = async (
   opts: ExportOpts & {
     mimeType?: string;
     quality?: number;
   },
 ): Promise<Blob | null> => {
-  const canvas = exportToCanvas(opts);
+  const canvas = await exportToCanvas(opts);
 
-  let { mimeType = "image/png", quality } = opts;
+  let { mimeType = MIME_TYPES.png, quality } = opts;
 
-  if (mimeType === "image/png" && typeof quality === "number") {
-    console.warn(`"quality" will be ignored for "image/png" mimeType`);
+  if (mimeType === MIME_TYPES.png && typeof quality === "number") {
+    console.warn(`"quality" will be ignored for "${MIME_TYPES.png}" mimeType`);
   }
 
+  // typo in MIME type (should be "jpeg")
   if (mimeType === "image/jpg") {
-    mimeType = "image/jpeg";
+    mimeType = MIME_TYPES.jpg;
   }
 
   quality = quality ? quality : /image\/jpe?g/.test(mimeType) ? 0.92 : 0.8;
@@ -78,6 +83,7 @@ export const exportToBlob = (
 export const exportToSvg = async ({
   elements,
   appState = getDefaultAppState(),
+  files = {},
   exportPadding,
 }: Omit<ExportOpts, "getDimensions"> & {
   exportPadding?: number;
@@ -87,10 +93,14 @@ export const exportToSvg = async ({
     null,
     null,
   );
-  return _exportToSvg(getNonDeletedElements(restoredElements), {
-    ...restoredAppState,
-    exportPadding,
-  });
+  return _exportToSvg(
+    getNonDeletedElements(restoredElements),
+    {
+      ...restoredAppState,
+      exportPadding,
+    },
+    files,
+  );
 };
 
 export { serializeAsJSON } from "../data/json";