Electron, not a walk in the park

by
Tags: , ,
Category: , ,

Recently, a project I worked on was considering using Electron as a fallback technology for an initial Progressive Web Application. At the time, the assumption was that since Electron uses Chromium, a browser, it should allow application developers to not only use the features of a PWA but also gain native access to technologies, such as local databases, file storage, enhanced networking, and it should still be able to run the existing PWA service worker for offline support.

After spending a couple of days evaluating the Electron platform and its capabilities compared to PWAs, I ultimately recommended against using it. In this article I’ll review some of the issues I found along the way that led to my position.

About Electron

Electron is an open-source technology for developing cross-platform (Linux, OS X and Windows) desktop applications, written in JavaScript, HTML and CSS, and hosted in Chromium and Node.js.

Electron applications are everywhere: from Visual Studio Code, to Slack, to Discord, Skype, Atom, and even Teams. The applications can be delivered and install natively to each OS desktop, and can be automatically updated.

There are a bevy of resources available in Electron, though you have to pay close attention to when they were written as the platform has undergone changes over the years. Here are a few key sites:

In short, Electron is a platform that makes creating desktop applications possible using HTML, CSS and JavaScript. It has integration with OS-level resources, and can do anything a Node.js application can do, running a user interface using Chromium, the open source browser that Google Chrome is based upon.

Electron Processes

There are basically two sides of every Electron application – the Main Process and BrowserWindow processes.

The BrowserWindow processes launch to provide a Chromium-based browser to provide a web-based user interface, and communicates with the Main Process via a couple of inter-process communication mechanisms (which I'll get into later).

The Main Process can do anything a Node.js application can do, including invoking native C methods, access networking, file storage, launch additional processes, and anything else you can dream up.

The BrowserWindow process does not access Node.js APIs, only standard browser APIs (with some limits) and Electron APIs that are allowed to be called from the BrowserWindow side of the application. There are several ways for the BrowserWindow process to access the engine running in the Main Process:

  • Use an inter-process communication API (ipcMain and ipcRenderer are the two classes provided to create event handlers and send events to the handlers). Both fire-and-forget send and synchronous response sendSync methods are exposed.

  • Expose an API to the BrowserWindow (window) via the contextBridge API, which can further front the messaging system and send messages via the inter-process communication APIs.

  • (NOT GENERALLY RECOMMENDED) Use Electron Remote, a project to expose Main Process objects in the renderer (Browser Window) as if they existed within the Sandbox. This replaces a deprecated built-in feature with an optional opt-in plugin.

A diagram of the platform:

Concern: Architecting the inner APIs can prove challenging

From what I can see, the contextBridge + IPC approach is probably the way to go for most applications. However, keep in mind, for a single method, you must:

  1. Create the method to expose

  2. If using TypeScript (don't get me started, more below on that) and exposing non-native (read: interfaces/classes) shapes for parameters/return values, create a shared type object and import it to the method and places that use it

  3. Expose the method via an Electron IPC EventHandler from the Main Process (loaded starting with main.js). Determine whether you want the event to return a value or send a return message on the IPC framework later

  4. In the preload.js file, configure your contextBridge to expose method calls which send the messages via IPC and process the return values

  5. In your front-end application (say, React), create a button/menu event handler and then have it call the API, which now hangs off of window.

An example follows. I'm using TypeScript to nail down specific types in this demo:

First, the shape of the data object. Ye olde' task manager app rears its ugly head. I spent my creative juices fighting crazy build errors, so no cool sample code here:

// src/shared/task.ts

export interface Task {
    id?: string;
    description: string;
    priority: number;
    due: Date;
    complete?: Date;
}

Next, I'll create a database API that talks to SQLite. The full class is available in the downloaded sample, but boils down to something like this:

// src/main/apis/task-list.ts

import {Task} from '../../shared/task';
export function getAll(): Promise {
    return new Promise((resolve, reject) => {
        try {
            const stmt = db.prepare(`
                SELECT id,
                       description,
                       priority,
                       due,
                       completed
                FROM Tasks
            `);
            // n.b. - all brings all rows back - don't use get (see below)
            const result = stmt.all();
            ...
            resolve(result);
            ... 
        } catch (e) {
            reject(e);
        }
    });
}

Wiring the handler to inter-process communication on the main bus :

// task-event-handlers.ts, called by main.ts
...
  ipcMain.on('add-task', (e: IpcMainEvent, arg: any) => {
    (async () => {
      try {
        console.log(`add-task. Got ${JSON.stringify(arg, null, 2)} as arg`);
        const uuid = await addTask(arg);
        console.log(`Got uuid back from addTask ${uuid}`);
        e.returnValue = uuid;
      } catch (error) {
        // TODO - how to best handle errors from IPC handlers.
        //        Not a fan of having a polymorphic returnValue here for errors.
        e.returnValue = error;
        console.error(error);
      }
    })();
...

Expose the IPC method using the preload.js script to configure the contextBridge:

import { contextBridge, ipcRenderer } from 'electron';
import {Task} from "../shared/task.ts";

contextBridge.exposeInMainWorld('MainAPI', {
    ...
    getAllTasks: (): Task[] => {
        const tasks = ipcRenderer.sendSync('get-all-tasks');
        return tasks;
    },
    ...
});

After doing the above, the BrowserWindow process can call window.MainAPI.getAllTasks(), which will send the IPC a request for the get-all-tasks handler, which runs getAll() from the main process. The event returns database rows (synchronously in this example) back to the caller. There are other ways to send async responses which we hope to cover in a future blog post.

Now, we wire the IPC call up from the component in React. This is from a component that sends a message to the IPC:

import { useEffect, useState } from 'react';
import { Task } from '../../../shared/task.ts';

export default function TaskList() {
  const [tasks, setTasks] = useState([]);
  useEffect(() => {
    try {
      const tasks = window.MainAPI.getAllTasks();
      console.log('got back results from loadTasks in component');
      console.dir(tasks);
      setTasks(tasks);
    } catch (e) {
      setTasks([]);
    }
  }, []);

  const taskRows = tasks.map((t: Task) => (
    
      {t.id}
      {t.description}
      {t.priority}
      {t.due ? t.due.toLocaleDateString() : 'No due date'} 
      {t.complete ? t.complete.toLocaleDateString() : 'No'}
    
  ))
  return (
    
    <section class="task-list container">
      <h1>Tasks
      { tasks?.length > 0 &&
        <table>
          <tr>
            <th>ID
            <th>Description
            <th>Priority
            <th>Due
            <th>Completed?
          </tr>
          { taskRows }
        </table>
      }
      { (!tasks || tasks.length === 0) &&
        <p>No tasks.

      }
    </section>
   );
}

Electron does not simplify desktop Javascript

In my opinion, writing desktop applications in Electron seems like a lot of work. You have to manage inter-process communications, suss out shared types, optionally install an event bridge to hide that complexity, then for each method bolt it onto this framework.

You also should come up with some good namespaces for your IPC call names so you don't collide with other developers (get-names is a bad call name, since there is no context to it).

Then there's the front-end BrowserWindow side of the Electron equation. Do you just write simple JavaScript pages? Add a SPA like React? You still get to work in browser styles so do you use sass/scss, css-in-js, etc?

I haven't even begun to discuss other parts of the platform, including native builds for various platforms, incompatible libraries, etc.

Starters / Boilerplates: lots of choices, not all good

When you're just starting out, and you want a good example Electron project, there are tons of boilerplates to get you started. Each one has its own challenges and corner cases. I tried five different ones. See this list from the Electron site and Awesome Electron, with the goal of finding one that used vite and TypeScript.

Many of these boilerplates don't explain how you can configure them in more detail, so you end up searching lots of StackOverflow messages hoping to find that one magic config setting that will unlock your current challenge. I'm sure if I was more familiar with Electron itself I would have spent less time on this and would be able to fix more issues on my own.

In the end I ended up with electrovite-react, which worked pretty well. It is not on the two lists above, but is based on Electron Forge, a build system that seems to work the best out of those I've tried. It uses plugins and tries to keep the boilerplate code to a minimum.

Note, you'll see that I'm using pnpm for my package management tools. That seems to be the recommended tool for electrovite-react. Your mileage may vary with other package managers.

Hard things I solved

Here are a few problems I solved along the way – hopefully you'll find one of these helpful.

Issue # 1 – Installing a testing tool

I wanted to run Jest tests from TypeScript, so I hacked around with all sorts of options across the boilerplates. In Electrovite-react, vitest is supported, which looks like jest, but runs under the vite build tool.

Integrating vitest with electrovite-react was just:

Install the module

$ pnpm install -D vitest

Configure the script in package.json

    ...
    "scripts": {
        "test": "vitest"
        ...
    }
    ...

which enabled a simple pnpm run test.

Before realizing I could use vitest I was messing around with ts-jest, setting up a jest config file, etc. This simply isn't needed with the vite-based runtime in electrovite-react, setting up a jest config file, etc. This simply isn't needed with the vite-based runtime in electrovite-react.

Issue #2 – Incompatible Node libraries

Oh boy. This one is painful. My first example database API call was written for SQLite. I wrote a basic object persistence manager for my tasks, tested them using Jest tests (originally) in the plain-old Node runtime, and they worked.

The library I used was sqlite3. Maybe some of you are starting to itch? If so, the reason is that the sqlite3 project has troubles building in Electron, due to a failing node-pre-gyp step, one in which three strange dependencies are requested ('aws-sdk', 'nock' and an s3 API). This leads to a very long rabbit hole which made me first question my lifelong career goals, and then led me to look for an alternative library.

I found several libraries, one of which worked (after resolving the NEXT issue): Better SQLite3. The problem with this library was a very strange error:

Error: The module '/Users/.../projects/examples/electron/electrovite-react/node_modules/
.pnpm/better-sqlite3@8.4.0/node_modules/better-sqlite3/build/Release/better_sqlite3.node'
was compiled against a different Node.js
version using NODE_MODULE_VERSION 116.

This version of Node.js requires
NODE_MODULE_VERSION 115.

Please try re-compiling or re-installing
the module (for instance, using `npm 
rebuild` or `npm install`).

NODE_MODULE_VERSION? Really strange error. In the end, to fix it, I had to run pnpm rebuild, which rebuilt the downloaded library locally for my version of Node.js. This has happened multiple times, and I am not thrilled by it.

So, I did my pnpm rebuild and got a bunch of errors. Apparently unlike jest, vitest makes you import the methods of your test engine:

    import { describe, beforeAll, it, expect } from 'vitest';

Problem solved, I go green, all is good.

Issue #3: Imports -vs- CommonJS Requires

This, right here, is the oddest of problems with Electron. Because some imports have to be processed on the backend (nodejs) side of the farm, they act strange when using standard import statements.

Instead of importing the Node.js path API in ECMAScript import syntax, I had to import it with require:

    const path = require('path');

Even worse, some libraries really have strange issues being loaded in Electron, so for the uuid library I had to rename the method call to get it to work (yes, this is a known issue:)

    const uuidv4 = require('uuid').v4;

I would love to hear an Electron engineer explain why we have to go through so many gyrations to get this stuff to work.

My complications which make this case harder than hello world are:

  • I wanted a working test system out of the box (or early in the process)
  • I wanted to use TypeScript
  • I was trying to avoid Webpack
  • I wanted to use Vite

If I stayed with JavaScript, all of my imports would instead be require statements, which would simplify my plight.

In the end, I got the app to create and retrieve tasks via sqlite, so I consider the research task finished. If you want to take a look at what I have so far, visit this git repo.

Evaluating versus PWAs

Pulling back a bit, why would people choose the Electron platform? Here are the factors, and the ways you can achieve the same effect using PWAs and a bit of additional architecture:

Requirement Electron Approach PWA Approach Notes
Access to other networked services Node.js networking APIs from the Main process Using CORS or a networked proxy (like /api) Each can access most HTTP APIs, but electron can do TCP-based networking.
Desktop file storage Use Node.js file APIs Use the Origin Private File System (OPFS) The OPFS standard, while mostly supported from Chromium-based browsers, provides local file storage attached to a domain. You can load files into OPFS and then use them from the application. For full desktop networking Electron has the edge here.
Installability Electron’s updater process (OS X and Windows) or a package manager PWAs can be installed on various OSes (Mac OS X coming in late 2023) Electron has more control here, PWAs can have strange rules on how to update and when it occurs
Database access Sort of everything you want, except if it needs to be built natively or uses incompatible build tools Whatever your server provides, via a REST or GraphQL API I found the challenge of building a SQLite driver to be very frustrating as you see above in the article. I worry about getting into dead ends so test your APIs in Electron in your earliest research

Those are a few of the technical challenges and my current opinions of where the technologies fall. This is not a proscriptive post, more of a reflection of the friction created by attempting to use Electron to host a React application. For a smaller application, or one with more compliant APIs, you may feel Electron is worth it.

What do you think of using Electron vs a PWA?? Discuss in the comments below.