Home

fedialgo

Fedialgo is a node.js library offering a customizable algorithm for the federated social media platform Mastodon that can free you from the tyranny of Mastodon's reverse chronological order timeline. It's based on pkreissel's original implementation and ideas though it has more or less been completely rewritten and has many more features like integration of trending toots, filtering of the feed by hashtag/user/etc., improved load times, and more.

Installation

You can install this library from github with npm:

npm install --save github:michelcrypt4d4mus/fedialgo

Or with yarn:

yarn add https://github.com/michelcrypt4d4mus/fedialgo

Troubleshooting

If you're using the library outside a browser and get a Buffer is not a function error you may also need the buffer package to simulate the one provided by most browsers. It's required by the class-transformer library FediAlgo uses to serialize data to browser storage.

npm install --save buffer

And then put this in your main entrypoint (e.g. App.tsx or something like that):

import { Buffer } from 'buffer'; // Required for class-transformer to work
(window as any).Buffer = Buffer;

Usage

The demo app's Feed component demonstrates the latest and greatest way to use Fedialgo but here's a quick overview of how to get up and running. This code assumes you already have an access token for a user and a registered Mastodon "application" on the Mastodon server allowing read scope access. If you don't have one see the masto.js documentation for how to get one.

The FediAlgo demo app also contains a working example of how to execute the OAuth flow for a user to both register an app and get an authorized OAuth token for them:

import TheAlgorithm from "fedialgo"
import { createRestAPIClient, mastodon } from "masto";

const mastodonServer = "https://mastodon.social";
const accessToken = getTheUserAccessTokenSomehow();
const api = createRestAPIClient({accessToken: accessToken, url: mastodonServer});
const currentUser = await api.v1.accounts.verifyCredentials();

// Instantiate a TheAlgorithm object
const algorithm = await TheAlgorithm.create({
    api: api,
    user: currentUser,
    locale: "en-GB",     // optional (available in navigator.language in browser)
});

The setTimelineInApp Callback

You are encouraged to pass an optional setTimelineInApp() callback to TheAlgorithm.create() and allow FediAlgo to manage the state of the timeline in your app. The callback will be invoked whenever the timeline is updated. More specifically it will be invoked when any of these things happen:

  • A batch of toots is retrieved from the fediverse and integrated into the timeline
  • You make a call to algorithm.updateUserWeights() (see below)
  • You make a call to algorithm.updateFilters() (see below)
  • A background data fetch incrementally loads more of the user's historical data (as the algorithm is responsive to the user's history of favourites, retoots, etc. additional data can reorder the feed slightly)

An example involving storing the timeline in a React component's state:

import { useState } from React;
import TheAlgorithm, { Toot } from "fedialgo"

const [timeline, setTimeline] = useState<Toot[]>([]);

const algorithm = await TheAlgorithm.create({
    api: api,
    user: currentUser,
    setTimelineInApp: setTimeline
});

Functionality

Once you've instantiated a TheAlgorithm object there's three primary ways of interacting with it:

Triggering Construction Of The Timeline

import { Toot } from "fedialgo";

// Trigger the feed builder. FediAlgo will start doing stuff asynchronously. If you passed
// setTimelineInApp in the constructor all you need to do is monitor the state of whatever
// variable contains the timeline (in the React example above that would be 'timeline').
algorithm.triggerFeedUpdate();

// After first invocation check if loading is in progress before calling to avoid exceptions
if (!algorithm.isLoading) {
    algorithm.triggerFeedUpdate();
}

// algorithm.timeline returns the current weight-ordered/filtered array of Toot objects
// Note there won't be anything there until the timeilne is at least partially built!
let timeline: Toot[] = algorithm.timeline;
// If you wanted to wait until the feed was fully constructed, wait for the Promise:
algorithm.triggerFeedUpdate().then(() => timeline = algorithm.timeline);

// You can pull additional past timeline toots beyond the configured initial amount:
algorithm.triggerHomeTimelineBackFill();

Setting Weights For The Various Feed Scorers

import { ScoreName, Toot, Weights, WeightPresetLabel } from "fedialgo";

// Get and set score weightings (the things controlled by the sliders in the demo app)
const weights: Weights = await algorithm.getUserWeights();
weights[ScoreName.NUM_REPLIES] = 0.5;
let timelineFeed: Toot[] = await algorithm.updateUserWeights(weights);

// Additional properties (description, minimum value, etc) can be found at algorithm.weightInfo.
for (const key in algorithm.weightInfo) {
    console.log(`Weight '${key}' info: ${algorithm.weightInfo[key]}`);
}

// Or choose a preset weight configuration using the WeightPresetLabel enum
timelineFeed = await algorithm.updateUserWeightsToPreset(WeightPresetLabel.CHRONOLOGICAL);
// All the presets can be found in algorithm.weightPresets
Object.entries(algorithm.weightPresets).forEach(([presetName, weights]) => {
    console.log(`${presetName}:`, weights)
});

Filtering The Feed

import { BooleanFilterName, ScoreName, TagTootsCacheKey, Toot, Weights } from "fedialgo";

// Set a filter for only German language toots
algorithm.filters.booleanFilters[BooleanFilterName.LANGUAGE].updateOption("de", true);
const filteredFeed: Toot[] = algorithm.updateFilters(filters);

// Set a filter for only toots with at least 3 replies
algorithm.filters.numericFilters[ScoreName.NUM_REPLIES].value = 3;
const filteredFeed: Toot[] = algorithm.updateFilters(filters);

// There's also a lot of information available about the options that can be chosen for each filter
filters.booleanFilters[BooleanFilterName.HASHTAG].options.forEach((option) => {
    console.log(`Tooted the hashtag "${option.name} ${option[TagTootsCacheKey.PARTICIPATED_TAG_TOOTS]} times`);
});

Errors

Most minor API errors will be handled silently so that if, for example, fedialgo only gets half of a user's recent toots or whatever that doesn't cause any issues for the client app. If you want to see what, if any, errors were encountered during the scoring process you can check apiErrorMsgs to find them.

console.log(`API errors:`, algorithm.apiErrorMsgs);

Resetting Everything

You can wipe the browser storage and reset all variables if needed.

// Delete the user's timeline and historical data but preserve the user session
await algorithm.reset();
// Wipe EVERYTHING (will force a logout and complete reauthentication)
await algorithm.reset(true);

Documentation

There is JSDoc generated documentation of most of fedialgo's public API (classes and methods).

Toot Object API

The timeline is returned as an array of Toot objects which are a minimal extension of the mastodon API's Status object with a few more properties and some helper methods. Check the documentation or toot.ts for details. In particular note that you can mark a Toot object's numTimesShown property, at which point the AlreadyShownScorer will return non-zero values for that Toot.

timelineFeed[0].numTimesShown = 1;

Other Data Available From TheAlgorithm

FediAlgo exports a number of types and enums; check the documentation or look at the bottom of index.ts for details on what is available. TheAlgorithm objects provide a bunch of data besides the timeline should you choose to access it.

Fediverse Trending Data

Current "trending" fediverse data can be accessed at algorithm.trendingData. See types.ts or the object API documentation for info on the data type.

// Trending links
algorithm.trendingData.links.foreach((link) => console.log(`Link '${link.uri}' tooted by ${link.numAccounts} accounts`));

// Trending tags
algorithm.trendingData.tags.foreach((tag) => console.log(`Tag '${tag.name}' tooted by ${tag.numAccounts} accounts`));

// Trending toots
algorithm.trendingData.toots.foreach((toot) => console.log(`Trending toot w/rank ${toot.trendingRank}: '${toot.describe()}'`));

// Popular servers
console.log(`Servers used to determine trending data:`, algorithm.mastodonServers);

User Data

The user's followed accounts, muted accounts, followed tags, and a few other bits and bobs used to compute the scoring of the timeline can be accessed at algorithm.userData. See the documentation or user_data.ts for info on the data type (and be aware this is probably the least stable / most subject to change part of the fedialgo API).

There's also a unified method to collect a bunch of information (fedialgo configuration, server configuration, user data, filter settings, etc.) with a single call:

const currentState = await algorithm.getCurrentState();

Package Configuration

Package configuration options can be found in src/config.ts. These can't currently be changed via the API though feel free to experiment with your local copy of the repo or ping me if you have a use case for updating some of the configuration variables.

Contributing

Developer Setup

If necessary install the dev dependencies with npm install --include=dev.

Debugging

If you set the environment variable FEDIALGO_DEBUG=true a lot more debugging info will be printed to the browser console. See .env.example for other environment variables you can play with.

Adding New Scorers

To add a new metric for scoring toots you must:

  1. Add an entry to the ScoreName enum
  2. Create a new subclass of Scorer
  3. Add a default weight for your scorer to DEFAULT_WEIGHTS
  4. Instantiate an instance of your new Scorer in the appropriate array in TheAlgorithm (featureScorers if it's a self contained score that requires only the information in a single toot, feedScorers if it's a scorer that requires the entire set of timeline toots to score a toot)

Deploying Changes

For changes to propagate you must run npm run build to generate changes to files in dist/ and then check those files into git.

Developing Against a Local Project

Clone this repo and cd into it. Then run:

npm install
npm link

Then cd to the node.js project that is going to host this package and run this:

npm link fedialgo

Running Test Suite

(The test suite is kind of useless unfortnately.) npm run test

Miscellaneous

Use // @ts-ignore if you run into Typescript warnings (because your project might also use masto)

npm run build

in fedialgo directory after changes and they will automatically be detected.

There's a pre-commit git hook that runs npm run build but unfortunately it doesn't seem to actually run before the commit :(

Resources

TODO

  1. Make use of the fact that you can see who favourited a post: https://docs.joinmastodon.org/methods/statuses/#favourited_by
  2. Why does this happen, where we get two copies of bridged bsky accounts?
"id": "113482690234776099",
"acct": "youranoncentral.bsky.social@bsky.brid.gy",
"url" (1): "https://bsky.brid.gy/r/https://bsky.app/profile/youranoncentral.bsky.social",
"url" (1): "https://bsky.brid.gy/ap/did:plc:mxc7liuon6iq5gzapmmwkq22",

What's slow:

According to Chrome profiler it's the retrieval of the user's favourites that is the biggest bottleneck at initial load time. Took ~10 seconds, getting user's old toots took ~5s.

filters/boolean_filter.js

Feed filtering information related to a single criterion on which toots can be filtered inclusively or exclusively based on an array of strings (e.g. language, hashtag, type of toot).