My leaflets now show on my personal blog

My personal blog https://pfrazee.com now renders my Leaflet posts . Here's a couple of notes on how I did it.

If you're not in the know, Leaflet is an EZ blogging platform. I call it "EZ blogging" because it's regular blogging but with a kind of casual F-it energy. Making a blogpost is a Business Decision, while making a Leaflet? that's just EZ blogging.

Leaflet is a part of the Atmosphere, so all of my Leaflets are available as public data in my repo .

That meant it wasn't too much hassle to pull my leaflets into my personal blog. In the spirit of straight shooting: I'd give the experience a B-. I'm on the Bluesky team and it still took me two casual evenings to pull this off.

The good

The new still-in-beta typescript API ( atproto/lex ) made a lot of stuff pretty darn easy.

Calling lex install pub.leaflet.document to fetch the Leaflet schemas (Lexicons) and then lex build to generate the handling code felt great.

Writing the leaflet renderer was pretty fun.

It works.

The meh

My blog is on nextjs and does a static build, and I don't have an easy way to auto-trigger rebuilds when I post a leaflet.

(Please don't take this as criticism, Matthieu) the new atproto/lex API is a little daunting at times. I'm pretty sure this will be solved with documentation and stabilization.

If I didn't know where my RichText facet rendering code was tucked away, I would've been totally hosed.

You can find my blog's sourcecode here but fair warning, the code is real slapped together. I use timlrx/tailwind-nextjs-starter-blog which is in many ways wonderful, but contentlayer really fought me and I found myself having to hack around it a lot.

The code

I'm kicking the tires of the still-in-beta atproto/lex API, so a fair amount of this will change in the future. I'm sharing this as a reference.

It's also "just my blog" code, so the rigor here is... low.

Installing the lexicon schemas

The way I built my schemas is a little unusual, writing to ./util because the codebase layout is bad and then including the ".ts" extension so I can call the code via node --experimental-strip-types

# install the tool
$ npm install -g @atproto/lex

# fetch all the schemas I need
$ lex install pub.leaflet.document

# build the code (weird usage, don't do it this way)
$ lex build --out ./util --import-ext ".ts"

Fetching the leaflets

This bit resolves my handle (@pfrazee.com) then resolves that to my DID document, which points me to my Personal Data Server (PDS).

Then I fetch all my leaflet documents, validating them as I do (automatically).

Then I fetch all the image blobs referenced in the leaflets.

import {fileURLToPath} from 'node:url'
import * as path from 'node:path'
import * as fsp from 'node:fs/promises'
import { IdResolver } from '@atproto/identity'
import { Client } from '@atproto/lex'
import type { ListRecord, DidString, LexMap } from '@atproto/lex'
import { lexStringify } from '@atproto/lex-json'
import * as leaflet from '../util/pub/leaflet.ts'
import { toExt, enumBlobRefs } from '../util/helpers.ts'

const __dirname = path.dirname(fileURLToPath(import.meta.url))

;(async()=>{
console.log('=================')
console.log('Fetching leaflets')
console.log('=================')

// Resolve DID and PDS
const resolver = new IdResolver()
const did = (await resolver.handle.resolve('pfrazee.com')) as DidString
console.log('Resolved pfrazee.com to', did)
const pds = (await resolver.did.resolveAtprotoData(did!)).pds
console.log('Resolved pds to', pds)

const client = new Client(pds)

// Fetch all leaflets
let leaflets: ListRecord<typeof leaflet.document.$defs.main>[] = []
let invalids: LexMap[] = []
let cursor: string | undefined = undefined
let i = 0
do {
const result = await client.list(leaflet.document, {
repo: did!,
limit: 50,
reverse: true,
cursor
})
cursor = result.cursor
leaflets = leaflets.concat(result.records)
invalids = invalids.concat(result.invalid)
} while(cursor && (++i < 100))
console.log('Fetched', leaflets.length, 'leaflets.', invalids.length, 'failed validation.')

// Write the leaflets to `/data/leaflets/*.json`
const leafletDataDir = path.join(__dirname, '..', 'data', 'leaflets')
if ((await fsp.stat(leafletDataDir).catch(e => undefined))) {
await fsp.rm(leafletDataDir, {recursive: true})
}
await fsp.mkdir(leafletDataDir).catch(e => undefined)
for (const leaflet of leaflets) {
const rkey = leaflet.uri.split('/').pop()
await fsp.writeFile(
path.join(leafletDataDir, rkey + '.json'),
lexStringify(leaflet.value),
'utf-8'
)
}

// Enumerate and fetch all missing images
const imageDir = path.join(__dirname, '..', 'public', 'static', 'images', 'leaflets')
await fsp.mkdir(imageDir).catch(e => undefined)
for (const leaflet of leaflets) {
const rkey = leaflet.uri.split('/').pop()!

for (const blobRef of enumBlobRefs(leaflet.value)) {
// images only
if (!blobRef.mimeType.startsWith('image/')) {
continue
}

// construct a filename
const ext = toExt(blobRef.mimeType)
if (!ext) {
console.warn('Unsupported mimetype', blobRef)
continue
}
const filename = `${blobRef.ref.toString()}.${ext}`.replaceAll('/', '' /* juuust in case */)
const imagePath = path.join(imageDir, filename)

// make sure the dir exists
await fsp.mkdir(imageDir).catch(e => undefined)

// check if the image exists
if ((await fsp.stat(imagePath).catch(e => undefined))) {
continue
}

// doesn't exist, fetch it
console.log('Fetching', filename, 'for leaflet', rkey, '...')
const blobRes = await client.getBlob(did, blobRef.ref.toString())

// sanity
if (blobRes.payload.encoding !== blobRef.mimeType) {
console.error('Response mimetype does not match blobref mimetype, skipping', blobRes.payload.encoding, blobRef)
continue
}

// write to disk
await fsp.writeFile(imagePath, blobRes.payload.body)
console.log('Fetched', blobRes.payload.body.byteLength, 'bytes')
}
}
})()

export {}

Here's how I enumerated those blob refs:

import type { LexMap } from '@atproto/lex'
import { isBlobRef, isLexMap } from '@atproto/lex-data'
import type { BlobRef } from '@atproto/lex-data'

export function toExt(mimeType: string): string {
if (mimeType === 'image/png') return 'png'
if (mimeType === 'image/jpeg') return 'jpg'
if (mimeType === 'image/webp') return 'webp'
return ''
}

export function* enumBlobRefs(map: LexMap): Generator<BlobRef> {
for (let v of Object.values(map)) {
if (isBlobRef(v)) {
yield v
} else if(isLexMap(v)) {
yield* enumBlobRefs(v)
} else if (Array.isArray(v)) {
for (let v2 of v) {
if(isLexMap(v2)) {
yield* enumBlobRefs(v2)
}
}
}
}
}

Rendering the leaflets

I'll refer you to the source file for this one since it's a little longer, but here's a preview.

There are almost certainly more elegant ways to write this code.

import type { linearDocument } from "@/util/pub/leaflet/pages"
import * as blocks from '@/util/pub/leaflet/blocks'
import * as facets from '@/util/pub/leaflet/richtext/facet'
import Image from '@/components/Image'
import { RichText, RichTextSegment } from "app/helpers"
import { jsonToLex, JsonValue } from '@atproto/lex-json'
import { toExt } from '@/util/helpers.ts'
import { BlueskyPostEmbed } from "./post-embed/BlueskyPostEmbed"

export function LeafletRenderer({pages}: {pages: linearDocument.Main[]}) {
// HACK due to limitations in the contentbuilder framework, we have to do this cast here
pages = jsonToLex(pages as JsonValue)! as linearDocument.Main[]

const blocks = pages[0].blocks
return (

{blocks.map((block, i) => (
<LeafletBlock key={`block-${i}`} block={block} />
))}

)
}

function LeafletBlock({block}: {block: linearDocument.Block}) {
const innerBlock = block.block
if (blocks.header.main.$matches(innerBlock)) {
return <LeafletBlockHeader block={innerBlock} />
}
if (blocks.text.main.$matches(innerBlock)) {
return <LeafletBlockText block={innerBlock} />
}
if (blocks.blockquote.main.$matches(innerBlock)) {
return <LeafletBlockQuote block={innerBlock} />
}
if (blocks.horizontalRule.main.$matches(innerBlock)) {
return
}
if (blocks.unorderedList.main.$matches(innerBlock)) {
return <LeafletBlockUl block={innerBlock} />
}
if (blocks.website.main.$matches(innerBlock)) {
return <LeafletBlockWebsite block={innerBlock} />
}
if (blocks.image.main.$matches(innerBlock)) {
return <LeafletBlockImage block={innerBlock} />
}
if (blocks.bskyPost.main.$matches(innerBlock)) {
return <LeafletBlockBskyPost block={innerBlock} />
}
console.warn('Warning! Unhandled block in leaflet', innerBlock.$type, innerBlock)
return TODO: {innerBlock.$type}
}

function LeafletBlockHeader({block}: {block: blocks.header.Main}) {
const level = (block.level || 1) + 1 // add 1 because the title is the h1
if (level === 2) {
return <RenderRichText rt={new RichText(block.plaintext, block.facets)} />
}
if (level === 3) {
return <RenderRichText rt={new RichText(block.plaintext, block.facets)} />
}
// ...
return <RenderRichText rt={new RichText(block.plaintext, block.facets)} />
}

function LeafletBlockText({block}: {block: blocks.text.Main}) {
return <RenderRichText rt={new RichText(block.plaintext, block.facets)} />
}

function LeafletBlockQuote({block}: {block: blocks.blockquote.Main}) {
return <RenderRichText rt={new RichText(block.plaintext, block.facets)} />
}

// etc...

Again, full source for that renderer is here .