What Is NLLB?

NLLB: No Language Left Behind is a multilingual translation model. It’s trained on data using data mining techniques tailored for low-resource languages and supports over 200 languages.

My mom is Yorùbá, specifically from an Ìjẹ̀bú background. Growing up with that background, one thing I caught on to; is that “Yorùbá” isn’t really one language. There’s a version everyone treats as the default (which is mostly drawn from Ọ̀yọ́), and then there’s what people actually speak at home, which can sound pretty different depending on where your family is from. Words, pronunciation, sometimes whole phrasing changes.

This isn’t just a personal observation, it’s a documented gap. Yorùbá has something like 47 million speakers across Nigeria, Benin, and Togo, and standard Yorùbá has gotten real attention from NLP researchers over the years, but until pretty recently, basically none of the non-standard dialects had any dedicated resources at all.

Yoruba has the 37th highest speakers in the world (https://en.wikipedia.org/wiki/List_of_languages_by_total_number_of_speakers#cite_note-9)

That’s the general pattern with low-resource language work, actually: the “low-resource” conversation almost always means most of the data about the ethnology of language is missing.

I came across a Global Voices interview with Aremu Anuoluwapo, a computational linguist who, along with Oreva Ahia (a PhD student at the University of Washington), built YorùLect specifically to address this.

The story behind it is a lot more grounded than “we scraped some data.” Anuoluwapo traces the idea back to an undergrad dialectology course, where he first noticed that even a basic word like “stool” differs between his hometown dialect and standard Yorùbá. He and Ahia later worked out the project’s framework in person, and he then traveled to the specific communities where each target dialect is spoken to actually collect the data, deliberately choosing dialects from different regional branches of the language (southwest, southeast, and beyond) rather than just picking whatever was easiest to reach.

A few things from that interview reframed how I’m thinking about this project. First, they’d already run the comparison I was planning to run myself:

They tested existing ASR and MT systems on these dialects before any fine-tuning and found the pure performance was bad, with fine-tuning helping but not closing the gap evenly.

Interestingly, they found Ifè came out closest to standard Yorùbá, and Ìlàje came out most different, partly because Ìlàje uses letters that don’t even exist in the standard alphabet.

Second, data collection itself was the hard part:

Some dialects barely have a population of fluent writers, so their approach was to record speech first and get native speakers to transcribe it afterward, rather than assuming written data would just be sitting there to find.

“An Egungun Priest”

That last point is worth sitting with. The actual hard, unglamorous part of low-resource NLP isn’t the model.

It’s the fact that someone had to have this idea, travel to specific communities, and do the work of collecting and transcribing dialect data with real speakers.

I’m not doing any of that. I’m standing entirely on top of it. Which is fine, that’s what public research datasets are for, but I want to be upfront that the genuinely hard part of this space already happened before I show up. What I’m doing is a much smaller, more contained question layered on top of their work.

My Experiment

So the question I actually want to poke at with my own small slice of this: if I take a model that already knows standard Yorùbá, can I nudge it to specifically understand Ìjẹ̀bú? Given their own finding that dialects vary a lot in how much fine-tuning helps, does Ìjẹ̀bú respond well to this, or is the gap stubborn?

what I actually think is going to happen

Here’s my prediction, written down now so I can’t quietly move the goalposts later:

• If I take NLLB (Meta’s translation model, already knows 200+ languages including standard Yorùbá) and just point it at Ìjẹ̀bú with zero extra training, it’ll do noticeably worse than it does on standard Yorùbá. Not useless, but visibly worse.
• If I then fine-tune it on a few hundred Ìjẹ̀bú parallel sentences from YorùLect, that gap should shrink. Not disappear — a few hundred examples is not a lot — but shrink in a way that’s measurable.

If neither of those things happen — if zero-shot is already fine, or fine-tuning does nothing — that’s also a real result and I’ll say so.

the actual plan

• Model: NLLB-200-distilled-600M. Not the bigger 1.3B/3.3B versions — this one’s small enough to actually train on what I’ve got access to, and it’s what basically every low-resource fine-tuning writeup I’ve read uses for exactly this reason.
• Data: YorùLect’s Ìjẹ̀bú split. Roughly 800 sentences to train on, 200 to check against while training, 500 held back to actually measure performance at the end.
• Metric: BLEU and chrF2, since that’s the standard pair for this kind of MT eval and it’s what most of the papers I’ve been reading report, so I can actually compare.
• Steps: get a standard-Yorùbá baseline running first (so I know my pipeline itself isn’t broken), then run zero-shot on Ìjẹ̀bú, then fine-tune, then compare.

I’ve never actually fine-tuned a translation model before, so there’s a real chance week one is just me fighting library versions and tokenizer weirdness before I see a single real number. If so, that’s going in the next post too.

More soon.

So I finally put on my engineering hat and started working on this project. The reason I wanted a blog I could publish from Telegram was simple: I wanted to be able to write and publish on my very long commute to work.

So what did I actually need?

A blog that was live, and a way for Telegram to talk to it.

The problem, structured

Before touching any code, the requirement broke down into 3 main concerns:

Constraint 1 (the trigger has to be a chat message)

Since the way I wanted the system to be designed was mostly conversational, since I would be writing this on my commute, on a phone.

Constraint 2 (the site should be easy to manage)

I needed a site with no server to run updates on, no database to back up, and nothing that cost money.

Constraint 3 (the conversational end and blog post needed to communicate)

I needed to come up with a simple protocol for communication between the 2 ends of my system.

After taking some time to think and consulting with over a thousand experts (prompting Claude’s 85,000 MOE), I finally arrived at a solution for each of my constraints.

My solution

For constraint 1 (the trigger):

I used a Telegram bot. Telegram makes it very easy to consult with the BotFather and get a bot created.

Telegram’s Bot API supports webhooks natively, so Telegram can push an event to a URL the moment a message arrives, which was perfect. This was exactly what I needed to create a blog conversationally from Telegram.

For constraint 2 (easy site):

The solution for the easy site was sitting right in front of me. I used GitHub Pages running Jekyll with static output. This provided me with free hosting, and made the source of truth just files in a Git repo. That meant “publishing a post” could be reduced to “committing a file”. The process of handling commits has an existing, well-documented API.

For constraint 3 (the connection):

Duh duh duh.

This is where I feel the actual skill sits. The Sprezzatura, the It factor, whatever you may call it.

Point being, this is where the actual engineering decision lived. A webhook needs something listening on a public URL, and I didn’t want to run or pay for a server that sits idle 99% of the time waiting for a commute.

So I needed serverless computers. Cloudflare Workers fit exactly: near-zero cost at low volume, no infrastructure to maintain, and a request/response model that matches what a webhook actually needs.

That decision immediately created a new problem, though, which turned out to be the most interesting part of the build.

State on stateless infrastructure

Serverless compute is stateless by design, so every request gets a fresh context with no memory of the last one.

But for what I wanted, I wanted to create the blog post piece by piece, conversationally. Every part of the blog post isn’t a single request. It’s a conversation: send a title, then an image, then several paragraphs, then confirm.

So what does this mean for my stateless worker? The system has to remember which step you’re on between messages that might arrive minutes apart, and the compute layer handling those messages has no built-in way to remember anything.

Hot Fix!

The fix, once I actually understood it: the Worker has amnesia after every single message. So instead of trying to remember anything itself, it writes a note to itself in a shared notebook (Cloudflare KV) after every message, and reads that note back before deciding what to do next.

The conversation isn’t held in the Worker’s memory at all.

So where’s the conversation actually living?

In a JSON object sitting in Cloudflare KV, labeled with my Telegram chat ID. That object is just four things: what step I’m on, my title so far, a reference to my header image, and a list of everything I’ve written or sent. Every message, the Worker’s entire job is: read that object, update it based on what I just sent, save it back.

After fixing that, we have our publishing pipeline.

Publishing: what actually happens

When you confirm a post, the Worker does a sequence of things against GitHub’s Contents API: fetch each image’s bytes from Telegram, commit each one, assemble the markdown and front matter, commit that.

So a single post I write might actually be four or five separate saves happening behind the scenes: one per image, plus one for the post itself.

Every part of my blog is decoupled when my Worker is building it.

Let that stick with you…

There’s no transaction wrapping the whole sequence. If the process fails partway through, you can end up with an orphaned image committed with no post referencing it, and no clean way to resume from where it stopped.

This is a limitation of my system right now, and I think the fix would be a more durable job record written before the calls to the GitHub API start. But that’s next weekend’s problem.

Closing

It works now. I write on the bus and it’s live by the time I get to my desk at work. There are still some rough edges, but I’m probably going to keep working and refactoring this project

My blog haha

two hundred already logged

Just commit, commit.

I’m only 23 and I feel like time is moving way faster than I can keep up with. Both socially, career-wise and mentally. As I grow I older and my responsibilities start to increase, I find myself realizing how little time we have on this planet. People talk about how your 20’s fly, but rarely talk about how 40 years can fly in an instant.

This post isn’t really a doom one and doesn’t really aim to do much, but I want you reading to understand that there is no time. That book you want to write, that project you want to finish, is going nowhere. Do it, or don’t = nothing is going to change. If you truly want It in you heart, you are going to work ard for it, and it will suck.

(https://www.youtube.com/watch?v=H9h-3KvBRww&t=362s)