First 20 downloads (and a lesson in moderation)

First user, first problem

When I opened Oceano di Versi to the public, the very first piece of content posted by an outside user was a vulgar, Nazi-themed “poem”. It did not go unnoticed — I saw it in my feed minutes later.

Two reactions at once:

1. Annoyance — the app is called Oceano di Versi, that does not belong there.
2. Technical relief — it was the first time someone was really using the system, so I finally had a real case to stress-test moderation against.

What I was getting wrong

The checks existed, but they were all client-side, before upload:

• Images: I used Apple’s Sensitive Content Analysis framework directly inside the app. When a user picked an explicit profile picture or cover (nudity), the app refused to upload it before the request left the device. It worked, but only covered explicit nudity — no violence, hate symbols, or other disturbing-but-not-sexual content.
• Text: a plain blacklist of banned words. Bypassable in two seconds by writing n@zi, n4zi, n a z i with spaces, emojis in place of letters, and so on. The incriminated “poem” slipped through exactly that way.

In both cases the check was bypassable by any client.

The fix: a server-side pipeline with Gemini

That same night I moved everything to the server, on the pipeline described in detail in the Oceano di Versi docs. In short: every poem first lands in the staging collection pendingPoems, a Gen 2 Cloud Function calls Gemini 2.5 Flash to analyze both text and images, and only if the verdict is clean is the content promoted to poems.

exports.moderatePendingPoem = onDocumentCreated(
  { document: "pendingPoems/{poemId}", secrets: [GEMINI_API_KEY] },
  async (event) => {
    const verdict = await runGeminiModeration(event.data.data());

    if (verdict === "block") return quarantinePoem(event);
    return promotePoemFromPending(event);
  }
);

The same pattern applies to comments and profile pictures. The verdict is fail-open (if Gemini errors out the content passes but is flagged for human review), so an API outage does not freeze the app — but the check can no longer be bypassed by the client and recognizes far more than nudity or blacklisted words.

And TikTok?

Between one Cloud Function and the next I filmed my screen on the fly and posted it on TikTok with an honest caption:

When someone finally uses your app but posts vulgar content, and you spend the whole night improving the moderation system.

A few days later, the numbers:

Those ~20 were the first real organic downloads of Oceano di Versi.

What I’m taking away

• Client-side isn’t enough. A filter before upload is useful as a first UX barrier (saves bandwidth, gives instant feedback), but it’s never a moderation policy: it’s bypassable and only covers what the local framework recognizes.
• Word blacklists are theater. Any moderately motivated user bypasses them with symbols, numbers, or spaces. You need a semantic classifier.
• The first real user is the best free pen-tester. No test I could have written would have surfaced this case so quickly.
• Honest content works. The video performed because it captured a real development moment, not a polished teaser.