Skip to content

Post to Telegram as rich messages with --rich - #356

Open
biodranik wants to merge 1 commit into
ab/tg-fixfrom
ab/telegram-markdown-api
Open

biodranik wants to merge 1 commit into
ab/tg-fixfrom
ab/telegram-markdown-api

Conversation

@biodranik

Copy link
Copy Markdown
Member

Bot API 10.1 added sendRichMessage, which takes GitHub-Flavored Markdown in one field and renders it natively. For a release post that means headings that are headings, lists that are lists, screenshots inside the post rather than in a trailing album, and a 32768-character limit in place of 4096 — the August notes go out as one message per channel instead of the two or three MarkdownV2 splits them into. Nothing is escaped, so the new path bypasses the convert/escape/split machinery rather than reusing it, and there is no unformatted fallback to reason about: a post either lands whole or does not land at all.

Three things do not survive the site markdown as they are.


becomes a Markdown hard break, not a newline: Rich Markdown joins consecutive lines exactly as CommonMark does, so a bare newline would turn the signature into one run-on line. Two trailing spaces degrade to a single space if unsupported, where the tag passed through would degrade to a visible "
" — the defect this replaces. A
inside code is left alone, as in the MarkdownV2 path.

Media "can be specified only as a separate block", so each file is appended as its own block and uploaded by that id in the same multipart request.

The title becomes a heading instead of the bold line MarkdownV2 has to fake.

Rich messages have neither the 10-file album cap nor the homogeneous-audio rule of sendMediaGroup, so validate_media_set() does not gate them; validate_rich_media() does. Length is checked as both code points and UTF-8 bytes, because "32768 UTF-8 characters" reads as the former but could mean the latter and the docs settle it nowhere — a false rejection is loud and early, while a wrong-unit pass would abort a run that has already published. Every post is measured before the first send for the same reason: translations run 15-25% longer than the English source, and discovering that at channel nine leaves the post live in eight languages only.

The "Resume with" hint now carries --rich, which otherwise would republish the remaining channels in the other format, leaving one post live in two shapes across the channel set.

is_rtl lays the Arabic and Persian channels out right-to-left, from the same language list base.html uses; a test asserts the two still agree.

video_metadata() is lifted out of send_media_group() so both send paths share the ffprobe probe and poster-frame extraction. That path is otherwise untouched, and its dry-run output is unchanged.

--rich is opt-in: no request has reached the live API yet.

Bot API 10.1 added sendRichMessage, which takes GitHub-Flavored Markdown in
one field and renders it natively. For a release post that means headings
that are headings, lists that are lists, screenshots inside the post rather
than in a trailing album, and a 32768-character limit in place of 4096 —
the August notes go out as one message per channel instead of the two or
three MarkdownV2 splits them into. Nothing is escaped, so the new path
bypasses the convert/escape/split machinery rather than reusing it, and
there is no unformatted fallback to reason about: a post either lands whole
or does not land at all.

Three things do not survive the site markdown as they are.

<br/> becomes a Markdown hard break, not a newline: Rich Markdown joins
consecutive lines exactly as CommonMark does, so a bare newline would turn
the signature into one run-on line. Two trailing spaces degrade to a single
space if unsupported, where the tag passed through would degrade to a
visible "<br/>" — the defect this replaces. A <br/> inside code is left
alone, as in the MarkdownV2 path.

Media "can be specified only as a separate block", so each file is appended
as its own ![](tg://photo?id=…) block and uploaded by that id in the same
multipart request.

The title becomes a heading instead of the bold line MarkdownV2 has to fake.

Rich messages have neither the 10-file album cap nor the homogeneous-audio
rule of sendMediaGroup, so validate_media_set() does not gate them;
validate_rich_media() does. Length is checked as both code points and UTF-8
bytes, because "32768 UTF-8 characters" reads as the former but could mean
the latter and the docs settle it nowhere — a false rejection is loud and
early, while a wrong-unit pass would abort a run that has already published.
Every post is measured before the first send for the same reason:
translations run 15-25% longer than the English source, and discovering that
at channel nine leaves the post live in eight languages only.

The "Resume with" hint now carries --rich, which otherwise would republish
the remaining channels in the other format, leaving one post live in two
shapes across the channel set.

is_rtl lays the Arabic and Persian channels out right-to-left, from the same
language list base.html uses; a test asserts the two still agree.

video_metadata() is lifted out of send_media_group() so both send paths
share the ffprobe probe and poster-frame extraction. That path is otherwise
untouched, and its dry-run output is unchanged.

--rich is opt-in: no request has reached the live API yet.
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying organicmaps with  Cloudflare Pages  Cloudflare Pages

Latest commit: 64d3012
Status: ✅  Deploy successful!
Preview URL: https://c82605a8.organicmaps.pages.dev
Branch Preview URL: https://ab-telegram-markdown-api.organicmaps.pages.dev

View logs

@biodranik biodranik left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the three changed files in full; python3 -m unittest discover -s tmp -p test_tooling.py is green (33 tests) on 64d3012. The <br/>→hard-break rule, the code-span protection, the up-front all-channels length check and the RTL_LANGS/base.html agreement test all hold up. Five findings inline; one is a real regression in the single-post script.

Cross-script drift. The one confirmed bug (inline at telegram_post.py:1463) is that telegram_post_all.py learned to choose its media validator by mode and telegram_post.py did not. The same shape appears twice more: telegram_post_all hard-errors on --rich --allow-plain-fallback, while telegram_post.py silently ignores --strict under --rich; and the title-heading decision ("# " if rich else "**") plus the strip-Tera / angle-bracket / blank-line cleanup are now duplicated verbatim in prepare_text() and main().

Alternative worth considering (larger than this PR, so a follow-up): export prepare_text() from telegram_post.py and let telegram_post_all import it, then have both mains call one send_post(token, chat_id, text, media, rich, ...) that owns the validator choice, the flag-compatibility check and the dispatch. telegram_post.py becomes the one-channel case of the same call. Pro: the three drifts above become structurally impossible — the format decision exists once. Con: touches the MarkdownV2 path, which this PR deliberately leaves alone; only worth it once --rich has been used against the live API.

No translated or user-facing strings in this diff, so nothing for the proofreading pass — the added text is CLI output and reads well.


Generated by Claude Code

Comment thread tools/telegram_post.py
print(f"Media files: {len(args.media)}")
print()

if args.rich:

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A --rich post is still gated by the album validator here. validate_media_set(args.media) (line 1379) runs unconditionally before this block, so --rich inherits the two sendMediaGroup rules that rich messages do not have:

$ TELEGRAM_BOT_TOKEN=x python3 tools/telegram_post.py --rich -g @x -t post.md -m img0.jpg … img10.jpg
Error: Telegram media albums support at most 10 files

Same for photo+audio → "Telegram audio albums must contain audio files only". validate_rich_media() allows both (50 files, mixed kinds), and telegram_post_all.py:269 already picks the validator by mode. Line 1379 needs the same:

media_problem = (
    validate_rich_media(args.media) if args.rich else validate_media_set(args.media)
)

Generated by Claude Code

Comment thread tools/telegram_post.py
chat_id,
text,
args.media,
is_rtl=is_rtl_lang((lang.group("lang") if lang else None) or "en"),

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is telegram_post_all.group_lang() re-inlined verbatim. That helper only needs ZOLA_FILENAME_RE, which lives in this module — moving it here leaves one implementation, gives this call site a name instead of a nested conditional, and lets telegram_post_all drop its ZOLA_FILENAME_RE import (it uses it nowhere else):

        is_rtl=is_rtl_lang(group_lang(args.text.name)),

Generated by Claude Code

Comment thread tools/telegram_post.py

blocks = []
for index, path in enumerate(media_paths):
scheme = RICH_MEDIA_SCHEMES.get(classify_media(path))

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two simplifications here:

  1. RICH_MEDIA_SCHEMES is an identity map over exactly the three values classify_media() can return, so the lookup and the constant can both go:
        kind = classify_media(path)
        if kind is None:
            ...
        blocks.append(f"[](tg://{kind}?id={rich_media_id(path, index)})")
  1. The skip branch can only ever fire on a post that is about to be rejected: send_rich_message() calls validate_rich_message() immediately after building, and telegram_post_all runs validate_rich_media() before either. So an unsupported file prints Skipping unsupported file: x and then aborts — two voices for one failure. Hoisting validate_rich_media(media_paths) above the build_rich_markdown() call in send_rich_message() (line ~1280) makes the fail-fast single and leaves the skip as pure defence.

Generated by Claude Code

Comment thread tools/telegram_post.py
# earlier channels, where a false rejection is merely loud and early.
characters = len(markdown)
utf8_bytes = len(markdown.encode("utf-8"))
if max(characters, utf8_bytes) > RICH_TEXT_LIMIT:

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

len(s.encode("utf-8")) >= len(s) holds for every string, so max(characters, utf8_bytes) is always utf8_bytes — only the byte reading is ever enforced. That is the right (stricter) choice, but the comment above reads as if two independent checks run, and the max() invites a reader to look for the case where the character count wins.

    # The byte count is the stricter of the two readings of "32768 UTF-8
    # characters", so enforcing it satisfies either. …
    if utf8_bytes > RICH_TEXT_LIMIT:

Both numbers stay useful in the message. test_length_is_refused_under_either_reading_of_the_limit passes unchanged.


Generated by Claude Code

Comment thread tools/telegram_post.py
raw_html = find_raw_html(text, rich=args.rich)
if raw_html:
print(
f"\nWarning: {len(raw_html)} HTML tag(s) Telegram will show "

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In rich mode this wording is now inverted. "Shown literally" is the MarkdownV2 behaviour (escaped, visible — the <br/> defect); what survives the RICH_HTML_TAGS filter is precisely the set that RICH_HTML_TAGS' own comment says Telegram "drops silently". An editor reading this warning will go looking for a visible tag that never appears. Same string in telegram_post_all.py:180 — suggest picking the verb from the mode ("Telegram will drop" / "Telegram will show literally").

Related: RICH_HTML_TAGS is documented as the tags the Rich HTML style parses, but it is applied to a markdown payload. That the same whitelist governs Rich Markdown's raw-HTML passthrough is the assumption that silences these warnings — worth stating in the constant's comment, since <br> is treated as not in the set on exactly the same reasoning.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant