code.ivysaur.me/nmdc-telegramfrontend
1
0
Fork 0
Code Issues 26 Pull Requests Releases 2 Wiki Activity

missing messages when DC ships bad utf8 #20

New Issue
Open
opened 2018-11-18 07:09:03 +00:00 by mappu04 · 3 comments
mappu04 commented 2018-11-18 07:09:03 +00:00
Owner
Copy Link

We're dealing with raw bytes almost everywhere. It's assumed these are UTF-8 strings.

When DC sends us a message that's not valid UTF-8 (e.g. today, a pasted em-dash in Windows-1252 encoding(?)) TG silently(?) drops the message.

We should either

  • better handle error messages returned from the message-submission API, or
  • filter out invalid characters before submitting them to TG
We're dealing with raw bytes almost everywhere. It's assumed these are UTF-8 strings. When DC sends us a message that's not valid UTF-8 (e.g. today, a pasted em-dash in Windows-1252 encoding(?)) TG silently(?) drops the message. We should either - better handle error messages returned from the message-submission API, or - filter out invalid characters before submitting them to TG
mappu04 added the
bug
 label 2018-11-18 07:09:39 +00:00
mappu04 commented 2019-04-01 05:43:47 +00:00
Author
Owner
Copy Link

Attempt to parse all incoming messages as UTF-8. If they fail, replace invalid characters with U+FFFD REPLACEMENT CHARACTER

Attempt to parse all incoming messages as UTF-8. If they fail, replace invalid characters with U+FFFD REPLACEMENT CHARACTER
mappu04 commented 2019-05-17 06:07:44 +00:00
Author
Owner
Copy Link
2019/04/07 13:02:08 Delivering PM to telegram user: Bad Request: strings must be encoded in UTF-8
``` 2019/04/07 13:02:08 Delivering PM to telegram user: Bad Request: strings must be encoded in UTF-8 ```
mappu04 commented 2020-03-26 00:05:24 +00:00
Author
Owner
Copy Link
2020/03/26 11:12:28 Delivering public message to group chat: Bad Request: text must be encoded in UTF-8
``` 2020/03/26 11:12:28 Delivering public message to group chat: Bad Request: text must be encoded in UTF-8 ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

   
wishlist
No Label
bug
wishlist
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: code.ivysaur.me/nmdc-telegramfrontend#20
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?