Allow a full range of unicode characters in text messages - #4937
Open
quis wants to merge 2 commits into
Open
Conversation
quis
force-pushed
the
allow-unicode-sms
branch
from
August 17, 2026 10:49
d24fec0 to
2eefcd6
Compare
quis
changed the title
2 tasks
quis
force-pushed
the
allow-unicode-sms
branch
from
August 21, 2026 09:09
2eefcd6 to
a10271c
Compare
quis
force-pushed
the
allow-unicode-sms
branch
from
September 1, 2026 09:38
a10271c to
27233d2
Compare
quis
marked this pull request as ready for review
September 1, 2026 09:40
CrystalPea
reviewed
Sep 1, 2026
| if non_gsm_characters := template.non_gsm_characters: | ||
| current_app.logger.warning( | ||
| "%s character(s) replaced with ? in SMS content for service %s and notification %s", | ||
| "%s character(s) caused UTF-16 encoding in SMS content for service %s and notification %s", |
Contributor
There was a problem hiding this comment.
do we still want this log to be a warning? Since this will be an expected thing?
Member
Author
There was a problem hiding this comment.
I figure it will be useful initially to see adoption of the feature, but we can remove it eventually
Contributor
There was a problem hiding this comment.
should we move the log type from warning to info though?
If they are in a template or personalisation, start sending them to the message’s recipients. *** Includes bumping utils to 128.0.0 ## 128.0.0 * Allows a full range of unicode characters in text messages * Removes `SanitiseSMS.downgrade_character` and `SanitiseSMS.encode_char`, `SanitiseSMS.WELSH_DIACRITICS` and `SanitiseSMS.ALLOWED_CHARACTERS` (should not be used anywhere) * Removes `SanitiseSMS.get_non_gsm_characters()` (use `template.non_gsm_characters` instead) * `formatters.normalise_whitespace_and_newlines` and `formatters.normalise_whitespace` have a new, optional `preserve_zero_width_joiner` argument (no change in behaviour if not set to `True`) ## 127.1.0 * Adds additional methods to `recipient_validation.phone_number.PhoneNumber`, `is_uk_mobile_number` and `get_carrier_info` *** Complete changes: alphagov/notifications-utils@127.0.0...128.0.0
Non-GSM characters in text message content is an expected behaviour now, not something which needs warning about.
quis
force-pushed
the
allow-unicode-sms
branch
from
September 1, 2026 13:35
27233d2 to
439c519
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on: