[docs] IO > Text Encoding info outdated

[davidgilbertson]

BPO	46143
Nosy	@methane, @eryksun, @davidgilbertson

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-12-21.04:42:17.894>
labels = ['type-bug', '3.10', 'docs']
title = '[docs] IO > Text Encoding info outdated'
updated_at = <Date 2021-12-21.09:07:08.224>
user = '/~https://github.com/davidgilbertson'

bugs.python.org fields:

activity = <Date 2021-12-21.09:07:08.224>
actor = 'eryksun'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2021-12-21.04:42:17.894>
creator = 'gilbertson.david'
dependencies = []
files = []
hgrepos = []
issue_num = 46143
keywords = []
message_count = 3.0
messages = ['408983', '408985', '408993']
nosy_count = 4.0
nosy_names = ['methane', 'docs@python', 'eryksun', 'gilbertson.david']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue46143'
versions = ['Python 3.10']

[davidgilbertson]

On this page: https://docs.python.org/3/library/io.html#text-encoding it says "there is no concrete plan as of yet, Python may change the default text file encoding to UTF-8 in the future".

On this page https://docs.python.org/3/library/os.html#utf8-mode is says that from 3.7 onwards UTF-8 will be selected by default.

Does that mean that the text in the first section is now outdated, as it was addressed by PEP-540?

I'm a newbie, so apologies if I'm missing something obvious or filing this in the wrong spot.

[methane]

UTF-8 mode is not enabled by default. So locale encoding is still the default encoding.

[eryksun]

The rare circumstance in which UTF-8 mode gets enabled automatically is described in the following paragraph [1]:

If the PYTHONUTF8 environment variable is not set at all, then the
interpreter defaults to using the current locale settings, unless the
current locale is identified as a legacy ASCII-based locale (as
described for PYTHONCOERCECLOCALE), and locale coercion is either 
disabled or fails. In such legacy locales, the interpreter will
default to enabling UTF-8 mode unless explicitly instructed not to do
so.

Note that UTF-8 mode is never enabled automatically in Windows. In contrast to POSIX, the locale encoding in Windows is unrelated to the current LC_CTYPE locale. Instead, the locale encoding gets set to the process code page, which is based on the system locale by default and never changes while a process is running. The system locale may be incompatible with the current LC_CTYPE locale, Windows user locale, and preferred UI language (e.g. for text resources such as error messages), so try to explicitly use UTF-8 for text files whenever possible.

---
[1] https://docs.python.org/3/library/os.html#utf8-mode

[slateny]

Is there anything to change on the os docs then since the comments on ... use the UTF-8 encoding apply when UTF-8 mode is enabled only? Or should it be clarified, so something like

... other higher level APIs also exhibit different default behaviours [if UTF-8 mode is enabled:]

[methane]

PEP 686 has been accepted.
There is a plan to enable UTF-8 mode by default for now.

It is better to update 3.11 and 3.12 document about PEP 686.

[orsenthil]

It is better to update 3.11 and 3.12 document about PEP 686.

SGTM. Thank you.