Commit 8d723b34 authored by Kirill Smelkov's avatar Kirill Smelkov

golang: tests: Spawn subprocess python with $PYTHONIOENCODING set to encoding...

golang: tests: Spawn subprocess python with $PYTHONIOENCODING set to encoding of stdin/stdout/stderr

We need to do it because on Windows `python x.py | ...` runs with stdio
encoding set to cp125X even if just `python x.py` runs with stdio
encoding=UTF-8.

    (1.wenv) Z:\home\kirr\src\tools\go\pygo-win\pygolang\golang\testprog>python golang_test_str.py
    print(qq(b)): "привет b"
    print(qq(u)): "привет u"

    (1.wenv) Z:\home\kirr\src\tools\go\pygo-win\pygolang\golang\testprog>python golang_test_str.py |more
    print(qq(b)): "яЁштхЄ b"
    print(qq(u)): "яЁштхЄ u"

    (1.wenv) Z:\home\kirr\src\tools\go\pygo-win\pygolang\golang\testprog>python golang_test_str.py >aaa.txt
    (1.wenv) Z:\home\kirr\src\tools\go\pygo-win\pygolang\golang\testprog>type aaa.txt
    print(qq(b)): "яЁштхЄ b"
    print(qq(u)): "яЁштхЄ u"

Which leads to the following test_strings_print failure:

    E           Failed: not equal:
    E           Expected:
    E               print(qq(b)): "привет b"
    E               print(qq(u)): "привет u"
    E           Got:
    E               print(qq(b)): "������ b"
    E               print(qq(u)): "������ u"

We also change golang_test_str.py to print not only russian and english
letters, but also greek ones. This is to make sure that stdout IO encoding would
not go cp125X unnoticed because then printing will fail with UnicodeEncodeError.

Note: in the above example both got and expected are wrong. Via
$PYTHONIOENCODING we only fix "got" and we will fix "expected" in the followup
patch. After current patch test_strings_print result is

    E           Failed: not equal:
    E           Expected:
    E               print(qq(b)): "привет αβγ b"
    E               print(qq(u)): "привет αβγ u"
    E           Got:
    E               print(qq(b)): "привет αβγ b"
    E               print(qq(u)): "привет αβγ u"
parent ee55e19d
......@@ -1739,6 +1739,17 @@ def _pyrun(argv, stdin=None, stdout=None, stderr=None, **kw): # -> retcode, st
pathv.extend(envpath.split(os.pathsep))
env['PYTHONPATH'] = os.pathsep.join(pathv)
# set $PYTHONIOENCODING to encoding of stdin/stdout/stderr
# we need to do it because on Windows `python x.py | ...` runs with stdio
# encoding set to cp125X even if just `python x.py` runs with stdio
# encoding=UTF-8.
if 'PYTHONIOENCODING' not in env:
enc = set([_.encoding for _ in (sys.stdin, sys.stdout, sys.stderr)])
if None in enc: # without -s pytest uses _pytest.capture.DontReadFromInput
enc.remove(None) # with None .encoding
assert len(enc) == 1
env['PYTHONIOENCODING'] = enc.pop()
p = Popen(argv, stdin=(PIPE if stdin else None), stdout=stdout, stderr=stderr, env=env, **kw)
stdout, stderr = p.communicate(stdin)
......
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (C) 2022 Nexedi SA and Contributors.
# Copyright (C) 2022-2023 Nexedi SA and Contributors.
# Kirill Smelkov <kirr@nexedi.com>
#
# This program is free software: you can Use, Study, Modify and Redistribute
......@@ -29,8 +29,8 @@ from golang import b, u
from golang.gcompat import qq
def main():
sb = b("привет b")
su = u("привет u")
sb = b("привет αβγ b")
su = u("привет αβγ u")
print("print(qq(b)):", qq(sb))
print("print(qq(u)):", qq(su))
......
print(qq(b)): "привет b"
print(qq(u)): "привет u"
print(qq(b)): "привет αβγ b"
print(qq(u)): "привет αβγ u"
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment