🔒 codec cant decode even with correct encoding | Python | Page 1

naive sand Jan 24, 2026, 5:41 PM

#

im trying to save the output in the logfile.txt but im getting this error:
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9c in position 196: character maps to <undefined>
decoding with 'cp1253' codec failed

tribal burrowBOT Jan 24, 2026, 5:41 PM

#

naive sand im trying to save the output in the logfile.txt but im getting this error: Unico...

#

@naive sand

Python help channel opened

Remember to:

Ask your Python question, not if you can ask or if there's an expert who can help.
Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

naive sand Jan 24, 2026, 5:41 PM

#

!paste

#

heres the code https://paste.pythondiscord.com/DJQA

#

i have almost no idea what any of these mean, im just messing around and encountered this

safe basalt Jan 24, 2026, 5:44 PM

#

why not just use UTF-8 instead of what ever the system reports as the locally preferred encoding?

naive sand Jan 24, 2026, 5:44 PM

#

because

#

uh

#

idk how to do that

#

the patch wasnt even made by me, it was made by a friend and idk how it works

midnight thunder Jan 24, 2026, 5:47 PM

#

The answer is to probably just use utf-8 but there is also a standard library encoding detector that may work depending on the output

#

!d tokenize.detect_encoding

tribal burrowBOT Jan 24, 2026, 5:47 PM

#

tokenize.detect\_encoding


tokenize.detect_encoding(readline)```
The [`detect_encoding()`](https://docs.python.org/3/library/tokenize.html#tokenize.detect_encoding) function is used to detect the encoding that should be used to decode a Python source file. It requires one argument, readline, in the same way as the [`tokenize()`](https://docs.python.org/3/library/tokenize.html#tokenize.tokenize) generator.

It will call readline a maximum of twice, and return the encoding used (as a string) and a list of any lines (not decoded from bytes) it has read in.

It detects the encoding from the presence of a UTF-8 BOM or an encoding cookie as specified in [**PEP 263**](https://peps.python.org/pep-0263/). If both a BOM and a cookie are present, but disagree, a [`SyntaxError`](https://docs.python.org/3/library/exceptions.html#SyntaxError) will be raised. Note that if the BOM is found, `'utf-8-sig'` will be returned as an encoding...

safe basalt Jan 24, 2026, 5:48 PM

#

naive sand idk how to do that

skip the line

encod = locale.getpreferredencoding(False)
```and change another line to
```py
output = output_bytes.decode("utf-8")

naive sand Jan 24, 2026, 5:49 PM

#

its throwing the error invalid start byte

#

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 185: invalid start byte

safe basalt Jan 24, 2026, 5:57 PM

#

naive sand UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 185: invali...

this at least works on my linux system (which is set to natively use utf-8 out of the box)

subprocess.check_output(["ifconfig"], encoding="utf-8")
```(as it's linux i changed `ipconfig` to `ifconfig`)

#

shell=True is the default for subprocess.check_output() anyways

naive sand Jan 24, 2026, 5:58 PM

#

mine(w11) is cp1253? idrk what that means

#

anyway

#

thanks in advance

safe basalt Jan 24, 2026, 6:01 PM

#

cp1253 ("Greek - ANSI") is one of the different character encodings that windows use depending on your language/locale settings

#

https://en.wikipedia.org/wiki/Windows-1253

naive sand Jan 24, 2026, 6:08 PM

#

ima just give up atp

#

it's been 2 hours of trying

wide mist Jan 24, 2026, 6:09 PM

#

I kinda want a problematic excerpt of the logfile... is it seriously an exotic text encoding?

#

Unless your text is ASCII or utf-8, something is weird.

naive sand Jan 24, 2026, 6:11 PM

#

could you specify what you mean by excerpt

#

my English isn't the best

safe basalt Jan 24, 2026, 6:12 PM

#

naive sand ima just give up atp

so neither

import subprocess
output = subprocess.check_output(["ipconfig"], encoding="utf-8")
with open("logfile.txt", "w") as file:
    file.write(output)
```or
```py
import subprocess
output = subprocess.check_output(["ipconfig"], encoding="cp1253")
with open("logfile.txt", "w") as file:
    file.write(output)
```works?

naive sand Jan 24, 2026, 6:12 PM

#

pretty sure they didn't when i last checked

wide mist Jan 24, 2026, 6:12 PM

#

naive sand could you specify what you mean by excerpt

Show us a problematic piece of the logfile, is what I mean

naive sand Jan 24, 2026, 6:12 PM

#

let me check again

naive sand Jan 24, 2026, 6:13 PM

#

wide mist Show us a problematic piece of the logfile, is what I mean

the logfile is empty

safe basalt Jan 24, 2026, 6:13 PM

#

or you could try the hacky way:

import subprocess
output = subprocess.check_output(["ipconfig"])
with open("logfile.txt", "wb") as file:
    file.write(output)
```and you'll have the raw output bytes in the log file instead

safe basalt Jan 24, 2026, 6:14 PM

#

naive sand the logfile is empty

due to the encoding step failing and the script aborts before it can be written to the file

naive sand Jan 24, 2026, 6:16 PM

#

safe basalt or you could try the hacky way: ```py import subprocess output = subprocess.chec...

can i decode them later on?

wide mist Jan 24, 2026, 6:17 PM

#

Well, by extension I mean what you're trying to write to the file...
Windows also likes to produce ansi/mbcs text. You should re-encode it if you get that stuff, it sucks.

safe basalt Jan 24, 2026, 6:17 PM

#

naive sand can i decode them later on?

in that last case you would have the raw output in the file in whatever encoding it was in, hopefully windows can recognize it when you open the file or you can set the encoding that it should be read as in for example notepad

naive sand Jan 24, 2026, 6:18 PM

#

safe basalt or you could try the hacky way: ```py import subprocess output = subprocess.chec...

only this works

naive sand Jan 24, 2026, 6:19 PM

#

safe basalt in that last case you would have the raw output in the file in whatever encoding...

some characters have a rhombus with a question mark

safe basalt Jan 24, 2026, 6:19 PM

#

naive sand only this works

it should work, but the log file content might look strange if you open the file and try to decode it using the wrong encoding

naive sand Jan 24, 2026, 6:19 PM

#

and there's some random chinese letters too

safe basalt Jan 24, 2026, 6:19 PM

#

yeah, that is probably due to that problem

wide mist Jan 24, 2026, 6:19 PM

#

Try viewing the log file using mbcs/ansi encoding.

#

It came from windows, so windows probably used their ugly encoding.

naive sand Jan 24, 2026, 6:20 PM

#

also how did my ipv4 address change shouldn't it be the same?

naive sand Jan 24, 2026, 6:21 PM

#

wide mist It came from windows, so windows probably used their ugly encoding.

alright

#

it still won't work

safe basalt Jan 24, 2026, 6:25 PM

#

naive sand it still won't work

maybe time to try out @midnight thunder suggestion above?

naive sand Jan 24, 2026, 6:25 PM

#

it's fine tbh, as long as i can still read the stuff written in English and the numbers

#

thanks for the help!

wide mist Jan 24, 2026, 6:26 PM

#

naive sand it still won't work

I meant write raw bytes to the file and use your text editor to view it as ANSI.

naive sand Jan 24, 2026, 6:26 PM

#

the only corrupted characters are the greek ones which idrc about

naive sand Jan 24, 2026, 6:26 PM

#

wide mist I meant write raw bytes to the file and use your text editor to view it as ANSI.

it defaults as ansi

wide mist Jan 24, 2026, 6:27 PM

#

Hmmm... alright. Maybe it really is windows trynna be smart with some weird encoding like 1253. Glad you got it working well enough tho

naive sand Jan 24, 2026, 6:28 PM

#

thanks! and sorry for the hassle

#

i must up my knowledge about encodings

#

i didn't even know the output text is encoded when i started out the script

safe basalt Jan 24, 2026, 6:31 PM

#

can you provide a sample (as text, not a screenshot) as it is output by print(output) on the screen? (you can change your ip address and mac address if you want to anonymize it)

#

!code

tribal burrowBOT Jan 24, 2026, 6:31 PM

#

Formatting code on Discord

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

For long code samples, you can use our pastebin.

naive sand Jan 24, 2026, 6:32 PM

#

one second

safe basalt Jan 24, 2026, 6:32 PM

#

because then we will see the bytes, and non-printable bytes and such will be encoded as hex or similar

naive sand Jan 24, 2026, 6:34 PM

#

b'\r\nWindows IP Configuration\r\n\r\n\r\nEthernet adapter Ethernet:\r\n\r\n Media State . . . . . . . . . . . : Media disconnected\r\n Connection-specific DNS Suffix . : \r\n\r\nWireless LAN adapter \x92\xa6\xa7\xa0\xa1\xe3 \xa9\xe7\xa4\x9b\x9c\xa9\x9e* 1:\r\n\r\n Media State . . . . . . . . . . . : Media disconnected\r\n Connection-specific DNS Suffix . : \r\n\r\nWireless LAN adapter \x92\xa6\xa7\xa0\xa1\xe3 \xa9\xe7\xa4\x9b\x9c\xa9\x9e* 2:\r\n\r\n Media State . . . . . . . . . . . : Media disconnected\r\n Connection-specific DNS Suffix . : \r\n\r\nMobile Broadband adapter \x83\xe5\xa1\xab\xac\xa6 \xa1\xa0\xa4\x9e\xab\xe3\xaa \xab\x9e\xa2\x9c\xad\xe0\xa4\xe5\x98\xaa:\r\n\r\n Media State . . . . . . . . . . . : Media disconnected\r\n Connection-specific DNS Suffix . : \r\n\r\nWireless LAN adapter Wi-Fi:\r\n\r\n Connection-specific DNS Suffix . : \r\n Link-local IPv6 Address . . . . . : (ipv6) IPv4 Address. . . . . . . . . . . : (ipv4) Subnet Mask . . . . . . . . . . . : (idek what this is) Default Gateway . . . . . . . . . :(this neither)'

#

if i missed any sensitive info pls dont dox me

safe basalt Jan 24, 2026, 6:44 PM

#

naive sand b'\r\nWindows IP Configuration\r\n\r\n\r\nEthernet adapter Ethernet:\r\n\r\n M...

can you try

import subprocess
output = subprocess.check_output(["ipconfig"], encoding="cp737)
with open("logfile.txt", "w") as file:
    file.write(output)
```and see if the content makes sense to you after that? (i don't know greek)
if it even works and not errors out on you again

naive sand Jan 24, 2026, 6:46 PM

#

i tried writing bytes and then decoding with 737 but it only shows the unknown character symbol instead of the mixed chinese

safe basalt Jan 24, 2026, 6:47 PM

#

you might also try iso8859-7

naive sand Jan 24, 2026, 6:48 PM

#

instead of cp737?

safe basalt Jan 24, 2026, 6:48 PM

#

yeah

naive sand Jan 24, 2026, 6:49 PM

#

still raises te

#

the initial error

safe basalt Jan 24, 2026, 6:49 PM

#

but for me it just errored out for the sample you sent

#

yeah, but cp737 at least doesn't throw the error

naive sand Jan 24, 2026, 6:49 PM

#

yea

safe basalt Jan 24, 2026, 6:50 PM

#

i need to go afk for a while

naive sand Jan 24, 2026, 6:51 PM

#

idk if i understood correctly but chcp and the locale command give out different encodings even tho they should be the same?

naive sand Jan 24, 2026, 6:51 PM

#

safe basalt i need to go afk for a while

alright

#

thanks for your help!

safe basalt Jan 24, 2026, 7:37 PM

#

naive sand alright

kind of a long-shot that seems to be windows specific as i can only use it with python on windows and not on linux
if you set the encoding to either oem or mbcs, for example:

import subprocess
output = subprocess.check_output(["ipconfig"], encoding="oem")
with open("logfile.txt", "w") as file:
    file.write(output)

#

see if it gives you anything readable and without throwing a exception at you 🤞

naive sand Jan 24, 2026, 7:39 PM

#

so windows just sucks?

naive sand Jan 24, 2026, 7:39 PM

#

safe basalt see if it gives you anything readable and without throwing a exception at you 🤞

on it

#

nope, still the same, unknown characters instead of greek letters

safe basalt Jan 24, 2026, 7:42 PM

#

well, microsoft does things their own way and historically hasn't been much for following standards (that they haven't invented them selves) and even kept a lot of stuff proprietary (but that has been changing but they still have a lot of legacy to drag around), but it's mostly just "different" 😉

safe basalt Jan 24, 2026, 7:43 PM

#

naive sand nope, still the same, unknown characters instead of greek letters

and you did change the mode in the open to be "w" and not "wb" when you ran this?

#

as when we are trying to decode we shouldn't try to write it as raw bytes to the file

naive sand Jan 24, 2026, 7:44 PM

#

yes

safe basalt Jan 24, 2026, 7:46 PM

#

this is really hard as i can't even see if it's greek characters or just some strange letters as i can't read greek at all

naive sand Jan 24, 2026, 7:46 PM

#

here's a preview

#

αβγδεζηθικλμνξοπρστυχψω

safe basalt Jan 24, 2026, 7:46 PM

#

can you try to print it out to the terminal instead of a file?

naive sand Jan 24, 2026, 7:47 PM

#

safe basalt can you try to print it out to the terminal instead of a file?

I'm pretty sure we did that before too

#

unless you mean with oem encoding

safe basalt Jan 24, 2026, 7:47 PM

#

import subprocess
prnt(isubprocess.check_output(["ipconfig"], encoding="oem"))

#

and try some different encodings besides "oem" as well

naive sand Jan 24, 2026, 7:48 PM

#

printing does work

safe basalt Jan 24, 2026, 7:49 PM

#

such as "mbcs" or "cp737" as well

naive sand Jan 24, 2026, 7:50 PM

#

printing works but writing to file doesn't

#

I've used cp737 and it doesn't work

safe basalt Jan 24, 2026, 7:50 PM

#

than that might be half the battle

#

but both "oem" as well as "mbcs" does?

naive sand Jan 24, 2026, 7:52 PM

#

mbcs doesn't work

safe basalt Jan 24, 2026, 7:52 PM

#

okay, so only "oem" works?

naive sand Jan 24, 2026, 7:53 PM

#

oem still doesn't write the greek letters in the file

#

it only decodes them to print them

safe basalt Jan 24, 2026, 7:53 PM

#

no, but at least the to terminal, right?

naive sand Jan 24, 2026, 7:54 PM

#

yea

safe basalt Jan 24, 2026, 7:56 PM

#

what if we try:

import subprocess
output = subprocess.check_output(["ipconfig"], encoding="oem")
with open("logfile.txt", "w", encoding="utf-8") as file:
    file.write(output)
```and you them make sure that the application that you are reading the log file with is set to "auto" or UTF-8 ?

#

trying to force it to make the right thing happen

naive sand Jan 24, 2026, 7:57 PM

#

bruh

#

it works

#

vs code decided to just not read it

#

notepad does tho

#

thank you SO much!

safe basalt Jan 24, 2026, 7:59 PM

#

naive sand vs code decided to just not read it

might be some setting in vs code that needs to be adjusted 🤷

naive sand Jan 24, 2026, 7:59 PM

#

possibly

safe basalt Jan 24, 2026, 7:59 PM

#

naive sand thank you SO much!

you're welcome and good luck with your project

naive sand Jan 24, 2026, 7:59 PM

#

thanks!

safe basalt Jan 24, 2026, 8:00 PM

#

you might want to delete the sample output

naive sand Jan 24, 2026, 8:00 PM

#

I've deleted the sensitive info but i will anyway

#

thanks again!

safe basalt Jan 24, 2026, 8:01 PM

#

just so that it doesn't stick around here in the read-only archives after this thread has been closed in-case the greek text that i can't read (but someone else might) contains anything sensitive

naive sand Jan 24, 2026, 8:01 PM

#

no it's just translation of the English text

safe basalt Jan 24, 2026, 8:06 PM

#

when you are done with the help thread you may close it with !close or you can wait for it to be closed by the bot after an hour without any new messages in the thread (as edits and reactions don't count towards the idle timeout)

#

👋

tribal burrowBOT Jan 24, 2026, 9:06 PM

#

Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.

#🔒 codec cant decode even with correct encoding