#UnicodeDecodeError: 'ascii' codec can't decode...

13 messages · Page 1 of 1 (latest)

knotty iron
#

I'm using postgresql with a utf8 charset. I have some unicode text that is stored in a story field of the GlobalLuminaria model.

I'm getting this error:

GlobalLuminaria.objects.filter(story__icontains=u'Hvis bare').first().delete()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/query.py", line 1057, in first
    for obj in queryset[:1]:
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/query.py", line 398, in __iter__
    self._fetch_all()
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1593, in execute_sql
    return list(result)
           ^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 2091, in cursor_iter
    for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 2091, in <lambda>
    for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/utils.py", line 98, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 84: ordinal not in range(128)```
#

The shortened version of the model is

    story = models.TextField(max_length=500, blank=True, null=True)```

Here is the text that was stored. ```Story: Mor... Jeg elsker dig og savner dig uendeligt meget. Hvis bare jeg kunne kramme dig én gang mere, og fortælle dig at du betyder alt for mig. Der er sket så meget siden du fik dine englevinger, men der er ikke gået en dag hvor jeg ikke har tænkt på dig.
Savnet men altid med i mit hjerte.```

My question: How do I fix this current entry so that my application can handle it within views, etc.?
pure pelican
#

What version of python are you using?

#

The u'' syntax is from python 2 no?

knotty iron
#

I'm using python 3.11.2 and Django 4.2

#

That one I was just testing with that syntax to try and fix. It also gives the same error if I call it without, for example.

#

GlobalLuminaria.objects.filter(approved=True).order_by('-creation_date')[1].story gives the same error

#

So I guess I'm saving it in utf8 but for some reason it can't be read back in the same encoding?

pure pelican
#

The system is trying to convert the text into ascii when it's utf8

knotty iron
#

Hmm hadn't seen that one

pure pelican
#

Good point. The db could have the wrong encoding