#Custom migration func not working with taggit TaggableManager

16 messages · Page 1 of 1 (latest)

modest zephyr
#

Hi All 👋

I'm trying to break up a custom taggit model that I overdid originally, the migration for it works fine on fresh/empty DB, but i want to convert some existing items during migration in DB, so i made a RunPython migration:

def convert_subcat_tag_to_subcategories_forward_func(apps, schema_editor):
    # create subcategories for each existing subcat tag
    # add subcategories to posts and also dropdownnavitems
    # remove the empty tags
    CategoryTag = apps.get_model("blog", "CategoryTag")
    Post = apps.get_model("blog", "Post")
    Subcategory = apps.get_model("blog", "Subcategory")
    DropdownNavItem = apps.get_model("blog", "DropdownNavItem")

    for tag in CategoryTag.objects.filter(is_sub_category=True):
        subcategory = Subcategory.objects.create(
            name=tag.name,
            slug=tag.slug,
            description=tag.description,
            preview_image=tag.preview_image,
        )
        subcategory.categories.set(tag.categories.all())

        for post in Post.objects.filter(tags=tag):
            post.subcategories.add(subcategory)
            post.tags.remove(tag)

        for item in DropdownNavItem.objects.filter(category_tag=tag):
            item.subcategory = subcategory
            item.save()

        tag.delete()

This fails on line for post in Post.objects.filter(tags=tag): when trying to use TaggableManager with error:

django.core.exceptions.FieldDoesNotExist: TaggedWithCategoryTags has no field named 'content_object'

I have not altered the tags relation in any way in the models, just added new model. I'm having trouble understanding why this is happening and what my next steps could be.

This pattern Post.objects.filter(tags=tag) works well everywhere else during normal runtime, how can i figure out what's wrong with the migration in particular?

I ran into character limit, posting more details below.

Thanks in advance!

#

This fails on line for post in Post.objects.filter(tags=tag): when trying to use TaggableManager with errors:

 /backend> .\manage.py migrate blog
Operations to perform:
  Apply all migrations: blog
Running migrations:
  Applying blog.0016_alter_category_options_alter_categorytag_options_and_more...Traceback (most recent call last):
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\options.py", line 681, in get_field
    return self.fields_map[field_name]
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'content_object'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "\Documents\Programming\Django-Blog\backend\manage.py", line 22, in <module>
    main()
  File "\Documents\Programming\Django-Blog\backend\manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\core\management\__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\core\management\__init__.py", line 436, in execute    
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\core\management\base.py", line 412, in run_from_argv  
    self.execute(*args, **cmd_options)
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\core\management\base.py", line 458, in execute        
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\core\management\base.py", line 106, in wrapper        
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\core\management\commands\migrate.py", line 356, in handle
    post_migrate_state = executor.migrate(
                         ^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\migrations\executor.py", line 135, in migrate      
    state = self._migrate_all_forwards(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\migrations\executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
            ^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\migrations\executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\migrations\migration.py", line 132, in apply       
    operation.database_forwards(
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\migrations\operations\special.py", line 193, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "\Documents\Programming\Django-Blog\backend\blog\migrations\0016_alter_category_options_alter_categorytag_options_and_more.py", line 25, in convert_subcat_tag_to_subcategories_forward_func
    for post in Post.objects.filter(tags=tag):
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\manager.py", line 87, in manager_method     
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\query.py", line 1436, in filter
    return self._filter_or_exclude(False, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\query.py", line 1454, in _filter_or_exclude 
    clone._filter_or_exclude_inplace(negate, args, kwargs)
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\query.py", line 1461, in _filter_or_exclude_inplace
    self._query.add_q(Q(*args, **kwargs))
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\sql\query.py", line 1545, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\sql\query.py", line 1576, in _add_q
    child_clause, needed_inner = self.build_filter(
                                 ^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\sql\query.py", line 1426, in build_filter   
    lookups, parts, reffed_expression = self.solve_lookup_type(arg, summarize)
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\sql\query.py", line 1236, in solve_lookup_type
    _, field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\sql\query.py", line 1738, in names_to_path  
    if hasattr(field, "path_infos"):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\utils\functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#
                                         ^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\taggit\managers.py", line 698, in path_infos
    return self.get_path_info()
           ^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\taggit\managers.py", line 692, in get_path_info
    return self._get_mm_case_path_info(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\taggit\managers.py", line 610, in _get_mm_case_path_info     
    linkfield1 = self.through._meta.get_field("content_object")
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.virtualenvs\Django-Blog-1jR6fO5V\Lib\site-packages\django\db\models\options.py", line 683, in get_field
    raise FieldDoesNotExist(
django.core.exceptions.FieldDoesNotExist: TaggedWithCategoryTags has no field named 'content_object'

#

I'm using taggit custom tags, here are the models:

from taggit.managers import TaggableManager
from taggit.models import GenericTaggedItemBase, TagBase

class CategoryTag(TagBase): ## <- model i'm trying to break up
    description = models.CharField(max_length=250, blank=True)
    categories = models.ManyToManyField(Category, blank=True)
    subcategories = models.ManyToManyField(Subcategory, blank=True)

    objects = models.Manager()
    non_empty = NonEmptyTagsManager()

    class Meta:
        verbose_name = gettext_lazy("Tag")
        verbose_name_plural = gettext_lazy("Tags")
        ordering = ["name"]
    ...

class TaggedWithCategoryTags(GenericTaggedItemBase):
    tag = models.ForeignKey(
        CategoryTag,
        on_delete=models.CASCADE,
        related_name="%(app_label)s_%(class)s_tags",
    )


class Post(ModelMeta, models.Model):
    class Status(models.TextChoices):
        DRAFT = "DF", "Draft"
        PUBLISHED = "PB", "Published"

    title = models.CharField(max_length=250)
    ...
    categories = models.ManyToManyField(Category)
    subcategories = models.ManyToManyField(Subcategory, blank=True)
    tags = TaggableManager(
        through=TaggedWithCategoryTags,
        verbose_name="Tags with categories",
        help_text="Tags, comma separated",
        related_name="tag_posts",
    )
#

Why is TaggableManager not working properly during migration?

I've been able to get the TaggedWithCategoryTags through model objects during migration. I'm guessing as a workaround I could try deleting the relation instances there directly, but that doesn't strike me as a great idea and I'm not super sure yet how to make it reversible.

sterile kelp
#

You don't have access to the actual models and managers in the migrations. You're working with a snapshot of the model state

#

Any custom methods you have for models, including managers are not usable in data migrations. You'll need to recreate them

modest zephyr
#

alright thanks so much, that makes sense, how can i go about recreating the managers? same as defining them in the models.py file?

sterile kelp
#

Hmm. I'm not sure. That's worth a search.

I would probably create the queryset from scratch

#

Or make the data migration a temporary management command

#

As long as this migration is elidable. Meaning if you were to squash the migrations of the app, this data migration would disappear

#

Edited. It is worth a search

modest zephyr
#

thanks, i think in terms of getting things done management command/manual intervention seems like the fastest option at the moment

i'll try searching some more, thanks again for the help

sterile kelp
#

You're welcome