#I need some help with Model creation

67 messages · Page 1 of 1 (latest)

stiff otter
#

I need some help with Models architecture
Lets say i have 8 categories
each category have shared fields with each other but also unique fields
for example
category1 can have aa, year, from_field, to, observation
category2 can have aa, from_field, to, security, office
etc

Now i have a file that should be uploaded in one of these categories
so file model has aa and categories fields that must be unique together

categories = [
    ("1", "..."),
    ("2", "..."),
    ("3", "..."),
    ("4", "..."),
    ("5", "..."),
    ("6", "..."),
    ("7", "..."),
    ("8", "..."),
    ("ΑΛΛΟ", "..."),
]


class BaseCategory(models.Model):
    name = models.CharField(max_length=100, choices=categories, primary_key=True)
    file_upload = models.FileField(upload_to="files/")
    observation = models.TextField(max_length=500)

    class Meta:
        abstract = True


class Category1(BaseCategory):
    year = models.IntegerField()
    from_field = models.CharField(max_length=100)
    to = models.CharField(max_length=100)


class Category2(BaseCategory):
    from_field = models.CharField(max_length=100)
    to = models.CharField(max_length=100)
    office = models.ForeignKey(Office, on_delete=models.SET_NULL, null=True)
    security_level = models.ForeignKey(Security, on_delete=models.SET_NULL, null=True)

class File(models.Model):
    category = models.ForeignKey(
        Category1, choices=categories, on_delete=models.CASCADE, related_name="files"
    )
    aa = models.CharField(max_length=100)
    year = models.IntegerField()
    from_field = models.CharField(max_length=100)
    to = models.CharField(max_length=100)
    office = models.ForeignKey(
        Office, on_delete=models.SET_NULL, null=True, related_name="files"
    )
    security_level = models.ForeignKey(
        Security, on_delete=models.SET_NULL, null=True, related_name="files"
    )
    observation = models.TextField(max_length=500)
    file_upload = models.FileField(upload_to="files/")
livid swift
#

And the question is?
Overall your model looks problematic to track File category

stiff otter
#

lets say i have 8 different categories
do i need to have a model for each one? or i can have a category model with choice in the category_name field?
Can i have some dynamic models by providing choice inside fields?
how would you do it?
you dont need to provide code just some architectural logic of the database

#

or maybe i should just have each category as a model and inside that model keep some file fields?
but if i do that approach i should forbid a file to be uploaded in 2 different categories how can i forbid that?

livid swift
#

Having 8 category models poses a problem - you can't easily set File to pick one of them

#

So I'd consider either single Category model or 9 models:

#

Categoty and CategoryExtension# with one-to-one relation to Category

#

9 models sounds quite a lot tho, even if it's not technically a problem, maybe single Category model with blank fields would be cleaner

stiff otter
# 
class BaseCategory(models.Model):
    file_name = models.CharField(max_length=200)
    file_upload = models.FileField(upload_to="files/")
    observation = models.TextField(max_length=500)
    date_added = models.DateTimeField(auto_now_add=True, editable=False)

    class Meta:
        abstract = True


class Category1(BaseCategory):
    year = models.IntegerField()
    from_field = models.DateField()
    to = models.DateField()


class Category2(BaseCategory):
    from_field = models.DateField()
    to = models.DateField()
    office = models.CharField(max_length=20, choices=offices)
    security_level = models.CharField(max_length=20, choices=security_levels)


class Category3(BaseCategory):
    from_field = models.DateField()
    to = models.DateField()
    office = models.CharField(max_length=20, choices=offices)
    security_level = models.CharField(max_length=20, choices=security_levels)


class Category4(BaseCategory):
    year = models.IntegerField()
    object = models.CharField(max_length=100)
    action_order = models.CharField(max_length=100)
    office = models.CharField(max_length=20, choices=offices)
    security_level = models.CharField(max_length=20, choices=security_levels)


class Category5(BaseCategory):
    year = models.IntegerField()
    from_field = models.DateField()
    to = models.DateField()
    office = models.CharField(max_length=20, choices=offices)
    security_level = models.CharField(max_length=20, choices=security_levels)
# 
class Category6(BaseCategory):
    from_field = models.DateField()
    to = models.DateField()
    office = models.CharField(max_length=20, choices=offices)
    security_level = models.CharField(max_length=20, choices=security_levels)


class Category7(BaseCategory):
    from_field = models.DateField()
    to = models.DateField()
    office = models.CharField(max_length=20, choices=offices)
    security_level = models.CharField(max_length=20, choices=security_levels)


class Category8(BaseCategory):
    from_field = models.DateField()
    to = models.DateField()
    mail_type = models.CharField(max_length=20, choices=mail_types)
    object = models.CharField(max_length=100)
    office = models.CharField(max_length=20, choices=offices)
    security_level = models.CharField(max_length=20, choices=security_levels)
livid swift
#

No, definetely not like that

stiff otter
#

yeah...
even tho in admin panel looks clean etc when you can go add an instance of a category you can specify the fields right but i cant rpevent duplication of files and i dont even have a file model here

idk its tricky in my mind

livid swift
#

Hm wait, you have exactly 8 categories?

#

not category types?

stiff otter
#

btw each category has a

class Meta:
        db_table = "..."
        verbose_name = "..."

-"Hm wait, you have exactly 8 categories?"
-yes

what do you mean types?

livid swift
#

basically for each model you have exactly 1 instance of that model?

stiff otter
#

i cant show a screenshot

#

basically each category can have multiple files in it

livid swift
# 
# pseudocode
class Category(models.Model):
    code = models.SmallIntegerField(choices=(1..8))
    name = ...
    year = models.IntegerField()
    from_field = models.DateField()
    to = models.DateField()
    object = models.CharField(max_length=100, blank=True)
    action_order = models.CharField(max_length=100,  blank=True)
    office = models.CharField(max_length=20, choices=offices, blank=True)
    security_level = models.CharField(max_length=20, choices=security_levels,  blank=True)
    
    def clean(self):
      if self.code == 2:
          if not self.office:
              raise ValidationError("Office must be filled for this category")
      # and so on
livid swift
#

So first idea is to have single category model then and have custom cleaning to ensure fields that are needed for each "type" of model (representated by hardcoded choice of code field here)

#

Another option that comes to mind is storing optional fields in JsonField

stiff otter
#

im not sure i understand tbh

livid swift
#

Which part? If you have only 1 row for each category breaking it on 8 tables looks not practical (I first was confusing you have like category types so you can have multiple Categoty3 for example)

#

So first thing to consider is dumping everything to single model. It might be not cleanest way, but cleaner than 8 tables

stiff otter
#

i think i need to clarify this
a category has fields which are the columns
but its row of that model is a file

#

should i think of it another way?

livid swift
#

Why row in category is a file?

#

and not row in FileModel?

stiff otter
#

so you say i have to do a file model that will have all the fields and make them available per category choice?

livid swift
#

hm, so you mean all fields you have in Category is actually a fields meant for Files?

stiff otter
#

yes i think

#

but depends on which category the file is going it will have different fields

livid swift
#

Ok, then Category is confusing name here

#

to store file fields

#

Ok, taking that in account I'll go for new iteration

stiff otter
#

basically files should be unique
but depending on the category they will have different fields

#

i have exactly 8 categories some fields are similar between them but also each category has a unique combination of fields

#

thats why i did all that category models

#

the fields that are the same i extend hem from an abstruct class

livid swift
# 
# pseucode included 
class FileModel(models.Model):
    name = models.CharField(max_length=200)
    upload = models.FileField(upload_to="files/")
    observation = models.TextField(max_length=500)
    datetime_added = models.DateTimeField(auto_now_add=True, editable=False)

    category = models.SmallIntegerField(choices=(1..8))

    year = models.IntegerField()
    from_field = models.DateField()
    to = models.DateField()

    # variant A - all blank fields on single model
    object = models.CharField(max_length=100, blank=True)
    action_order = models.CharField(max_length=100,  blank=True)
    office = models.CharField(max_length=20, choices=offices, blank=True)
    security_level = models.CharField(max_length=20, choices=security_levels,  blank=True)

    # variant B - all extra data on JsonField
    extra_data = models.JsonField()

    # variant C - one-to-one extensions:
    def get_extension_obj(self):
        extension_attr = f"fileext{self.category}"
        return getattr(self, extension_attr)

class FileExtBase(models.Model):
    class Meta:
        abstract = True
    parent = models.OneToOneField('FileModel')

class FileExt2(FileExtBase):
    office = models.CharField(max_length=20, choices=offices)
    security_level = models.CharField(max_length=20, choices=security_levels)
#

lower classes are also part of var. C

stiff otter
#

I have to do some doc reading to understand exactly how this would look like
Thanks for the answers but I have to go it's 2 in the morning here. I will check them again as soon as I wake up.

stiff otter
#

@livid swift this is the version i came up with, with some suggestions you made


class Rank(models.Model):
    name = models.CharField(max_length=50, choices=ranks_choices, unique=True)


class CorpArm(models.Model):
    name = models.CharField(max_length=50, choices=corps_arms_choices, unique=True)


class Office(models.Model):
    name = models.CharField(max_length=50, choices=offices_choices, unique=True)

class SecurityLevel(models.Model):
    name = models.CharField(max_length=50, choices=security_levels_choices, unique=True)


class MailType(models.Model):
    name = models.CharField(max_length=50, choices=mail_types_choices, unique=True)


class User(models.Model):
    user_name = models.CharField(max_length=50)
    surname = models.CharField(max_length=50)
    rank = models.ForeignKey(Rank, on_delete=models.SET_NULL, null=True)
    corp_arm = models.ForeignKey(CorpArm, on_delete=models.SET_NULL, null=True)
    office = models.ForeignKey(Office, on_delete=models.SET_NULL, null=True)


class BaseCategory(models.Model):
    file_name = models.CharField(max_length=200)
    file_upload = models.FileField(upload_to="files/")
    observations = models.TextField(max_length=500)
    date_added = models.DateTimeField(auto_now_add=True, editable=False)

    class Meta:
        abstract = True


class Category(models.Model):
    category_name = models.CharField(
        max_length=50, choices=categories_choices, unique=True
    )
    
    class Meta:
        verbose_name = "Κατηγορία"
# 

class CategoryItem(BaseCategory):
    category = models.ForeignKey(Category, on_delete=models.SET_NULL, null=True)
    aa = models.IntegerField()
    from_date = models.DateField(null=True, blank=True)
    to_date = models.DateField(null=True, blank=True)
    year = models.IntegerField(null=True, blank=True)
    object = models.CharField(max_length=100, null=True, blank=True)
    action_order = models.CharField(max_length=100, null=True, blank=True)
    office = models.ForeignKey(Office, on_delete=models.SET_NULL, null=True, blank=True)
    security_level = models.ForeignKey(
        SecurityLevel, on_delete=models.SET_NULL, null=True, blank=True
    )
    mail_type = models.ForeignKey(
        MailType, on_delete=models.SET_NULL, null=True, blank=True
    )

    class Meta:
        verbose_name = "Αντικείμενο Κατηγορίας"
        unique_together = ["aa", "category"]
#

the issue i see here is that depending on the category, the category item would be in there, would have a massive amount of blank fields

livid swift
#

I really dislike usage of Category* here unless I'm missing some context

#

only Category model here is actually a category

#

BaseCategory and CategoryItem are files or whatever is the object of categorization

#

And it's not clear to me if you need Category model here. I've suggested it to be a simple choice field. Ofc if Categories can be added or renamed model is right choice

#

Well, same question to other models that only have name choicefield

stiff otter
#
  1. as of now category model will have 8 categories as rows they are known but some1 can add more and so on but they are unique so i made this model
  2. i followed similar approach with other models as well rank mailtype etc

"BaseCategory and CategoryItem are files or whatever is the object of categorization"
they are files belonging in a category

#

the problem i have now with that implemetnation is that
in the categoryitem table there will be a lot of blank fields

for example i upload a file i choose category and i already know based on that category which fields are to be filled and which would not

#

so if i upload thousands of files per category you can see the pattern of the blank wasted cells

livid swift
#

they are files belonging in a category
Yes, but you named the "Category"
If you say have animals and animal kinds: Feline, Canine, etc,
Dog is AnimalItem of type Canine
Not KindItem. KindItem is "Canine"

#
  1. as of now category model will have 8 categories as rows they are known but some1 can add more and so on but they are unique so i made this model
    Then remove choices from name. Having both makes little sense, since you can't add more objects without changing code. And if you can change code you don't need model
stiff otter
#

honestly this is a side project im making so i can learn the techs im using. I dont know the entire potential of django models, and the docs are hard to read.
As of now you could characterize the project as a library or an archive or a file digitalization system that you have so you can add files in a database based on categories
also i need some info on users so i know who added what and when and where that user belongs to (office etc)

#

i use
django-drf mysql
gunicorn nginx
vite typescript react

so i will also make serializers etc so it can be a rest api

#

"Yes, but you named the "Category"
If you say have animals and animal kinds: Feline, Canine, etc,
Dog is AnimalItem of type Canine
Not KindItem. KindItem is "Canine""

so how would i go about doing it?

"Then remove choices from name. Having both makes little sense, since you can't add more objects without changing code. And if you can change code you don't need model"
each category of files has fixed fields
but some fields are in multiple categories
and categories canalso have unique fields

livid swift
#

so how would i go about doing it?
As I said name CategoryItem something that describes actual object
FileModel, FileItem, FileEntry
Then FileItem has a category of class Category - this makes perfect sense

#

each category of files has fixed fields
but some fields are in multiple categories
and categories canalso have unique fields
Yes, I know it's neither a simple nor rare problem

#

I proposed 3 ways to approach, neither is perfect

#

4th could be having Files of each Category a separate model all together. It solves issue of blank fields neatly, but then it becomes hard to manage files at mass

#

So it really down to consideration what you are going to do with Files

#

Some approaches makes one thing easier and another one harder

stiff otter
#

"
variant B - all extra data on JsonField
variant C - one-to-one extensions
"

I dont know what these are so i will have to do some research
As i understand it now there is not a single best way of doing that and/or I lack knowledge on db architecture (which is a for sure thing)

honestly lets say the data has to be easily queried normalized and properly categorized
I am guessing i can show or hide the fields in the frontend form when some1 will upload a file
but i also want some1 to search/sort/export/edit files and that should be optimal
maybe add some statistics per user and/or per file
also add permissions which file can be visible etc based on security clearance or office