#Converting a PostgreSQL model to Loki

3 messages · Page 1 of 1 (latest)

tough pine
#

Currently, I am logging a lot of event into a model that looks like

class EventLog(models.Model):
    class Meta:
        ordering = ["-date"]

    team = models.ForeignKey("Team", null=True, on_delete=models.CASCADE)
    date = models.DateTimeField(auto_now_add=True, blank=True, null=True)
    user = models.ForeignKey("User", null=True, on_delete=models.SET_NULL)
    user2 = models.ForeignKey(
        "User",
        null=True,
        on_delete=models.SET_NULL,
        related_name="eventlog2_set",
    )
    device = models.ForeignKey("Device", null=True, on_delete=models.SET_NULL)
    message = models.ForeignKey(
        "Message", null=True, on_delete=models.SET_NULL
    )
    email = models.CharField(max_length=256, null=True)

It's kind of hard to manage such an amount of model, so we thought about migrating the data to Loki (https://grafana.com/oss/loki/).

Would it be worthy to spend time implementing Loki as a Django database backend ? Does anyone already done it ?

lusty helm
#

Does this model get updated or deleted? What kind of selects are you issuing? What's the table size? In what ways is this not working right now?

tough pine
#

No update.
Events older than a month are removed.
The table size is quite large (~200k events per day)
Right now, it's working, but filtering is sometime slow, and Loki is kind of a preferred way for storing logs (event are kind of log).