So, I'm building something like Disco Elysium or Planescape: Torment, and I'm currently refactoring the old dialogue graph system to make it simpler, clearer, and more organized. My current goal is to make the system as easy to understand and edit as possible, while preserving the original flexibility where possible.
One of the things I'm thinking about is the data structure. In the old system, all dialogues were stored in several JSON files that were populated in the editor and then parsed into dictionaries in the game. All elements, such as conditions or actions that a dialogue can trigger, are also JSON fields for a specific phrase, since each phrase must have both the conditions to be met and any possible actions it can trigger (such as setting flags or triggering world events).
In this case, resources are not used. However, I use resources heavily in all other parts of the game, and I'm wondering if using resources is an efficient way to store dialogue for a game with tens of thousands of dialogues.
My main concern is the number of objects. If we represent each dialogue as a set of phrases, and each phrase as a separate resource (for reuse, flexibility, and convenience), the total number of memory resources could exceed hundreds of thousands during long-term game maintenance. I don't know if this number of objects is a real problem (since I don't know much about memory).
I'm also not sure that any lazy loading option would work here, since, although I don't need all the dialogue at once, I might need access to individual phrases from individual characters or events at any time (the game has procedural generation so it's not much easy to predict). So just storing links-paths to dialogues also seems crazy, and I don’t quite understand...
Since I'm using CSV table keys, phrases actually store only their keys and not the entire text, but that would still be quite a lot of text. Or would it? What do you think?