Post by chriscrawford on Sept 15, 2015 7:22:04 GMT -8
I have been studying Facundo's LogLizard code, and I am dismayed by its complexity. He set it up as an extension of the Java TreeLogger class, which is designed specifically for this kind of problem. This sets up the log as a huge tree, allowing the data to be organized in a nice neat structure that is easy to navigate so that the user can zero in on the problem.
Unfortunately, that tree structure is really complicated! Especially difficult is the fact that Facundo added a special facility to save memory: some nodes in the tree are left incomplete, but the code can regenerate nodes from the engine as necessary. In other words, Facundo stores only a skeleton of the nodes, and then fleshes out the skeleton only as required by the user. It's a brilliant scheme, but utterly beyond my capacity to understand.
At this point, it appears that my best strategy is to simply throw away all of Facundo's brilliant code and build a much dumber system that simply saves everything. When we first came up with the idea for LogLizard, we expected that the data structure would be too big to store in memory, but that was back in 2007 when memory wasn't quite as large as it is nowadays. A few megabytes of text won't be a problem.
However, I'll still need some kind of tree structure to organize the data. Here's what the output of LogLizard looks like:
Note how the user can expand or contract any of the nodes to see what it contains. This is a critical feature of LogLizard and must be retained. That means that I'll need to use a tree structure, and it seems likely that I'll want to use the same TreeLogger class from Java. So now it's time to study the TreeLogger class.
Unfortunately, that tree structure is really complicated! Especially difficult is the fact that Facundo added a special facility to save memory: some nodes in the tree are left incomplete, but the code can regenerate nodes from the engine as necessary. In other words, Facundo stores only a skeleton of the nodes, and then fleshes out the skeleton only as required by the user. It's a brilliant scheme, but utterly beyond my capacity to understand.
At this point, it appears that my best strategy is to simply throw away all of Facundo's brilliant code and build a much dumber system that simply saves everything. When we first came up with the idea for LogLizard, we expected that the data structure would be too big to store in memory, but that was back in 2007 when memory wasn't quite as large as it is nowadays. A few megabytes of text won't be a problem.
However, I'll still need some kind of tree structure to organize the data. Here's what the output of LogLizard looks like:
Note how the user can expand or contract any of the nodes to see what it contains. This is a critical feature of LogLizard and must be retained. That means that I'll need to use a tree structure, and it seems likely that I'll want to use the same TreeLogger class from Java. So now it's time to study the TreeLogger class.