woooee.
It is true that databases will ease access after creation. The time of loading say 20 billion records is however usually is prohibitive, especially if the data is volatile. My background is telecommunications, and huge files are coming in and out steadily, never stopping. The only way to process data of this type is serially, and requires special handling. Typically these files are captured at set intervals, sort of like putting a pitcher under a waterfall. fill the pitcher, process the data, get another pitcher, in a never ending chain.
So a relational database sounds like a great idea, but only works where the volume is small enough to process it in a set amount of time.
It is true that databases will ease access after creation. The time of loading say 20 billion records is however usually is prohibitive, especially if the data is volatile. My background is telecommunications, and huge files are coming in and out steadily, never stopping. The only way to process data of this type is serially, and requires special handling. Typically these files are captured at set intervals, sort of like putting a pitcher under a waterfall. fill the pitcher, process the data, get another pitcher, in a never ending chain.
So a relational database sounds like a great idea, but only works where the volume is small enough to process it in a set amount of time.