That is a problem, I agree,
I constantly ran across this problem when working in the telecommunications industry
I was constantly complaining about the use of text fields for non textual data.
I couldn't justify the cost of conversion from one state to the other.
However, text is the native format for relational database columns or fields, so I
did understand where the government folks were coming from.
In my own data, I used indexed flat files for my own work.
The index was a hash into the actual data (which was always binary)
When it became necessary to deal with sorting text based files, the usual solution
was to pre-process the text file, creating a secondary file that contained two fields.
The first would be all of the sort keys in the order of precedence, and the second
being an offset into the original file (the physical location of the text record).
I would then sort the index file.
A copy of the original was made by reading the sorted records in sequence.
This is a very expensive and only temporary solution (as the input data must be frozen during the process).
The index is then discarded.
I constantly ran across this problem when working in the telecommunications industry
I was constantly complaining about the use of text fields for non textual data.
I couldn't justify the cost of conversion from one state to the other.
However, text is the native format for relational database columns or fields, so I
did understand where the government folks were coming from.
In my own data, I used indexed flat files for my own work.
The index was a hash into the actual data (which was always binary)
When it became necessary to deal with sorting text based files, the usual solution
was to pre-process the text file, creating a secondary file that contained two fields.
The first would be all of the sort keys in the order of precedence, and the second
being an offset into the original file (the physical location of the text record).
I would then sort the index file.
A copy of the original was made by reading the sorted records in sequence.
This is a very expensive and only temporary solution (as the input data must be frozen during the process).
The index is then discarded.