I had a rather lengthy email exchange about ISAM with a friend whose been around longer than me. Too much gets lost over time. What I found to be reprehensible were all of these so-called professors and other educators claiming B-tree and just about any other indexed file system was ISAM. The exchange was proof that nobody should ever use Gemini.
I was so upset with what I found online that I went to Thriftbooks and ordered ISBN 0-88236-11-2. That is the Shelly & Cashman COBOL book I used in Junior College. We had to spend over a week with paper, pencil, and chalkboard manually implementing ISAM. We had a PDP-11/70 running RSTS/E and it only had VSAM. I don’t remember where I found the featured image, but it was the only place online which documented ISAM correctly. When my new old book shows up I will scan the wonderful images that book had.
Why ISAM Happened
You need to see the following.
That is a removable disk pack in a beloved washing machine of the day. Despite all of the platters, you only had a few hundred MB of storage.
Initially we had paper tape and punched card. When we got to 2400 foot reels of magnetic tape we really had something for storage! Then we got these removable packs. IBM didn’t really have a Disk Operating System (or even subsystem). Instead it had the Hierarchical Storage Manager. From the 10,000 foot level, the primary responsibility of HSM was to stop an IBM mainframe from running out of storage. You had to have a minimum number of tape drives and “scratch” disks.
The HSM would migrate files currently not being used from DASD (Direct Access Storage Device) to tape or tape robot or Data Cell to free up disk space for a job. The wall of tape drives was a common thing.
All of this data shuffling kept computer operators hopping. DASD, what you now call a hard drive, was a shiny new thing. Life got difficult for HSM when you had DASD of different sizes. An ISAM file created on a big disk could only be restored from tape to a disk of the same specifications.
If it doesn’t have overflow it isn’t ISAM
We are going to look at the featured image again.
ISAM was mapped onto the hardware of DASD.
A Cylinder is some number of tracks in the same place on every platter. It very much is that thing from high school geometry you have such fond memories about.
A Track is a ring of sectors going all the way around a single platter. Yes, sectors change size as you move to the spindle in the center. Historically we wasted a lot of storage because nobody had figured out how to implement disk blocks allowing us to use all of the rust on the platter.
Today many would refer to a sector as a disk block, but it wasn’t. It was a slice of a track. They were different sizes in each track, but they were the same percentage of that track. That concession was required for sanity.
When you re-organized an ISAM file, some portion of the “records” or primary data area was left empty. The same was true for the index at the top. Inserting records was inefficient but the data had to be kept in sorted order. You could not span a track. If the record, or index, did not fit, it had to go to overflow.
There were a lot of keyed hits to identify the volume, then track where your data “should” be. The address was to the start of the track. You then had to sequentially process records on that track hoping to find yours. If it wasn’t found, then you had to sequentially walk through the overflow area.
This is how it got its name. Indexed hit to skip over a lot of shit, then a sequential hunt forward.
COBOL Did Not Have Indexed File Support
At least not initially. May companies like DEC and other midrange computer manufacturers came up with incredible (and some flubs) for indexed file support. This is why I had to study IBM 360/370 Assembly language when I made the mistake of attending DeVry. The only way to support ISAM was with Assembly language. You had to link the object file to your COBOL program.
The DEC indexed file system from the PDP days forward was basically VSAM, but IBM used the name. Files and different logical areas for indexes and data. There was no overflow area. A file could extend an area by a predetermined number of blocks depending on disk quotas and available disk space as much as needed. Extends for an index area slowed down your hits some.
VSAM
From the 10,000 foot level, every index contains with it the logical block address of the block containing that data. Things get a wee bit wonky if you allow block spanning. This can happen when you have very small records. A block is 512 bytes. If you only have 50-byte records you would be wasting a whole lot of disk storing one per block. You have to make the decision as to how much you value speed.
Make no mistake. On VMS, when you use a DEC language like COBOL, BASIC, FORTRAN, DIBOL, and some others, you do nothing special. You establish a key of reference, perform a keyed it with an index value and boom, the record shows up (or not.) How Files-11 did what it did you neither know nor care.
Too some extend VSAM on IBM is this was as well. I’m not a blue guy though.
IPP and IDCAMS
It shouldn’t take a rocket scientist to figure out IBM was locked into a bad situation. Yep, seemed like a great idea at the time, then disks got bigger, and bigger. Trouble with being the biggest computer vendor since the 1960s is your customer base has hundreds of millions of lines of COBOL code using ISAM. You can’t move your OS forward unless someone bites the bullet. In the end, it wasn’t the customers.
Even today you will see job listings online mentioning IIP and/or IDCAMS. This was the “fake-it-till-ya-make-it” solution IBM came up with. You now use IDCAMS to create indexed files. One type is KSDS. Feel free to look up what that acronym is for. It’s a flavor of VSAM that IIP uses so legacy ISAM programs don’t have to be modified . . . well, don’t have to be re-written.
Summary
All indexed file systems suffer from fragmentation if they actively have additions or deletions. Each operating system has some set of tools to determine how fragmented things are. While the tools differ, the solution is the same.
- Analyze indexed file creating whatever generation script is used on your platform.
- SORT indexed file to sequential.
- Nuke original indexed file.
- Edit generation script to add some size.
- Use script to convert sequential file to indexed using new sizes.
ISAM no longer exists. No B-tree was not ISAM. ISAM was tied to physical disk storage. As drives got bigger we moved to logical disk blocks. Intelligence got put out on the disk drive itself. The drive told the operating system how many disk blocks existed, number used and number available. Programs simply request a logical block, where it is on disk they do not care. ISAM no longer made sense. Today most indexed files are some form of VSAM, even B-tree. Nothing has an overflow area.
Really old PC users will remember having to run the DOS DEBUG command and manually enter the number of heads and cylinders for a disk drive into the BIOS. Eventually we got a BIOS you could see on the screen where we could enter that stuff. Then hard drives had to fake how many heads and cylinders they had. I seem to remember 255 heads and 16 tracks per cylinder was the biggest you could go due to BIOS INT 13H limits.
Don’t look that up. AI has it wrong too. I never had an 8GB hard drive under MS DOS. Used to have to split an 80MB drive into two 40MB drives due to size limitations.
