CopperSpice Experiments – Pt. 14

Why EDT?

The short answer is “Because I wanted it.”

The slightly longer answer is “I needed something of value to do while kicking the tires on CopperSpice. Adding themes and EDT keypad support to Diamond not only fit the bill but would benefit the world at large for as long as it could still compile.”

The much longer answer is the rest of this post.

I started out on the PDP 11/70 running RSTS/E. It was called a PDP because central buying policy by the US Government meant it was much more difficult to sell a “computer” than a “Programmable Data Processor”. Regrettably I can’t find an image on-line of the “RSTS/E is sexy” bumper sticker but I did find a scan of this history book covering Digital Equipment Corporation. Yes, an ancient machine, but as The Register reported in 2013, it will be running GE nuclear power plants to 2050.

https://forums.theregister.com/forum/all/2013/06/19/nuke_plants_to_keep_pdp11_until_2050/

I moved on to the VAX, then Alpha. As for editors, I used LSE (Language Sensitive Editor) when it was available, otherwise EDT. Both supported EDT keypad navigation. LSE just had a few more features. As long as we had terminals, it wasn’t a huge mental shift using a PC editor.

VT-320 terminal

The terminal was in a different spot with a different keyboard and had a visibly different screen. The hardship started when we used terminal emulators. Don’t get me wrong. Reflections was and still is a fantabulous terminal emulator.

Reflections version 7

It was just that now I was using keypad navigation in one window and it didn’t work in the editor on the other screen. Many many many VMS developers were infuriated by this. PC coders couldn’t understand. They had never walked in the light. Emacs (and by extension Jed) did a pretty decent job if and only if you modified your terminal definition so they could successfully map the NumLock key; otherwise it was even more infuriating.

For more than a decade now I have been off in the medical device/embedded systems world. I’ve not much thought about it. Then came a few calls about VMS work while I was writing The Minimum You Need to Know About the Phallus of AGILE. That got me waxing nostalgic for COBOL and I started writing a book on GnuCOBOL. That lead to the discovery that every PC editor, even VSCodium, well and truly sucks at COBOL. Emacs was the closest thing to a “useable” COBOL editor out there. That lead to the writing (out early next year) of The Minimum You Need to Know About GUI Emacs. When I was testing GnuCOBOL I created an account on Eisner so I could compare it to DEC/COMPAQ/HP COBOL.

Had remapping the terminal so Emacs could find the NumLock key not been a violation of my religion, this particular journey might not have happened, at least for me. Someone would have made a similar journey. The OpenVMS operating system is being ported to x86_64 hardware. These old conversations are coming up again in comp.os.vms. Some people have made their peace with Emacs. Others are lamenting the lack of a modern EDT compatible editor.

We are talking about people with thirty to fifty plus years of software development under their belts. None of them would start writing an editor from scratch. Some of them would gladly pitch in once something got reasonably close. There are thousands of retired and semi-retired OpenVMS developers out there who are more than a bit bored during COVID-19. One has been bored enough that they are solving all of the C programming problems for some programming site using COBOL on a PDP for no other reason that to have something to do with a keyboard.

I wrote this for me but I hope to leave it for them.

EDT Keypad Help

When you have EDT emulation on and hit the [HELP] key you will get a big html written help screen in a browser window. The first image on it is this one. It then explains what is and isn’t implemented and how to use stuff. The real problem with the PC isn’t just the mentality of that universe, but the origins behind the keyboard.

Midrange and mainframe systems were purchased by businesses. During the 1960s through the mid 1980s much of the “work” done on these systems revolved around data entry. When the numeric keypad shown on the left side of the above image was designed it was designed by Efficiency Experts to reduce both strain and errors. We measured the efficiency of data entry screens by how often a user would have to move from “home row.”

Home Row

Yes, even on character cell terminals we created drop down boxes for entry. A typist tabbed to the field and started typing. The displayed list would update with each keystroke showing only the entries that began with what they typed. When it got down to a single entry, it either auto selected that or there some something like [Return] they could hit to select it.

Today’s PC user interfaces throw efficiency out of a twelve story window in front of a fully loaded cement truck. They keep forcing the user to reach for the mouse. Combo boxes for many libraries/platforms don’t even allow a user to type in them. We are seeing some minor improvements in Web forms, at least when entering your state. When I type “i” for Illinois some actually scroll to the part of the list starting with “I” but far too many don’t.

Please take a moment and think about just how shitty the user interface is of your favorite editor when viewed for data entry efficiency. To select text you hit <Ctrl>-b or some key combination to “begin block” then you reach over to the arrow keys to navigate around, then you come back to home row to indicate “end of block” usually via some control key combination to copy or delete. “Oh, I use the mouse!” Well have a good long thought about that.

Movement from home row was always supposed to be an oddity or a limited event. That’s why when you look at old time data entry screens you see one of two things:

  1. All of the text fields on the left side of the screen and all of the numeric fields on the right. Entry proceeds down the left side then down the right.
  2. All text fields on the first screen and all numeric fields on the second screen.

When it came to inventory or other lookup type items that could not be just numbers, the letters used were always left handed letters, leaving the right hand over the keypad. The first person to suggest part number P1234 was fired.

Data entry needed a period, comma, and minus key for numeric values because not all forms formatted. The applications needed a few “special keys” so we got Programmable Function keys PF1 through PF4. EDT made use of those. The application told the terminal the keypad was in application mode and the terminal complied.

Standard PC numeric keypad

When PCs were “sold to business” they were initially sold to the accounting department. They were the only ones who could justify spending north of $5,000 to do spreadsheets. Spreadsheet users needed math keys. They basically needed a calculator keypad with a “Total” key they allowed someone to call “Enter.” There was great animosity between the PC world and the world of real computers. They absolutely refused to utilize the DEC keyboard despite it being the Unix standard. As a result you are missing a key. EDT users usually have to make some half-assed accommodation for this. The WORD key usually gets mapped as <ALT><KeyPad+> using the left <ALT> key.

Actual EDT help screen

Take a good look at the EDT keypad help again and consider selecting some text. (This is a shot of a Putty terminal actually logged into Eisner.) Forget about the left side, just look at the keypad. What are the steps to select text?

  1. First you have to navigate to the text you want.
  2. Start a selection.
  3. Navigate to end of text you want.
  4. Do something with the text.

EDT navigation was direction sensitive. It defaulted to ADVANCE but could be toggled between ADVANCE and BACKUP via the 4 and 5 keys on the keypad. Navigating a CHAR, LINE, Word, SECTion, or PAGE honored the current direction. Even EOL (End of Line) honored it as did DELete Line and UNDelete Line.

Prior to the PC, all of your navigation and basic editing was done with one hand. On the PC, if you wish to perform Word operations, you will need two hands, but the bulk of your operations are performed with one hand centered over the keypad.

If you needed text from various parts of various programs to be pasted into the program you are working on now, you simply opened those files up in other buffers and selected the text you needed. The APPEND would append your currently selected text to whatever was in the paste buffer (clipboard for you PC types.) Yes, I actually implemented that. If you want a rundown on the features I’ve been working on you can read it here. I had to add a clipboard viewer just so I could test this feature.

Clipboard contents

Once I have Diamond to where it is “good enough” I will expose it to my fellow EDT/VMS enthusiasts. At the very least there will be a few hundred who “give it a whirl.” These are all very seasoned developers. One or two joining the Diamond (assuming this code gets accepted) project would allow for serious enhancements. I can tell you right now some of the things that would be very high on their list of wants/priorities.

  1. Ability to support both ANSI-COBOL and DEC-COBOL-85. The current hard coding of comments in the syntax highlighting make this impossible. DEC-COBOL-85 was both free format and the comment indicator could be anywhere on the line. Haven’t dug deep into it for a while, but I seem to remember it also allowed for the “!” to be a comment-to-end-of-line indicator. ANSI-COBOL indicated a comment by having a “*” in column 7.
  2. FORTRAN-IV, 77, and 90 support. Again this spans the standards from fixed position card format to freeform coding. It also has an array of comment indicators.
  3. VAX BASIC. Still a rather large code base. Depending on the year written either had line numbers or did not. “REM” is the ANSI commend indicator. “!” was the DEC extension and it could be anywhere on the line. Hopefully they don’t want to deal with BASIC/PLUS from the PDP cross over era because that adds many parsing issues including “&” in column 78 (or 80) being a LINE CONTINUATION character.
  4. Bracket matching. I may actually get to this one on my own because I want it. Found some code that claims to do it, I just haven’t had time.
  5. Numeric constant highlighting. All of the regular expressions I have found don’t work when loaded from the syntax file. It’s something stupid and some other set of eyes will instantly see it.

I’m pretty sure you will see BLISS and a bunch of other languages pop up as well. Some of the people from VSI might contribute time (as if they have any spare time) as well. They would be the most in need at the moment as they are doing the OS port.

So, to sum it all up, myself and a bunch of others have suffered no end of frustration at the inefficient “standard” PC navigation. During these years when our hairs are turning either gray or white we want to see a modern editor with EDT navigation. I wanted to see it enough to do it. Like my brothers and sisters from that platform, I didn’t want to start from scratch.

Besides, the PC world needs to learn. Most of the shit they shun from the real computer world exists for a reason.

That’s why I added EDT keypad support. It’s not complete, but it is mostly there.

<Previous-part Next-part>

CopperSpice Experiments – Pt. 9

Diamond text editor now has many new features, including backups. Here is a run-down.

Yeah, it’s been a while. Life keeps getting in the way of actually writing something here. I’m also migrating from BlueHost.com through HostGator.com to inMotion hosting service. That gets its own blog post once complete. Despite all of that, a lot of code has changed in the repo. Just be sure you choose the diamond-themes branch.

Themes work really well. Not perfect, but really well. Perfect would be instant coloring. I can seemingly get that to happen even with large source files if the syntax is set on a tab before the file is loaded. It appears the QPlainTextEdit class is smart enough to only paint the visible rows first. The problem is I haven’t unwound the MainWindow class object creation far enough to stop a segment fault crash with autoload enabled. If I set the syntax before setting the document in the tab during program load there is some “resource not yet available” segment fault death that happens. I’ve sped things up as fast as I can for right now without causing that problem.

So, let’s review my original ideas:

  1. Themes
  2. A decent default font
  3. EDT numeric keypad emulation
  4. Compiling
  5. COBOL syntax

The first three are basically done. Part of me wants to add custom background and foreground colors for selected text; but living with default white text on blue background for now.

Colors dialog

You can import/export and even change color themes. There are even an acceptable number of “default” themes built in.

Built in Themes

Sorry I had to use my phone for that shot. Have I mentioned the screen shot utility built into Ubuntu 20.04 LTS completely sucks? Eventually someone will set up a site where all of you dear people can upload your personal favorite themes along with screen shots. It should encourage wider use of the editor. Everyone wants their own personal theme.

Do you think I spent all of this time just sitting here going “oooh look at the colors?” Silly Wabbit!

Even more options

As part of improving overall performance, Overlord now holds all of the syntax highlighting. You have the option of preloading your favorite. Whenever an editor needs a syntax it asks Overlord for it. We only bite the bullet loading the JSON once.

There are three new check boxes on the right. One for enabling EDT. Another for Astyle on Save and finally one for to enabled backup versions. You will notice I have a field to enter the Maximum number of backup versions to keep and allow you to choose a directory other than the default for said backups.

I’ve used a lot of editors over the years. Far too many editors on wanna-be-a-real-computer-some-day x86 based platforms just create one backup with a ~ or # or something like that at the end of the name in the exact same directory as the original file. This “one” backup tends to be created either the moment you open the file for editing or just in front of your last save. If you are a save-early-save-often type developer these backups are Fahrvergnugen useless. Adding insult to injury, on any brand new project you always end up with half a dozen of these files getting into your source control until someone takes the time to both delete them from source control and create an ignore file.

Backups and file versioning

I come from an OpenVMS background. FILES-11 Records Management System had file versioning built in. Every time you saved a file it created a new version. You could have versions from 1-32767. You could PURGE all older versions and rename the current version back to 1 to start the numbering process all over again. In short, on a real platform, it is very difficult to “lose” something. We had this long before we had source control systems. When you combine file versioning with source control, it takes a very deliberate and stupid thing to “lose” a file.

We didn’t have to create utilities to manage these versions either. The PURGE command had a /KEEP=nn qualifier. If you only wanted the last 6 versions of any file on a backup tape, you just purged down the entire drive with /KEEP=6 before running the backup.

Some PC based editors try to implement the central backup directory. There are varying levels of success with it. I like what Emacs did with their backups. They mungify the full path and stick a version on the end. This keeps main.cpp from Project A from pushing main.cpp from Project B backups off the end of your keep chain. I kinda ripped that off but changed it a bit.

View Backups

When you have a file open in the current tab you can choose View->Backups and

All backups for that file

Yeah, the names don’t all appear the same here because I was doing battle with just how to do this. Eventually those will purge off or I will just recreate the directory.

Backups became very important to me while I was working on the Astyle interface. Let’s just say there were a few zero length styles that wiped out my buffer and the file on disk which of course hadn’t been checked into Git for hours. When you open a backup file you should note the tiny change to the status line.

Read-Only

Ordinary files will have Write listed where ReadOnly is.

Ordinary file opened for Write

There is one issue I haven’t decided about fixing. There is code in Diamond to always set a last directory value based on the file you just opened. When you open a backup file that gets set to your backup directory. I haven’t decided if it is worth blocking that in the code or not. I found it annoying when getting all of this working. Now that it is working I have to wonder just how many times one will open a backup file and shouldn’t I make it a wee bit painful? You are opening the backup because you did something stupid. Shouldn’t there be a bit of a price to re-enforce you learning not to do that again?

EDT Options

I cannot make this work the way I want without making changes to QKeySequence and then creating CsPlainTextEdit from the QPlainTextEdit source code and getting that accepted into the project. It is too big of a chunk for right now. I’m not proud of the code I had to write to get EDT functionality into this editor, but it does make a quite usable editor. It’s not perfect. There are a few minor issues.

You will note I had to put a radio button box for Delete Word. The built in help text tied to the HELP key on the keypad kind of explains this. QKeySequence really only deals with strings. Ctrl+, Alt+, DEL is a different key sequence than Alt+, Ctrl+, DEL. QKeyEvent deals with a binary modifier block and a binary key value. With QKeySequence order is important but QKeyEvent simply doesn’t care. I have to fix that little piece before rewriting plain text edit so it can support editor states. Putting it bluntly, those of you with a “proper” keyboard won’t be able to use this right now.

Wyse ‘transtec branded’ DEC LK401 layout keyboard

Honestly, even if I did have this set up for you to enter a key sequence, I don’t think Qt has mapping values for a proper keypad. I know you can make it work in Emacs if you mod the terminal. Modifying the terminal definition is against my religion though. I just used the Scroll Lock and cussed about it every time I hit the wrong key. Until I had to log back into a proper computer and edit source code I had almost adapted. All it took was about twenty minutes on there and two decades of muscle memory returned. I was completely useless in Emacs because I kept hitting NumLock for GOLD. That’s pretty much why you are getting this editor. My level of pain got too high.

All of those non-standard GOLD key functions on the right must be tied to a single keystroke sans any modifiers. I really liked GOLD = for goto line in Emacs so I ripped that off. You will notice the last one is GOLD A for Astyling a buffer.

All of this functionality is checked in if you wish to pull the branch down and build it to try things out for yourself. I’m currently working at adding a Debian build procedure to the source base so I can install this on multiple machines without having to local build Copperspice followed by Diamond.

Hopefully I will have the energy to document my attempts at getting Copperspice to build on OpenSuSE Leap 15.2. So far it has not gone well. Of the RPM based distros, OpenSuSE is the one I find least offensive so that is the one I use when I have to use an RPM.

Oh! You probably noticed the Clipboard option. I needed a method of validating the EDT “append to paste buffer” functionality so the clipboard viewer got added.

Clipboard contents

There is no GOLD-7 [command] implementation in this. Yes, back in the day I used it a lot. Some of the functions could be useful today, but I had to draw the line somewhere. The GOLD-ENTER [subs] function isn’t implemented either. When you hit it you see the advanced search and replace dialog. Honestly, I had to look up what that key combination actually did. I had never used it in all my years on EDT. It was always soooo much easier to hit GOLD-7 and type

SUBST!OLD-STRING!NEW-STRING!QUERY

With the SUBSTITUTE command I also had the options of REST, WHOLE, SEL instead of QUERY. If I was ever going to implement the command window it would be to bring in this functionality, but I have more important priorities right now.

As long as I’m talking about “not done” I should mention FILL isn’t done. I have no plans. I remember using it pre-wordwrap days typing up documentation. This editor has word wrap and spell check. I realize the critical difference was FILL was a hard wrap. A newline character was physically placed at the end of each line. This was for various things like printers and Usenet newsgroups with fixed column limits. If someone really needs that today I guess they can join the project and add it in.

My personal view is that creating a Debian of the current functionality is more important than any additional features at this point. I dabbled with OpenSuSE contemplating an RPM as well. I’ve always managed to duck out of RPM creation, not so much when it comes to DEB creation. When you only do it once every few years it always takes a while to get back into.

I don’t enjoy doing it. Well, that’s not actually true. I always get asked to do the convoluted sh*t.

We need to create a package that is going to create a user account and install all of this in to run like a Kiosk. It will also include a tool to completely remove the installation.

Typical client request

I’ve never created a need to know nothing straight forward hard coded install directory tree DEB. This is completely uncharted territory for me. I may actually have completely empty pre and post install files.

<Previous-part Next-part>