CopperSpice Experiments – Pt. 18 – The Pull Request

It has been a very long time coming, life kept getting in the way here, but my Diamond editor changes are ready for pull. For those who don’t give a rat’s behind about the pull request, here are the new packages. These are not official. I have tested these Diamond editor packages in bare installs of said operating systems. For Ubuntu installs you need to use the Software Installer, not dpkg from the command line because dpkg won’t install dependencies.

Note: I don’t remember if I fixed it or not. If you have an existing diamond config, sometimes the only theme you get is the one you had. Just rename your config file before installing. There aren’t that many settings.

Packages

45 Downloads

46 Downloads

64 Downloads

Package Creation

Yes, we now have both RPM and Debian creation.

shell scripts
Shell scripts

build-diamond-deb.sh checks for a Debian based distro then does everything necessary for a full build of source to create a Debian package. One of the things it does up front is run Artistic Style on the src directory.

artistic style control file
Artistic Style control file

When the shell scripts run they use this configuration file. The same configuration file is run when you choose to execute Astyle from within Diamond if you are editing in the src (or any sub-directory.) The Artistic Style command looks first in your local directory then in the project root and I forget where else.

If multiple developers will work in a code base a method of enforcing common style so things are readable is needed. Artistic Style works well in most of the shops I’m at.

The astyle-diamond.sh is really just a helper.

#!/bin/sh
#
astyle -n src/*.cpp src/*.h
#

build-diamond-rpm.sh does the same thing for RPM creation. It checks for a RedHat based distro and checks to ensure rpmdevtools has been installed.

Both packaging scripts dump the diamond tree into /opt/diamond. The .desktop file is copied to the appropriate location and a symbolic link is created in the appropriate directory so one can type diamond at the command line and have it run no matter what directory the user is in.

The third digit of a release really should no longer be allowed to be a zero. It has to do with RPM packaging viewing that as the “build number.” Most counting systems don’t start at zero.

Summary of changes

The summary at this post is still pretty accurate. Life has just been in the way and I have been focused on getting the packing done. Probably best to install one of the packages to kick the tires.

For “Why EDT?” Read this post.

For “Why Themes?” Read this post. If you don’t want to read the entire post I will include the ending paragraph here.

Do you know why Betty Crocker cake mix crushed the competition despite there being dozens of other brands on the shelf?

This was the days when women actually called themselves a “House Wife” and were proud of it. All of those other cake mixes were “instant.” You just added water and put them in the oven. With Betty Crocker you added an egg. Adding an egg crossed the psychological barrier and the conscientious house wife could honestly say “Yes, I baked it myself.” Adding an egg made it not instant and therefore okay. Instant is what the bad wives did and you would never find them featured in Good Housekeeping.

Putting in place the groundwork for community themes is the text editor equivalent of “adding an egg.” Allowing users to create plug-ins is the text editor equivalent of allowing too many cooks into the kitchen. That’s a condition Emacs suffers quite badly from now. Most of the plug-ins conflict horribly or cause random oddities in unrelated plug-ins. With every plug-in you install it is like tap dancing in a mine field.

Some of the other things off the top of my head:

  • Data needed by more than one class/object is now in a singleton named Overlord. Settings, options, themes, syntax highlighting, etc. are all in Overlord.
  • Default font is now Monospace. It will still do Courier if it cannot find Monospace. I forget the rest of the fallback patterns. Monospace is everywhere in a post-Java world. It also tends to be far more legible than Courier.
  • Can now preload syntax highlight files in Overlord. Editor instances no longer load themes directly. They reference them from overlord so the editor isn’t continually parsing JSON. On slow I/O machines that is significant overhead.
  • Lots of source code consolidation. One Class One Source File has been introduced into the code base so methods could be easily found.
  • Astyle on Save can now be turned on. One can also Astyle the document they are editing. All of this assumes you actually have Astyle installed on the machine.
  • Backup versions with a central backup directory like Emacs using Emacs style naming. Unlike Emacs we can also browse backup versions and open them in ReadOnly mode. Options allows you to enable backups and to set the maximum number for each file. Backups are created on each and every save. Older backup files purged off.
What backups look like in dialog
  • F12 can send Backspace just like a real VT terminal
  • One of the many benefits coming from EDT is that we can now append many cut/copy segments into the system clipboard. To be able to verify this I added a clipboard viewer as well.
  • All dictionaries are now deployed with package/install. Dictionary dialog as well as handling still needs a bit of re-architecting. It should just be keeping a reference to the primary language dictionary, not a copy of it. Because of the way it works now you still can’t “switch” dictionaries without manually copying them into the .config/diamond directory.

Developer documentation

A completely new directory tree within the project named dev_doc. It is a place to store RTF formatted developer documentation as well as hand holding tutorials for noobs and non-developers alike. Files “Getting-Started.rtf” and “Rules-and-Goals.rtf” are living documents that will evolve as the project evolves. “Getting-Started.rtf” includes many ways for noobs and non-developers to contribute in a very meaningful way right out of the gate. They don’t have to know a single thing about C++ or CopperSpice.

The OS_setup directory includes step-by-step instructions for setting up both development and test virtual machines under Oracle Virtualbox. This makes the project far more accessible to non-experts and an OpenSource editor should never be an expert-friendly project. There are far too many of those projects already.

Probably forgot hundreds of other little things. Now is the time to see if this was going to be rolled into the base Diamond project or if it would have to be forked into its own project. The instructions I have for snap creation were rather repository specific. Did not want to go down that rabbit hole if things were going to need a different project name.

No matter which way this goes, the next set of changes will make these look small. Need to have a solid baseline either as part of Diamond or a new project before doing them.

<Previous-part

CopperSpice Experiments – Pt. 16

Now we must create a minimal GUI “Hello World” to use testing out the theory. It will also be needed as a bug report test case if this turns out to be a bug rather than “should have had this switch on when building CopperSpice.”

In a terminal I did the following:

mkdir cs_gui_hello
mkdir cs_gui_hello/src
mkdir cs_gui_hello/resources
mkdir cs_gui_hello_debug
mkdir cs_gui_hello_build
cp diamond/resources/*.gif cs_gui_hello/resources

cd cs_gui_hello

roland@roland-amd-desktop:~/Projects/cs_gui_hello$ cp ../cs_hello/*.txt .
roland@roland-amd-desktop:~/Projects/cs_gui_hello$ cp ../cs_hello/*.md .
roland@roland-amd-desktop:~/Projects/cs_gui_hello$ 

In both CMakeLists.txt files I changed all cs_hello to cs_gui_hello and changed all Cs_Hello to Cs_GUI_Hello. I also updated the README.md. Then I renamed the build_info input file to: cs_hello_build_info.h.in and changed the important part of it to read:

#ifndef CS_GUI_HELLO_BUILD_INFO_H_IN
#define CS_GUI_HELLO_BUILD_INFO_H_IN

// Cs_GUI_Hello Version  "x.y.z"
constexpr const char *versionString = "@PACKAGE_VERSION@";

// Cs_GUI_Hello Build Date "08/17/2017"
constexpr const char *buildDate = "@BUILD_DATE@";

#endif

The copy of the *.gif earlier was just to get the animated gif files to test with. Since I really needed only one file I manually created cs_gui_hello.qrc in a text editor.

<RCC>
    <qresource prefix="/animations">
        <file>resources/spinning-red-diamond-4.gif</file>
    </qresource>
</RCC>

That means there was one other change in the CMakeLists.txt file in the src directory. The following lines had to be uncommented and a new source file has to be added.

Have to compile qrc to source and add it to source list

Now we open a terminal and test the assumption.

cd ~/Projects/cs_gui_hello_build

cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=../cs_gui_hello_debug -DCMAKE_PREFIX_PATH=/usr/lib/cs_lib/lib/cmake/CopperSpice ../cs_gui_hello

ninja
ninja install

valgrind --tool=memcheck --trace-children=yes --leak-check=full --show-leak-kinds=all --log-file=/home/roland/hello.log ./../cs_gui_hello_debug/cs_gui_hello
What I see in CopperSpice

No, I didn’t run it in valgrind the first few times. I did towards the end because I wanted to point out a bad thing in the code. It is code like I see in many many places on the Internet. You are in a hurry and bang something out fast then create this kind of self replicating screw-up as many many people find and re-use your stuff because it “seems to work.”

The code

Yes, I deliberately pasted an image so you couldn’t just scrape the code and go! I need to paste a few more images so you have a frame of reference. I went to the bottom of the log file and searched backwards for “lost.”

memory leak one
memory leak two
memory leak 3
Repeating memory leak

Hopefully you can zoom in on the images to see the 10+K leak in the first image and the 700+ byte leak in the second followed by a pair of 500+ byte leaks. What is interesting about the last 512 byte leak image is that it repeats for every disk block read for the gif.

All of this happened because of line 19 (or so I thought). QMovie doesn’t provide a constructor with a non-optional parent pointer as the first parameter. If you want to create in one line much like I did at line 19 you have to provide an empty QByteArray then the parent.

Now we get to why I pointed out the qDebug() statement in the last post.

no supported formats

There are no supported formats. I am hoping there is just a switch needed for compilation of CopperSpice that was missing when I built. It would be odd for the documentation to be way ahead of the implementation.

I changed the code to see if parentage would fix the leak because you can’t really do much cleanup when the source ends with:

return a.exec();

I have had to do cleanup over the years. You end up coding something like:

int retVal = a.exec();
/*
 *  Do all of of your cleanup paying special attention
 * to the fact you no longer have an event loop so much
 * of the Qt cleanup won't work because a surprising amount
 * queues an event.
 */
return retVal;

I changed the code to read as follows:

#include <QApplication>
#include <QMainWindow>
#include <QLabel>
#include <QVBoxLayout>
#include <QMovie>
#include <QByteArray>


int main( int argc, char *argv[] )
{
    QApplication a( argc, argv );

    QMainWindow mw;

    QLabel *hello = new QLabel( "Hello World!" );

    qDebug() << "supported formats: " << QMovie::supportedFormats();

    QLabel *movieLabel = new QLabel();
    QByteArray emptyFormat;
    movieLabel->setMovie( new QMovie( "://animations/spinning-red-diamond-4.gif", empty_format, movieLabel ) );

    QVBoxLayout *layout = new QVBoxLayout();

    layout->addWidget( hello );
    layout->addWidget( movieLabel );

    QWidget *w = new QWidget();

    w->setLayout( layout );
    mw.setCentralWidget( w );

    mw.show();

    return a.exec();
}

Not only did I have all of the same leakage, there was new leakage for the QByteArray. This is the problem with these quick little test programs. Just a few lines of code gives you a whole lot of leakage.

Few developers take the time to actually run valgrind or any other memory checking tool. Using tools like this in an AGILE world is something that is “never in the sprint.”

To be fair, I went over to the other machine and ran valgrind on that little demo. There were lots of little leaks that seemed to all be in some Qt internal thing. Nothing I could really “fix.”

valgrind --tool=memcheck --trace-children=yes --leak-check=full --show-leak-kinds=all --log-file=/home/roland/hello.log ./gui-hello

The ideal situation is for your application, no matter how small, to never have the word “lost” on any of the blocks in your valgrind report. You won’t be able to get rid of “still reachable” without severely impacting performance. This generally happens because the compiler tool chain has its own memory management library helping new/malloc out. Instead of allocating RAM from the system it goes and gets a big chunk then doles it out per each request your application makes.

Sometimes there is even a switch you can add to the build command line to set the size. If you’ve done some stats or simply wrote a hokey little “free memory” slot hooked to a timer so you “know” about how much free memory their is during your test runs, you can set the initial allocation to be

(TOTAL_MEMORY - Smallest_Free_Memory - Estimated_Binary_Load_Size) 

This will gain you a bit of performance. On a desktop running on grid power with high quality RAM chips, you won’t gain a lot. On an embedded system using the lowest power RAM one could find to conserve battery life, you can breath new life into the application by finding and using such a setting. Dynamic allocation on battery powered embedded targets typically sucks. This is especially true when the system has no GPU for some reason.

mainwindow.h
mainwindow.cpp

So I created a MainWindow class and added it to my little gui hello test application. Yes, you see those deletes commented out. I ran the application with those deletes in place fully expecting a crash. A layout is supposed to become the parent of all non-parented things directly added to it. I should have died with a double delete error message but I didn’t.

I honestly thought deleting m_widget would delete m_layout. It should have in my mind. My first run at this deleted only m_widget and I still had the big leaks from the movie. Adding deletion of the layout got rid of the big leaks the movie was previously creating.

Leakage summary

Leaking 4K doesn’t sound bad until you realize the program only ran for a few seconds. If this was a medical device where you had 512MEG or less, I would be worried if the leakage increased over time. Something left on long enough leaking 3-4K every few seconds will rather quickly run out of RAM.

Most of this stuff seems to be things I can do very little about.

XCB leakage

I know the developers of CopperSpice are currently working on XCB things as they recently released some code for XCB. Probably haven’t gotten around to running valgrind on things.

QAccessible loss one
QAccessible loss two

Part of me distinctly remembers QAccessible leakage in Qt 4.x . The other part of me worries about the QString8 I see there because that is new with CopperSpice. The 697 indirect loss repeated quite a few times in the file. A large part of that may well be tied to this 504 byte indirect loss.

It is these sub-1K leaks that really bite you. People ignore them because they aren’t large. What people fail to realize is that they tend to happen often. I’m not picking on just these leaks. I’m talking about sub-1K leaks in every project with every tool set. It’s the end of the project and you have already run out of time so they just get ignored. Now you fail your ten day run-time test because your sub-1K leak happens often enough to consume all RAM.

fontconfig leak

The fontconfig leak most likely has to be kicked upstream to the libexpat people. XML libraries and tools are notorious for leakage. Honestly I’m surprised this one is so small.

Why am I harping on memory leaks? Because I’m currently working on an editor. I don’t know about you, but on Linux machines I’ve been known to leave an editor up for over a week. I mean I save everything before I go home, but many times I never shut down. I want the cursor to be right where I left it in every file when I come in the next day. Sometimes I get called away and the thing is up for weeks on its own. Even on my machines with 16Gig or 24Gig of RAM just how long do you think it can last if it has a leak, especially in a background thread?

<Previous-part Next-part>

CopperSpice Experiments – Pt. 15

I do apologize for the burstiness of these posts and sometimes skipping around. I don’t have much free time so I write these posts when something tends to piss me off or befuddle me. Today it was an animated gif.

I got that gif from Icons8. Yes, a site generating nice simple “loading” and/or “busy” animated gifs that are free to use and even has a little tool on the site that allows you to customize them before downloading is getting a plug. Just how cool is that?

So, I went the “standard” path of stuffing a QMovie into a QLabel like we’ve done for years in Qt. No love. Just a black box. This is when it hit me I was going to have to create a cs_gui_hello application. First I needed to verify I wasn’t suffering from old age and the effects of years of Chardonnay at night. I hop over to a Ubuntu 20.04 LTS machine and load whatever QtCreator was installed on it using whatever Qt version that it found.

Yes, many of you would just have a VM if you had that. I have multiple machines with multiple desks and some of the machines have multiple VMs. It’s how I roll. Honestly, it is because that is the way work comes in.

The tiger I bought when I was in college, deal with it. Whatever type of photography was in vogue then had a market and this looked cool. This is actually cleaned up office. Over the past year or so I’ve thrown out about four machines or so.

When clients want a custom Debian package that will install on N releases of Ubuntu but they are using high end features of library X so it can’t go in a Snap or any other low end container, you have to throw hardware at it. Finding the right combination of options and libraries to thread that needle can take weeks. It was even more fun when they wanted a .deb that would install on either 32-bit or 64-bit versions. Been there, done that, got the T-shirt.

So, first I need a GUI “Hello World” to verify what I believe to be true. After starting QtCreator I tell it to create a new project for me, Qt Widgets based. I put a File->Exit option on the main menu and endure the incredibly slow signal-slot editor connecting exit up to MainWindow::close(). I add two labels, one with “Hello World!” text and another big empty label I called movie_label.

Labels in the designer

Okay, I took the screenshot before I filled the menu in.

Took longer for this editor than the entire rest of the project

Adding a Resource File

This part is for the new people. Maybe you’ve only done QML so you don’t know Qt at all? Maybe you are new to the world of programming or just new to the world of Qt. I’m taking you down this detailed journey because even I had to poke a tiny bit. As more and more worthless (&()&* gets added to QtCreator (QML, Java, Python, etc.) it becomes harder and harder to find things.

Right Click and choose add new

I usually right click on the “Forms” in the window on the right. Theoretically you can right click on any of those entries for this. That wasn’t always the case or at least I remember a time that it wouldn’t work everywhere.

Resource file is underneath Qt
Add it to your project
After adding a prefix

I never like the default prefix. Half of the time I just enter “/” but this time I chose “/animations.” After that I choose “Add Files” then navigated to a series of gif files with different colors and speeds.

Save and close the resource file once you have added your images or whatever.

Note: Just about anything can be in your resource file. For this Diamond editor I’m working on I stuck HTML help files in there. You kids who’ve never been more than six feet from an Internet connection need to learn the concepts of air-gapped and stand-alone. Machines on air-gapped networks have no Internet access. If you want help, all of the help has to be installed with the editor/IDE/whatever. The same is true for stand-alone because those machines aren’t even on a network.

Our Application

The code

Code wise, there isn’t much. I have always been a bit perturbed about parenting of QMovie objects. You either have to provide an empty QByteArray for the format to parent one or you have to construct the object with just a parent then assign the movie in another step. It would seem the could provide a constructor having a non-optional first parameter of parent. What is important is the QDebug statement.

output of application

You will note that this installation pulled from the Ubuntu repos only supports gif. Why this is important will be the subject of my next post.

To play the video below on my Ubuntu 20.04 LTS desktop I had to issue the following from the command line.

sudo apt install libdvdnav4 libdvdread7 gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly libdvd-pkg ubuntu-restricted-extras

Some browsers have their own codecs and such so might play video just fine. I was trying to look at the video locally before uploading to the post. If you can’t see it, then I guess you know what to try. It will also prompt you to run this command once.

sudo dpkg-reconfigure libdvd-pkg
The test application running

<Previous-part Next-part>

CopperSpice Experiments – Pt. 14

Why EDT?

The short answer is “Because I wanted it.”

The slightly longer answer is “I needed something of value to do while kicking the tires on CopperSpice. Adding themes and EDT keypad support to Diamond not only fit the bill but would benefit the world at large for as long as it could still compile.”

The much longer answer is the rest of this post.

I started out on the PDP 11/70 running RSTS/E. It was called a PDP because central buying policy by the US Government meant it was much more difficult to sell a “computer” than a “Programmable Data Processor”. Regrettably I can’t find an image on-line of the “RSTS/E is sexy” bumper sticker but I did find a scan of this history book covering Digital Equipment Corporation. Yes, an ancient machine, but as The Register reported in 2013, it will be running GE nuclear power plants to 2050.

https://forums.theregister.com/forum/all/2013/06/19/nuke_plants_to_keep_pdp11_until_2050/

I moved on to the VAX, then Alpha. As for editors, I used LSE (Language Sensitive Editor) when it was available, otherwise EDT. Both supported EDT keypad navigation. LSE just had a few more features. As long as we had terminals, it wasn’t a huge mental shift using a PC editor.

VT-320 terminal

The terminal was in a different spot with a different keyboard and had a visibly different screen. The hardship started when we used terminal emulators. Don’t get me wrong. Reflections was and still is a fantabulous terminal emulator.

Reflections version 7

It was just that now I was using keypad navigation in one window and it didn’t work in the editor on the other screen. Many many many VMS developers were infuriated by this. PC coders couldn’t understand. They had never walked in the light. Emacs (and by extension Jed) did a pretty decent job if and only if you modified your terminal definition so they could successfully map the NumLock key; otherwise it was even more infuriating.

For more than a decade now I have been off in the medical device/embedded systems world. I’ve not much thought about it. Then came a few calls about VMS work while I was writing The Minimum You Need to Know About the Phallus of AGILE. That got me waxing nostalgic for COBOL and I started writing a book on GnuCOBOL. That lead to the discovery that every PC editor, even VSCodium, well and truly sucks at COBOL. Emacs was the closest thing to a “useable” COBOL editor out there. That lead to the writing (out early next year) of The Minimum You Need to Know About GUI Emacs. When I was testing GnuCOBOL I created an account on Eisner so I could compare it to DEC/COMPAQ/HP COBOL.

Had remapping the terminal so Emacs could find the NumLock key not been a violation of my religion, this particular journey might not have happened, at least for me. Someone would have made a similar journey. The OpenVMS operating system is being ported to x86_64 hardware. These old conversations are coming up again in comp.os.vms. Some people have made their peace with Emacs. Others are lamenting the lack of a modern EDT compatible editor.

We are talking about people with thirty to fifty plus years of software development under their belts. None of them would start writing an editor from scratch. Some of them would gladly pitch in once something got reasonably close. There are thousands of retired and semi-retired OpenVMS developers out there who are more than a bit bored during COVID-19. One has been bored enough that they are solving all of the C programming problems for some programming site using COBOL on a PDP for no other reason that to have something to do with a keyboard.

I wrote this for me but I hope to leave it for them.

EDT Keypad Help

When you have EDT emulation on and hit the [HELP] key you will get a big html written help screen in a browser window. The first image on it is this one. It then explains what is and isn’t implemented and how to use stuff. The real problem with the PC isn’t just the mentality of that universe, but the origins behind the keyboard.

Midrange and mainframe systems were purchased by businesses. During the 1960s through the mid 1980s much of the “work” done on these systems revolved around data entry. When the numeric keypad shown on the left side of the above image was designed it was designed by Efficiency Experts to reduce both strain and errors. We measured the efficiency of data entry screens by how often a user would have to move from “home row.”

Home Row

Yes, even on character cell terminals we created drop down boxes for entry. A typist tabbed to the field and started typing. The displayed list would update with each keystroke showing only the entries that began with what they typed. When it got down to a single entry, it either auto selected that or there some something like [Return] they could hit to select it.

Today’s PC user interfaces throw efficiency out of a twelve story window in front of a fully loaded cement truck. They keep forcing the user to reach for the mouse. Combo boxes for many libraries/platforms don’t even allow a user to type in them. We are seeing some minor improvements in Web forms, at least when entering your state. When I type “i” for Illinois some actually scroll to the part of the list starting with “I” but far too many don’t.

Please take a moment and think about just how shitty the user interface is of your favorite editor when viewed for data entry efficiency. To select text you hit <Ctrl>-b or some key combination to “begin block” then you reach over to the arrow keys to navigate around, then you come back to home row to indicate “end of block” usually via some control key combination to copy or delete. “Oh, I use the mouse!” Well have a good long thought about that.

Movement from home row was always supposed to be an oddity or a limited event. That’s why when you look at old time data entry screens you see one of two things:

  1. All of the text fields on the left side of the screen and all of the numeric fields on the right. Entry proceeds down the left side then down the right.
  2. All text fields on the first screen and all numeric fields on the second screen.

When it came to inventory or other lookup type items that could not be just numbers, the letters used were always left handed letters, leaving the right hand over the keypad. The first person to suggest part number P1234 was fired.

Data entry needed a period, comma, and minus key for numeric values because not all forms formatted. The applications needed a few “special keys” so we got Programmable Function keys PF1 through PF4. EDT made use of those. The application told the terminal the keypad was in application mode and the terminal complied.

Standard PC numeric keypad

When PCs were “sold to business” they were initially sold to the accounting department. They were the only ones who could justify spending north of $5,000 to do spreadsheets. Spreadsheet users needed math keys. They basically needed a calculator keypad with a “Total” key they allowed someone to call “Enter.” There was great animosity between the PC world and the world of real computers. They absolutely refused to utilize the DEC keyboard despite it being the Unix standard. As a result you are missing a key. EDT users usually have to make some half-assed accommodation for this. The WORD key usually gets mapped as <ALT><KeyPad+> using the left <ALT> key.

Actual EDT help screen

Take a good look at the EDT keypad help again and consider selecting some text. (This is a shot of a Putty terminal actually logged into Eisner.) Forget about the left side, just look at the keypad. What are the steps to select text?

  1. First you have to navigate to the text you want.
  2. Start a selection.
  3. Navigate to end of text you want.
  4. Do something with the text.

EDT navigation was direction sensitive. It defaulted to ADVANCE but could be toggled between ADVANCE and BACKUP via the 4 and 5 keys on the keypad. Navigating a CHAR, LINE, Word, SECTion, or PAGE honored the current direction. Even EOL (End of Line) honored it as did DELete Line and UNDelete Line.

Prior to the PC, all of your navigation and basic editing was done with one hand. On the PC, if you wish to perform Word operations, you will need two hands, but the bulk of your operations are performed with one hand centered over the keypad.

If you needed text from various parts of various programs to be pasted into the program you are working on now, you simply opened those files up in other buffers and selected the text you needed. The APPEND would append your currently selected text to whatever was in the paste buffer (clipboard for you PC types.) Yes, I actually implemented that. If you want a rundown on the features I’ve been working on you can read it here. I had to add a clipboard viewer just so I could test this feature.

Clipboard contents

Once I have Diamond to where it is “good enough” I will expose it to my fellow EDT/VMS enthusiasts. At the very least there will be a few hundred who “give it a whirl.” These are all very seasoned developers. One or two joining the Diamond (assuming this code gets accepted) project would allow for serious enhancements. I can tell you right now some of the things that would be very high on their list of wants/priorities.

  1. Ability to support both ANSI-COBOL and DEC-COBOL-85. The current hard coding of comments in the syntax highlighting make this impossible. DEC-COBOL-85 was both free format and the comment indicator could be anywhere on the line. Haven’t dug deep into it for a while, but I seem to remember it also allowed for the “!” to be a comment-to-end-of-line indicator. ANSI-COBOL indicated a comment by having a “*” in column 7.
  2. FORTRAN-IV, 77, and 90 support. Again this spans the standards from fixed position card format to freeform coding. It also has an array of comment indicators.
  3. VAX BASIC. Still a rather large code base. Depending on the year written either had line numbers or did not. “REM” is the ANSI commend indicator. “!” was the DEC extension and it could be anywhere on the line. Hopefully they don’t want to deal with BASIC/PLUS from the PDP cross over era because that adds many parsing issues including “&” in column 78 (or 80) being a LINE CONTINUATION character.
  4. Bracket matching. I may actually get to this one on my own because I want it. Found some code that claims to do it, I just haven’t had time.
  5. Numeric constant highlighting. All of the regular expressions I have found don’t work when loaded from the syntax file. It’s something stupid and some other set of eyes will instantly see it.

I’m pretty sure you will see BLISS and a bunch of other languages pop up as well. Some of the people from VSI might contribute time (as if they have any spare time) as well. They would be the most in need at the moment as they are doing the OS port.

So, to sum it all up, myself and a bunch of others have suffered no end of frustration at the inefficient “standard” PC navigation. During these years when our hairs are turning either gray or white we want to see a modern editor with EDT navigation. I wanted to see it enough to do it. Like my brothers and sisters from that platform, I didn’t want to start from scratch.

Besides, the PC world needs to learn. Most of the shit they shun from the real computer world exists for a reason.

That’s why I added EDT keypad support. It’s not complete, but it is mostly there.

<Previous-part Next-part>

CopperSpice Experiments – Pt. 13

Why Themes?

From time to time people ask that question. Why did I want to add themes to the Diamond text editor? There are many reasons. Some of the reasons I identified in an answer I gave on Quora. Since most of you don’t read Quora I will regurgitate a bit of it here.

For eons I simply used whatever color scheme was default. I came up via terminals and you could have any color you wanted as long as it was the color of the phosphor in the DEC terminal.

Even then we were moving to different colors. The early white on black wasn’t really white and wasn’t really black. Then ghosting green became popular. At some point amber made an appearance.

IBM actually managed to have three colors in some of their terminals. Green red and something else.

As life went on I found out I needed to change the colors on PC editors. Then I found out I needed glasses. Later I found out I was diabetic.

1 in 10 Americans have diabetes

National Diabetes Statistics Report, 2020

Diamond Text Editor COBALT theme

COBALT is probably one of the oldest “themes” around and now I understand why. Usually I can start out with a white or white-ish background in the morning but sometime after lunch I’m cranky, irritable, and should have changed to a darker background hours before.

Years ago I spoke with my doctor about this. He told me the fluid in the eye is “mostly sugar.” As your glucose level rises and falls throughout the day so does the level of sugar in this fluid. The eye components got used to being bathed in lots of sugar. When the sugar content changes, they change their shape. It takes quite a bit of time with a consistent sugar level for the eyes to “adjust to a new normal.” That’s why diabetics are supposed to see an eye doctor so often.

There are actual studies being used by vending machine and medical device manufacturers showing which color schemes work well for which situations. If you are going to be working with visually impaired individuals you need black text on a gray background. It is a certain gray, not just any gray. I just don’t remember the RGB code for it off the top of my head. For some reason people who are legally blind yet not totally blind can see that scheme the best. They generally either cannot function or have extreme difficulty with black on white.

In 1990 America passed the Americans With Disabilities Act. Basically a company of a certain size must provide “reasonable accommodations” to people with a disability. That “certain size” is rather small too. According to the link it is a business with 15 or more employees. While it may not be obvious to you, providing an editor with color themes is one of the easier accommodations. Having that editor start with a white background is generally not good due to vision problems so someone probably needs to initially switch it to gray.

Diamond with greygray theme

When it comes to syntax highlighting editors one must also consider that some really famous people are color blind. Keanu Reeves and Mark Zuckerberg are among them. That link also provides a lot of information about how to use a tool to come up with color blind friendly palette. While they are talking about for charts and presentations, the same applies for syntax highlighting editors.

No, I haven’t used the tool. At some point, as development moves forward and others discover the editor, someone will either use the tool or they will simply know what palettes to use because they do this color thing every day. There are a great many really smart people out there who couldn’t code “Hello World!” but are amazing at graphics design and use “proper workplace color palettes” every day. They also tend to need a text editor from time to time to take notes and keep track of to-do lists.

Diamond color dialog

Most of you didn’t pay attention or at least didn’t consider the possibilities of the Import/Export buttons I added to the colors dialog. This is for them and by extension, all of us. When non-coding people who know a lot about proper color palettes required by this specific community they can contribute to the project by creating a theme and exporting it. What we currently don’t have is a central repository of themes with screenshots where people could pick and choose the theme they want. I’m not thinking of a “git pit.” I’m thinking something along the lines of the Emacs themes site.

I have had the distinct honor of working with some phenomenal UX and design professionals over the decades. Did I learn how to do this from them? No. I learned that it must be done. I learned that it must be done by a true professional, not just someone getting paid to do the job.

Believe it or not, I learned that last part at a vending machine company. That dude immersed himself in all of the nooks and crannies of this topic. He even told me why. “Because one of the jobs visually impaired people tend to get is restocking vending machines.” This is particularly true when it comes to the crews that drive rest stop to rest stop on the Interstate cleaning them and filling the vending machines. In some states they are all state employees. Only one person in the vehicle needs to see well enough to drive. The rest only need to see well enough to do their jobs. This company put great effort into making sure their machines were accessible by all.

The next time you walk past one of those vending machines with a fancy touch screen display streaming advertisements while nobody is using it ask yourself one thing: “How does a blind person use it?” The rush to full touch-screen interface on many products has excluded a market segment. There is a reason you see touchscreen displays with one of these:

Vending machine keypad with braille

What gets forgotten in the OpenSource world, despite all of the platitudes, is that it isn’t “just us” or “just me.” People like to talk about all of the “freedom and creativity” of OpenSource projects. What they have to realize is that you have to abide by “gubmint rulz” when it comes to design. Far too many projects don’t which is why they whither and die.

I will close this topic with a link to this short blog post about color blind friendly palettes. Thankfully it also has this link to the actual RGB scan codes. Creating these things is something I don’t have time to do right now, but anyone can use the Colors dialog and enter the RGB codes, save, test, and export a theme.

Having worked at an advertising agency before let me ask you a question:

Do you know why Betty Crocker cake mix crushed the competition despite there being dozens of other brands on the shelf?

This was the days when women actually called themselves a “House Wife” and were proud of it. All of those other cake mixes were “instant.” You just added water and put them in the oven. With Betty Crocker you added an egg. Adding an egg crossed the psychological barrier and the conscientious house wife could honestly say “Yes, I baked it myself.” Adding an egg made it not instant and therefore okay. Instant is what the bad wives did and you would never find them featured in Good Housekeeping.

Putting in place the groundwork for community themes is the text editor equivalent of “adding an egg.” Allowing users to create plug-ins is the text editor equivalent of allowing too many cooks into the kitchen. That’s a condition Emacs suffers quite badly from now. Most of the plug-ins conflict horribly or cause random oddities in unrelated plug-ins. With every plug-in you install it is like tap dancing in a mine field.

<Previous-part Next-part>