16 Minutes to Build a QList

It took CopperSpice almost 16 minutes to build the same QList Qt built in under a second. I have spent more than a day creating the test case and running the test data. It is currently 6:24pm local time. I started my callgrind test of this at about a quarter to 2pm. It’s still running.

Callgrind start time

Here are the current stats.

roland@roland-HP-EliteDesk-800-G2-SFF:~/sf_projects/ListTest_debug$ callgrind_control -e -b
 PID 81466: ./listtest
 sending command status internal to pid 81466
 Totals:               Ir 
    Th 1  2,138,722,737,075 
    Th 2         38,724,327 
 Frame:                Ir Backtrace for Thread 1
    [ 0]    180,647,087,012 __gnu_cxx::__normal_iterator > >::base() const (25806726717 x)
    [ 1]    903,232,403,521 bool __gnu_cxx::operator!= > >(__gnu_cxx::__normal_iterator > > const&, __gnu_cxx::__normal_iterator > > const&) (25806640101 x)
    [ 2]  1,935,503,204,513 long CsString::utf8::distance<__gnu_cxx::__normal_iterator > > >(__gnu_cxx::__normal_iterator > >, __gnu_cxx::__normal_iterator > >) (86616 x)
    [ 3]  1,935,515,065,506 CsString::CsBasicString >::size() const (86605 x)
    [ 4]  1,935,493,582,772 QString8::size() const (86604 x)
    [ 5]  2,137,931,512,290 MainWindow::tokenize(QString8) (1 x)
    [ 6]  2,138,197,350,792 MainWindow::startTest() (1 x)
    [ 7]  2,138,197,350,814 void CsSignal::Internal::cs_unpack_method_args_internal(MainWindow, void (MainWindow::)(), std::tuple<> const&, std::integer_sequence) (1 x)
    [ 8]  2,138,197,350,832 CsSignal::Internal::CSVoidReturn CsSignal::Internal::cs_unpack_method_args(MainWindow, void (MainWindow::)(), std::tuple<> const&) (1 x)
    [ 9]  2,138,197,351,622 CsSignal::Internal::Bento::invoke(CsSignal::SlotBase, CsSignal::Internal::TeaCupAbstract const) const (1 x)
    [10]  2,138,197,353,213 void CsSignal::activate(QAbstractButton&, void (QAbstractButton::)(bool), bool&) (1 x)    [11]  2,138,197,354,342 _dl_runtime_resolve_xsave (1 x)    [12]  2,138,197,354,349 0x00000000052764c0 (1 x)    [13]  2,138,197,354,436 QAbstractButtonPrivate::emitClicked() (1 x)    [14]  2,138,197,355,289 _dl_runtime_resolve_xsave (1 x)    [15]  2,138,197,355,296 0x000000000528e210 (1 x)    [16]  2,138,197,639,154 QAbstractButtonPrivate::click() (1 x)    [17]  2,138,197,639,951 _dl_runtime_resolve_xsave (1 x)    [18]  2,138,197,639,958 0x0000000005294760 (1 x)    [19]  2,138,197,640,057 QAbstractButton::mouseReleaseEvent(QMouseEvent) (1 x)
    [20]  2,138,208,742,219 QWidget::event(QEvent) (320 x)    [21]  2,138,198,960,804 0x000000000525e110 (64 x)    [22]  2,138,200,447,252 QAbstractButton::event(QEvent) (64 x)
    [23]  2,138,200,446,077 0x000000000526ef20 (60 x)
    [24]  2,138,200,431,643 QPushButton::event(QEvent) (52 x)    [25]  2,138,213,997,253 QApplicationPrivate::notify_helper(QObject, QEvent) (187 x)    [26]  2,138,198,174,253 0x000000000528fb40 (2 x)    [27]  2,138,214,176,491 QApplication::notify(QObject, QEvent) (144 x)    [28]  2,138,210,280,964 QCoreApplication::notifyInternal(QObject, QEvent) (115 x)    [29]  2,138,198,412,550 0x00000000052940b0 (46 x)    [30]  2,138,198,439,141 QApplicationPrivate::sendMouseEvent(QWidget, QMouseEvent, QWidget, QWidget, QWidget, QPointer&, bool) (45 x)    [31]  2,138,198,453,527 0x000000000528ece0 (46 x)    [32]  2,138,198,628,219 QWidgetWindow::handleMouseEvent(QMouseEvent) (45 x)
    [33]  2,138,198,654,192 0x00000000052574a0 (46 x)
    [34]  2,138,206,007,189 QWidgetWindow::event(QEvent) (59 x)    [35]  2,138,215,800,987 QApplicationPrivate::notify_helper(QObject, QEvent) (424 x)    [36]  2,138,215,810,580 0x000000000528fb40 (425 x)    [37]  2,138,215,920,651 QApplication::notify(QObject, QEvent) (425 x)    [38]  2,138,209,267,733 QCoreApplication::notifyInternal(QObject, QEvent) (294 x)    [39]  2,138,198,682,215 0x00000000052940b0 (46 x)    [40]  2,138,198,681,680 QApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent) (46 x)
    [41]  2,138,198,713,456 0x000000000525ce00 (47 x)
    [42]  2,138,207,133,489 QApplicationPrivate::processWindowSystemEvent(QWindowSystemInterfacePrivate::WindowSystemEvent*) (54 x)
    [43]  2,138,207,149,986 0x000000000527f9e0 (55 x)
    [44]  2,138,198,758,614 QWindowSystemInterface::sendWindowSystemEvents(QFlags) (98 x)
    [45]  2,138,207,170,943 0x0000000009387590 (99 x)
    [46]  2,138,214,384,853 QUnixEventDispatcher::processEvents(QFlags) (99 x)
    [47]  2,138,202,168,869 QEventLoop::processEvents(QFlags) (98 x)
    [48]  2,138,214,389,585 0x0000000007d8a820 (99 x)
    [49]  2,138,214,393,930 QEventLoop::exec(QFlags) (1 x)
    [50]  2,138,214,394,913 _dl_runtime_resolve_xsave (1 x)
    [51]  2,138,214,394,920 0x0000000007d89e50 (1 x)
    [52]  2,138,214,398,749 QCoreApplication::exec() (1 x)
    [53]  2,138,214,399,613 _dl_runtime_resolve_xsave (1 x)
    [54]  2,138,214,399,620 0x0000000005274480 (1 x)
    [55]  2,138,214,400,492 QApplication::exec() (1 x)
    [56]  2,138,214,400,494 0x0000000000112c80 (1 x)
    [57]  2,138,652,121,489 main (1 x)
    [58]  2,138,652,305,048 (below main) (1 x)
    [59]  2,138,652,305,060 _start (1 x)
    [60]                  . 0x0000000000001100
 Frame:                Ir Backtrace for Thread 2
    [ 0]            154,047 poll (3218 x)
    [ 1]            161,333 0x00000000094b1560 (3219 x)
    [ 2]          7,224,790 0x000000000000bb80 (3027 x)
    [ 3]          7,750,239 xcb_wait_for_event (2874 x)
    [ 4]          7,953,840 0x0000000009385040 (2875 x)
    [ 5]         38,695,270 QXcbEventReader::run() (1 x)
    [ 6]         38,722,486 QThreadPrivate::start(void*) (1 x)
    [ 7]         38,724,320 start_thread (1 x)
    [ 8]                  . clone

Here is the code:

38 Downloads

28 Downloads

The Qt version is a CMake project. I developed this on Ubuntu 20.04 LTS using Qt version from the repo. My CopperSpice build was current as about a week or so ago.

Under Qt I got what I expected. Sub-second list build time.

Qt test run

Under CopperSpice, not so much.

CopperSpice Test Run

Okay, so it was only 15 minutes and change, not 16 minutes. Are you going to quibble about that last minute when an application has already hung for more than 10?

This isn’t a random bogus test. I was porting the Gede debugger to CopperSpice and the text file included in the source directory for use in testing contains a gdb response. (Test application looks for text file in $HOME.)

(gdb) 
 -file-list-exec-source-files

In one giant formatted string gdb sends back all of the source files relevant for this debug session. Not the files themselves, but the full path and name. Any front end needs to be aware of what is in play. Gede (and I imagine most gdb front ends) tokenize this stuff so they can deal with it. You will see the tokenize() routine is basically the same. The CopperSpice version has some additional qDebug() calls so I could actually see something.

There appears to be something horribly inefficient in either QString8 or QList.

[ 0]    180,647,087,012 __gnu_cxx::__normal_iterator > >::base() const (25806726717 x)
    [ 1]    903,232,403,521 bool __gnu_cxx::operator!= > >(__gnu_cxx::__normal_iterator > > const&, __gnu_cxx::__normal_iterator > > const&) (25806640101 x)
    [ 2]  1,935,503,204,513 long CsString::utf8::distance<__gnu_cxx::__normal_iterator > > >(__gnu_cxx::__normal_iterator > >, __gnu_cxx::__normal_iterator > >) (86616 x)
    [ 3]  1,935,515,065,506 CsString::CsBasicString >::size() const (86605 x)

I haven’t noticed a huge problem with QString8 so I’m assuming the issue is in the QList code. When working on my Diamond port I noticed really sluggish behavior with syntax highlighting that was using lists of regular expressions.

Hopefully this somewhat compact example will help the team track it down. I will try to update this post tomorrow with the completion screen shot. Assuming it completes overnight.

You can read more about my CopperSpice experiments in this post thread.

Edit – 2021-02-24 – 4:30am

I love Callgrind and Valgrind. You run something under them and days later it gets done. As of 4:30am this morning this appears to be no closer to completion. I do have newer stats at the top to share.

roland@roland-HP-EliteDesk-800-G2-SFF:~/sf_projects/ListTest_debug$ callgrind_control -e -b
 PID 81466: ./listtest
 sending command status internal to pid 81466
 Totals:               Ir 
    Th 1  5,208,933,012,211 
    Th 2         47,402,648 
 Frame:                Ir Backtrace for Thread 1
    [ 0]  1,977,743,839,393 bool __gnu_cxx::operator!= > >(__gnu_cxx::__normal_iterator > > const&, __gnu_cxx::__normal_iterator > > const&) (56506966840 x)
    [ 1]  4,238,033,891,625 long CsString::utf8::distance<__gnu_cxx::__normal_iterator > > >(__gnu_cxx::__normal_iterator > >, __gnu_cxx::__normal_iterator > >) (189644 x)
    [ 2]  4,238,059,867,454 CsString::CsBasicString >::size() const (189633 x)
    [ 3]  4,238,039,415,000 QString8::size() const (189632 x)

Copy on Write

While I’m holding out hope this is just a horrible bug in QList, my gut tells me this is really do to a fundamental change between Qt and CopperSpice. Qt has a “copy on write” policy implemented under the hood for QObject based things. I haven’t looked at the actual code in quite some time but here is the gist of it.

When you assign QObject based entity A to entity B there is just some reference magic manipulated under the hood. There is no need to actually copy the values until something alters entity A or B. This makes things very fast.

You can also take a big hit if there happens to be 100+ things referring to this particular value instance when it needs to change. Think a working/scratch object you load a “default” value into from some external source then use to initialize a hundred element list. Not an integer, but a substantial object like a page of text or a QPixmap image. Later on in the code your working QPixmap needs to load a different image. That image pays the price. A hundred copies now have to be made before the first image change can happen. The second image change pays no such price so it is very fast. Beginners chase their tail trying to figure out why the first one takes so long. This is why.

Improve Battery Life

This minimal use of dynamic memory is incredibly important today in battery operated embedded systems. Many of them have lower powered slower everything inside them because they are trying to add minutes or hours to battery life. You have only two options when it comes to extending battery life.

  1. Use less juice
  2. Buy a bigger battery.

Option one is cheaper and doesn’t add pounds to the device. That’s the route most device manufacturers try to go first. The result of this is you can be using the exact same Linux kernel and exact same compiler for your target device and dynamic memory allocation will suck.

The XML Trap

A former coworker who became manager didn’t believe me when I said this. All of his embedded systems work had been gambling machines connected to grid power. Manufacturers spared no expense in speeding those devices up and they didn’t give a hoot about how much juice it took. The manager who would be retiring was just nodding his head in agreement with me as I spoke this.

We had to preload our images on that device. Multiple images for one screen came in a single SVG file and you loaded them via their tag. SVG is really XML so it isn’t such a big deal. What was a big deal was someone deciding that code needed to be re-ordered so it loaded images alphabetically instead of a screen at a time. I think the power-on boot time climbed to something like 45 minutes. You didn’t power off unless you had to.

Re-ordering to load all images from a single file before moving on to the next trimmed our boot time down to three minutes. We obviously still had a ways to go, but it was now obvious where to look. Remove all needless dynamic memory allocation. Every time we opened an SVG file a plethora of XML class objects had to be dynamically created and fully populated. When we closed the file these all had to be reclaimed.

Dynamic memory allocation in most/many battery operated embedded systems sucks. Know this. Plan for it. Your life will be happier.

Desktops Used to Suck

NCR PC4

The NCR PC4 was the first computer I ever purchased for myself. A starving college kid, but I saved up the $1700+ to buy one. A whopping 4.77 MHZ and dual floppy disk drives with a ghosting green phosphorus monitor. Tipping the scales around 50 pounds if memory serves.

I didn’t play with Qt until Qt3 was out and then it was near the end of Qt3 waiting for Qt4. I’m telling you this so you can understand why copy on write was implemented.

  • Intel released the Celeron 533 MHz with a 66 MHz bus processor on January 4, 2000.
  • AMD first released the Duron processor on June 19, 2000, with speeds of 600 MHz to 1.8 GHz and bus speeds of 200 MHz to 266 MHz. The Duron was built on the same K7 architecture as the Athlon processor.

None of you would consider buying one of these computers today, but they were smoking fast when they were released. All kinds of hardware and OS level techniques were developed over the years to avoid actually having to transfer significant amounts of memory because we had limited abilities.

Don’t try and tell me that isn’t how things are now.

Remote controls

Just how much processor and RAM do you think is in one of these? How often do you have to change the 2-3 AA or AAA batteries powering them?

You see, production programmers used to control the various language specifications. They were interested in solving real world problems not pursuing pie-in-the-sky-once-in-a-blue-moon topics.

Enter the Academics

More and more production coders had to devote more time to production coding. Academics wriggled into the controlling positions of language standards. I always picture them dressed in a white robe of some kind, sitting cross legged, palms skyward, head tilted back, chanting “Oooooohhhhhhhmmmmmm” as they try to become one with the object.

While some good things came out of this initially, lots of bad is the most recent contribution. You can read the discussion about MVVM in my latest book to understand one of the more recent tragedies. At some point the academics took it upon themselves to declare copy-on-write a violation of the C++ standard and forbid it.

Under the hood CopperSpice (per the documentation and Web site) adheres to the C++ standard. My gut tells me this is the root of the beyond sucky performance our test application exhibits. Under a second for Qt and 15+minutes for CopperSpice.

Let’s Look at a Few Things

QList<Token *> MainWindow::tokenize( QString str )

Officially, assuming my memory is accurate, this statement requires both the parameter and return value to be passed by value. I don’t read the standard on a daily basis, I’ve been working with Qt a long time. Anyone who has will have a large base of code looking much like this. If you are thinking of porting this is going to be a major issue.

input file size

According to the ls command, the size of the input txt file is 291K. That’s in raw UTF-8. Things are in QChar32 under the hood in CopperSpice so we can agree that it is more than 300k in memory.

 QChar c = str[i];

Every time we do this we have to make a copy of the character. (The alternative would be dereferencing each and every place you find c being used in the code and hoping the compiler could optimize.)

 cur->m_text += c;

Every one of those statements is going to cause some reallocation. I can’t find it now, but I swear to God I remember reading about a compile/build option when building Qt from source where you could set the default initial allocation of all QString objects. It would be like you called reserve() every time you declared a QString.

I can’t find the parameter/option now. I ran into it on one embedded system project. Really sucky dynamic memory allocation had us looking for solutions. We knew that 15 characters would handle about 80% of the strings in the system. We had the RAM, it was the allocation price that was killing us. Forcing a reserve(15) into every string under the hood breathed new life into the application.

Due to C++’s type system and the fact that QString is implicitly shared, QStrings may be treated like ints or other basic types. For example:

https://doc.qt.io/qt-5/qstring.html

I’m going to steal this example from that doc as well.

QString Widget::boolToString(bool b)
 {
     QString result;
     if (b)
         result = "True";
     else
         result = "False";
     return result;
 }

The result variable, is a normal variable allocated on the stack. When return is called, and because we’re returning by value, the copy constructor is called and a copy of the string is returned. No actual copying takes place thanks to the implicit sharing.

https://doc.qt.io/qt-5/qstring.html

Yes, you can find raving discussions against CoW (Copy on Write as they use it) on places like StackOverflow. But there is now nearly 30 years of code treating objects like integers when passing things around because of the performace CoW gives us.

If you are trying to port an existing codebase to this library you are going to have to redesign. It may be the least painful to make compile and run, but it won’t run well with gotchas like these.

From the CopperSpice Web site

Implicit sharing is being purged from CopperSpice. You will note that QString and QList aren’t in the list of classes still supporting implicit sharing if you click that link.

Killed off Callgrind

It was 8:50am and still not done. If I let it run 24 hours my elapsed time thing wasn’t going to work.

I really wanted to get to the bottom of this. Everybody emptying their colons on the CoW way of life claimed you could solve all performance problems via reference. Just how much truth is there to such a wild claim?

changed method

I changed the method to be void, accepting the list to be created and the string to create it from via reference. Got a clean ninja install and ran. Removing these two copies should dramatically improve the time if there is any truth to that argument.

After changes

Yeah, 32 seconds slower according to that run. Next I replaced every list.push_back() call with:

list.append( cur );

Just in case this wasn’t as direct a replacement as the documentation says.

after change to .append()

Possibly saved 5 seconds or there was just less going on.

Yes, you are correct. I should have looked for a better profiling tool and spent days learning how to use it if I wanted to dig all the way to the bottom of this problem.

Having said that, there isn’t much going on here. If the Zen Masters are correct and this Holier more Pioused C++ way of doing things really is better, I should be able to change just one thing and see this drop down to at least under a minute, right?

Next I changed all of the cur->m_text += statements to be:

 cur->m_text.append( c );

Naturally the ones with hard coded characters kept their hard coded characters. Why take these baby steps? I want to get to the root cause without going into the CopperSpice code itself. In this case it would be going all the way into GNU if these really are wrappers on the STL. I wondered if there weren’t a few instances of bad code decisions adding to the performance issues.

Overloads could have caused confusion
Shaved another 12 seconds

You would be correct in believing that should not have fixed anything. Obviously the appropriate method wasn’t always being called.

Next I wanted to try reserve() but that wasn’t pious and holy enough.

I really don’t know what would have been the difference to leave reserve() in and just change the meaning of the parameter from bytes to numOfChars. That appears to be what they did with resize().

token.h change

Changed the Token constructor to pre-extend m_text by 50 characters. If the needle wiggles enough we can poke around looking for “magic size.”

It should be painfully obvious by now that one cannot fix this problem from the outside. Obviously resize() isn’t well implemented.

After resize tweak

So, we take the resize() code out of Token and just for grins add it here:

Try pre-extending the QList
No meaningful difference

Summary

This is a substantial problem. Most likely it is why I hear from so many who have tried CopperSpice that it is slow. I certainly noticed it working on my fork of Diamond.

While CoW may be considered something only the great unwashed masses use, the great unwashed masses actually get something done. When reserve() was removed from QString, QList, and wherever else, I don’t think it was really understood how those benefitted the community. Reserve allocated a bunch of empty units that append(), insert(), etc. would use until they were exhausted. This let you stall off the overhead of dynamic memory allocation for every object added. You could increase performance by knowing the size which could handle roughly 80% of your needs.

Judging from the elapsed time of the tests, it is pretty obvious that resize() doesn’t create this nice big contiguous buffer you can use. While this system isn’t a blinding make-the-gamer-kids-drool machine, it’s not a slouch.

Test system

There has been talk for years about Qt just becoming a wrapper over STL containers. I’ve heard talk on the mailing list about “Why should we spend the resources to maintain that which is already there?’

This would be the argument!

The STL does not perform CoW. Until there is some exponential speed improvement in STL containers they are not viable solutions once you get beyond trivial accademic applications.

This wasn’t even that difficult of a task! Creating a list of tokens from a string that is probably 500K in memory. Those looking to leave Qt need to take this sample application and re-implement it with whatever cross platform framework they are thinking about using.

So You Can’t Get Your Qt Models to Work With QML?

This particular rant was started by my looking to divert my mind by answering a question on StackOverflow. Never ever ever go to StackOverflow. Kids today just don’t know squat and there is nothing you can do to help them. Let’s start with my initial response an a slightly improved image.

The gist of the issue

QML is just a hand polished turd. There is no polite way to put it. Qt made a desperate ploy to attract script kiddies in the iDiot phone market with QML. The problem is, by and large, script kiddies don’t know anything. They tend to be “self-taught” with far more emphasis on “self” than “taught.” No grounding in the fundamentals of application design and software development. They just want to hack.

They like scripting languages because you can “just hack” without rules. If you want to get an idea for just how much disinformation exists in the script kiddie world, check out the claims about Python being type safe.

Hopefully you will all click the link and look at the original question and code. Please do so now!

My Initial Response

This image may not be 100% accurate but it is how one must visualize things in their mind. You have three distinct lanes an object lives in when you introduce QML: C++, QML Engine, and JavaScript engine. Each and every one of those lanes believes beyond a shadow of a doubt that they control the life and death of the object.

When you are just passing integers around that are passed by value, this is no issue. When you are passing QStrings around, because of the copy-on-write charter of Qt, this is only a minor issue. When you are passing real objects around, worse yet, complex containers of real objects, you have to understand this completely. The answer that “fixed” your problem really only masked it. You will find there is a lot of QML and JavaScript code out there that exists only to mask an application not respecting the lanes.

Had this application used an actual database there would be a fourth lane. Yes, SQLite provides an SQL interface and allows many things, but an actual database has an external engine providing shared access and controlling the life of cursors. SQLite files tend to be single user. Yes, multiple threads within your application can access it, but while your application is running you cannot open a terminal window and use command line tools to examine the database. Think of it more as a really nice indexed file system without sharing.

So, you create an object in C++ and then expose it to QML. The QML engine now believes beyond a doubt that it controls the life and death of that object despite not having its own copy.

QML is really feeble. It can’t actually do much, so it has to hand any significant object off to JavaScript. The JavaScript engine now believes beyond a shadow of a doubt that it now controls the life and death of that object.

You need to also envision these lanes as independent threads. There will most likely be many threads within each, but, in general, any signal or communication between these will go on the event loop of the target as a queued event. That means it will only be processed when it finally bubbles to the top of the queue for that event loop.

This, btw, is why you can never use Smart Pointers when also using QML/JavaScript. Especially the kind that do reference counting and delete the object when there are no more references. There is no way for the C++ lane to know that QML or JavaScript are still using the object.

The answer telling you to check for undefined property is masking the problem that your code is drunk driving across all lanes. Eventually, on a faster (or sometimes slower) processor garbage collection for one of the lanes will run at a most inopportune moment and you will be greeted with a stack dump. (The drunk driving code will hit a tree that does not yield.)

Correct Solution #1: Never use QML or JavaScript. Just use C++ and Widgets. Stay entirely within the C++ lane. If that is a route open to you it’s a good way to go. There is an awful lot of production code out there doing just that. You can obtain a copy of this book (or just download the source code from the page) and muddle through building it.

Correct Solution #2: Never actually do anything in QML or JavaScript. This is an all together different solution than #1. You can use QML for UI only, leaving all logic in C++. Your code is failing because you are trying to actually do something. I haven’t built or tried your code. I simply saw

function deleteRowFromDatabase(row)

which shouldn’t exist at all. C++ holds your model. You emit a signal from your QML/JavaScript lanes when user action requires deletion. This signal becomes a queued event in the C++ lane. When it is processed the row will be deleted and the model updated. If you have properly exposed your model to QML it will emit some form of “model changed” signal and the UI will update accordingly. One of the main points of MVC (Model-View-Controler) is communication to/from the model. When the data changes it notifies the view(s).

Correct Solution #3: Never use C++. Have your C++ be a “Hello World!” shell that just launches your QML. Never create and share an object between C++ and the other two lanes. Do everything inside of JavaScript and QML.

Binding loop errors, in general, happen when code drunk drives across all three lanes. They also happen when code doesn’t view each lane as at least one distinct thread with its own event loop. The C++ lane hasn’t finished (perhaps not even started) the delete operation but your code is already trying to use the result.

The Final Solution

Lots of back and forth happened. When someone has locked themselves into a failed architecture they are generally the last ones to see the failure of their plan. I know. I’ve been on the other side of that. You have to be on the other side of that a few times to learn how to avoid it in the future.

First problem is they chose to use QML. That locked them into using a model. You will find out why that was bad in a bit.

Second problem was drunk driving across all of the lanes. When you stay in the green lane, life is good. Theoretically, when you stay in the red lane life is just as good, you simply can’t do anything real. If you choose to exist only in the yellow lane, then you don’t need Qt at all.

Third problem was the fact there was absolutely no reason for a proxy. You are supposed to use a proxy when you need to transform data for display, like formatting a date or populating a combo box of reference table values. Some also use them to ensure a read-only data source cannot accidentally receive a write request.

Fourth problem was not really knowing anything about a relational database. I see this a lot in the self-taught universe. I even see it with recent college graduates because they can pursue some kind of Web/Game development degree and nobody thinks to include a course on how to care for and feed a relational database.

The Models

QML Turd

There is just no polite way to describe QML. This forcing of models upon the world would be fine if it was properly architected. Hell, suitable debugging and error information would go a long way.

QVariant data(const QModelIndex &index, int role) const;

That one line causes more problems than anything else. If it doesn’t exactly match that in your concrete implementation, you will get no data. The documentation and the IDE will lead many to believe it should have a different signature.

“Oh, but if it’s not right it will just give me an error, right?”

Nope!

If you are using this model in a table the model will return the row count and your table will be all NULL data. If QML can’t find this, it won’t tell you. Adding insult to injury there is not a flag or debugging option you can set to force QML to identify which data() method it used.

So, if you have 60 rows in your table the model will tell the UI there are 60 rows and you will have 60 NULL rows and, most likely a scrollbar.

Broken

This is where OOP jumped the shark. You can get my latest book and read about MVVM. I won’t repeat that essay. What I will tell you is that MVVM was created by people who don’t know how to use a relational database and are physically incapable of understanding data, without a relation, is useless.

If someone walks into a room and says “25” then leaves you have no idea WTF they are talking about. If someone walks into a room and says “John Smith is 25” then leaves, you may or may not know who John Smith is, but you now know he is 25. The 25 now has meaning because it has a relation. With the possible exception of NULL, there is no meaning without relation.

You need to know the previous information because it provides a frame of reference. People who don’t know how to properly use relational databases create the models. When they create automated tests for them these tests will be worse than non-existent.

void SQLiteModel::deleteRow(int rowNo)
 {
     qDebug() << "removeRow called for " << rowNo << "\n";
     bool rslt = removeRow(rowNo);
     qDebug() << "result of removeRow() " << rslt << "\n";
     qDebug() << "Error: " << lastError().text() << "\n";
     qDebug() << "query: " << query().lastQuery() << "\n";
     submitAll();
 }

When the above executes you see the following.

removeRow called for  3
 result of removeRow()  false
 Error:  "near \"WHERE\": syntax error Unable to execute statement"
 query:  "select * from questions;" 

It didn’t matter if the table was created with this

query.exec("CREATE TABLE IF NOT EXISTS questions (Q_NO INTEGER PRIMARY KEY, Q_TEXT TEXT);"); 

or with this

query.exec("CREATE TABLE IF NOT EXISTS questions (Q_NO INTEGER PRIMARY KEY AUTOINCREMENT, Q_TEXT TEXT NOT NULL);" 

It appears this has been broken for over a year, if it ever worked at all.

Lack of Formal Training

Formal training. Actually attending a good school for computer science cannot be overvalued. Sadly there are a lot of run-for-profit shit schools along with just plain shit schools.

Formal training. That’s what teaches you to generate large test data sets. Testing with only ten or fewer records won’t help you find anything. Even for some hokey little program like this you need to initially test with at least enough records to force a scrollbar onto the UI. At least 60 would be good. 500 would be better. There is an ipsum generator if you really hate typing that much.

Formal training. That’s what teaches you the first step to solving a database table issue of any kind is verifying the creation and population of the table from the command line. If you read through the original code you will see it was just stuffing records into the proxy.

Formal training. That’s what teaches you the “default debugger” and moving onto other I/O operations like deleting rows. Other things that can be verified from the command line.

Formal training. That is what teaches you proper software development technique. The language and tools you use don’t matter. Proper technique is what matters.

You can solve any programming problem with the proper technique formal training provides. That and a formal logic class that teaches no programming language is what one needs to succeed in IT. Tools come and go. Those two things are constants.

The Code

QT += quick sql
 CONFIG += c++11
 #
 This is important
 #
 CONFIG += qmltypes
 QML_IMPORT_NAME = Fred
 QML_IMPORT_MAJOR_VERSION = 1
 You can make your code fail to compile if it uses deprecated APIs.
 In order to do so, uncomment the following line.
 DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0
 SOURCES += \
         main.cpp \
         sqlitemodel.cpp
 RESOURCES += qml.qrc
 Additional import path used to resolve QML modules in Qt Creator's code model
 QML_IMPORT_PATH =
 Additional import path used to resolve QML modules just for Qt Quick Designer
 QML_DESIGNER_IMPORT_PATH =
 Default rules for deployment.
 qnx: target.path = /tmp/$${TARGET}/bin
 else: unix:!android: target.path = /opt/$${TARGET}/bin
 !isEmpty(target.path): INSTALLS += target
 HEADERS += \
     sqlitemodel.h
ifndef SQLITEMODEL_H
 define SQLITEMODEL_H
 include 
 class SQLiteModel : public QSqlTableModel
 {
 Q_OBJECT
 public:
     explicit SQLiteModel(QObject *parent = NULL);
     ~SQLiteModel();
 bool setQuery(const QString &query); void defaultQuery(); virtual QHash<int, QByteArray> roleNames() const; QVariant data(const QModelIndex &index, int role) const; void declareQML(); Q_INVOKABLE QString getQuestion( int rowNo);
 public slots:
     void deleteRow( int rowNo);
     void copyRow( int rowNo);
     void updateRow( int rowNo, QString txt);
 private:
     bool openDb();
     void addStarterData();
 QSqlDatabase db; int m_recordCount;
 };
 endif // SQLITEMODEL_H
include "sqlitemodel.h"
 include 
 include 
 include 
 include 
 include 
 include 
 include 
 include 
 include 
 include 
 void SQLiteModel::declareQML()
 {
     qmlRegisterSingletonInstance("Fred", 1, 0, "MyModel", this);
 }
 SQLiteModel::SQLiteModel(QObject *parent)
     :QSqlTableModel(parent)
 {
     setEditStrategy(QSqlTableModel::OnFieldChange);
     openDb();
 }
 SQLiteModel::~SQLiteModel()
 {
 }
 bool SQLiteModel::openDb()
 {
     const QString DB_NAME = "test.db";
     QDir hDir = QDir::home();
     bool retVal = false;
 db = QSqlDatabase::addDatabase("QSQLITE"); db.setDatabaseName(hDir.absoluteFilePath(DB_NAME)); retVal = db.open(); if (retVal) {     QSqlQuery query(db);     query.exec("CREATE TABLE IF NOT EXISTS questions (Q_NO INTEGER PRIMARY KEY AUTOINCREMENT, Q_TEXT TEXT NOT NULL);");     query.exec("SELECT COUNT(*) FROM questions;");     while (query.next())     {         QSqlRecord rec = query.record();         int recordCount = rec.field(0).value().toInt();         if (recordCount < 10)         {             addStarterData();         }     } } return retVal;
 }
 bool SQLiteModel::setQuery(const QString &query)
 {
     QSqlQueryModel::setQuery(query);
 if (this->query().record().isEmpty()) {     qWarning() << "SQLiteModel::setQuery() -" << this->query().lastError();     return false; } m_recordCount = record().count(); return true;
 }
 void SQLiteModel::defaultQuery()
 {
     setQuery("select * from questions;");
 }
 QHash SQLiteModel::roleNames() const
 {
     QHash roles;
 for( int i = 0; i < record().count(); i++) {     roles[Qt::UserRole + i + 1] = record().fieldName(i).toLatin1(); } qDebug() << "Roles: " << roles << "\n"; return roles;
 }
 QVariant SQLiteModel::data(const QModelIndex &index, int role) const
 {
     QVariant value = QSqlQueryModel::data(index, role);
     if(role < Qt::UserRole)     {         value = QSqlQueryModel::data(index, role);     }     else     {         int columnIdx = role - Qt::UserRole - 1;         QModelIndex modelIndex = this->index(index.row(), columnIdx);
         value = QSqlQueryModel::data(modelIndex, Qt::DisplayRole);
     }
 return value;
 }
 void SQLiteModel::addStarterData()
 {
     QSqlQuery preparedQuery(db);
     QStringList lst;
 lst << "Why is the sky blue"     << "What color is the sky on your planet"     << "What makes water wet"     << "Do you drink tea"     << "Is left the opposite of right"     << "If Fred is a boy, what is the name of his dog"     << "Who are you"     << "What do you want"     << "Which way is up"     << "Who knows what evil lurks in the hearts of men"     << "Two wrongs don't make a right, but do three lefts"     << "If boiling has a point, what is it"; preparedQuery.prepare("INSERT INTO questions (Q_TEXT) VALUES(:txt);"); for (QString txt : lst) {     preparedQuery.bindValue(":txt", txt);     preparedQuery.exec(); }
 }
 QString SQLiteModel::getQuestion(int rowNo)
 {
     QSqlRecord rec = record(rowNo);
     return rec.field("Q_TEXT").value().toString();
 }
 /* Qt's view of the world really kind of sucks when it comes to
 relational databases. They want you to do everything within the model.
 A real programmer wants to do everything directly at the database
 and let some watchdog re-run selects as necessary. Admittedly that
 approach is really sucky for the user when a table has a billion rows
 and the query is SELECT * FROM. That approach, however, understands
 that it is not a single user world. In the real world hundreds of
 users will be adding/deleting/modifying records. Each situation has to
 decide how they want to monitor and update the model.
 *
 There are various rules in the documentation as to when Qt will
 update the database. Call submitAll() for each change and remove
 all doubt.
 *
 I put a huge block of comments here because Qt models, at least from
 what I've seen well and truly suck when the primary key is an
 auto-increment integer. You physically have to write to the database
 to get the next key value.
 *
 The ultimate insult here is that
 *
 ONE SHOULD NEVER DO DATABASE I/O IN THE MAIN EVENT LOOP
 *
 This is a constant problem with Qt's design and its developers. When you
 work on real computers with massive distributed databases, you understand
 one query could take as long as half an hour, especially with a "BigData"
 database where many spindles are spun down. This is why banks and other
 institutions make you submit a query when you want something from outside
 the tiny amount they keep online. Later in the day you get a message
 saying your report is ready.
 *
 Adding further insult to injury, the models are busted.
 *
 removeRow called for  3
 *
 result of removeRow()  false
 *
 Error:  "near \"WHERE\": syntax error Unable to execute statement"
 *
 query:  "select * from questions;"
 *
 Every table modification fails with that exact error and you can't dump the
 query to see just where the problem really is.
 *
 */
 void SQLiteModel::deleteRow(int rowNo)
 { 
 if 0
 qDebug() << "removeRow called for " << rowNo << "\n"; bool rslt = removeRow(rowNo); qDebug() << "result of removeRow() " << rslt << "\n"; qDebug() << "Error: " << lastError().text() << "\n"; qDebug() << "query: " << query().lastQuery() << "\n"; submitAll();
 else
 QSqlRecord rec = record(rowNo); QSqlQuery query(db); query.prepare("DELETE FROM questions WHERE Q_NO = :rowNo ;"); query.bindValue(":rowNo", rec.field("Q_NO").value().toInt()); bool rslt = query.exec(); qDebug() << "Result of removing row: " << rslt << "\n"; defaultQuery();
 endif
 }
 void SQLiteModel::copyRow(int rowNo)
 {
 if 0
 QSqlRecord rec = record(rowNo); QSqlRecord rec2 = record(); rec2.setValue("Q_TEXT", rec.field("Q_TEXT").value().toString()); bool rslt = insertRecord(rowCount(), rec2);  // negative appends to end qDebug() << "result of inserting row: " << rslt << "\n"; qDebug() << "Error: " << lastError().text() << "\n"; qDebug() << "query: " << query().lastQuery() << "\n"; if (rslt) {     qDebug() << "submitting changes\n";     submitAll(); }
 else
 qDebug() << "copyRow called for " << rowNo << "\n"; QSqlRecord rec = record(rowNo); QSqlQuery query(db); query.prepare("INSERT INTO questions (Q_TEXT) VALUES(:txt);"); query.bindValue(":txt", rec.field("Q_TEXT").value().toString()); bool rslt = query.exec(); qDebug() << "Result of copying row: " << rslt << "\n"; defaultQuery();
 endif
 }
 void SQLiteModel::updateRow(int rowNo, QString txt)
 {
     qDebug() << "updateRow called for " << rowNo << "  with: " << txt << "\n";
 if 0
 QSqlRecord rec = record(rowNo); rec.field("Q_TEXT").setValue(txt); bool rslt = setRecord(rowNo, rec); qDebug() << "result of update: " << rslt << "\n"; qDebug() << "Error: " << lastError().text() << "\n"; qDebug() << "query: " << query().lastQuery() << "\n"; if (rslt) {     submitAll(); }
 else
 QSqlRecord rec = record(rowNo); QSqlQuery query(db); query.prepare("UPDATE questions SET Q_TEXT = :txt WHERE Q_NO = :rowNo ;"); query.bindValue(":txt", txt); query.bindValue(":rowNo", rec.field("Q_NO").value().toInt()); bool rslt = query.exec(); qDebug() << "Result of removing row: " << rslt << "\n"; defaultQuery();
 endif
 }
 p, li { white-space: pre-wrap; } 
 #include <QGuiApplication>
 #include <QQmlApplicationEngine>
 #include <QQmlContext>
 

 

 #include "sqlitemodel.h"
 

 int main(int argc, char *argv[])
 {
     QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
 

     QGuiApplication app(argc, argv);
 

     QScopedPointer<SQLiteModel> sqmodel( new SQLiteModel(nullptr));
     sqmodel->defaultQuery();
     sqmodel->declareQML();
 

     QQmlApplicationEngine engine;
     const QUrl url(QStringLiteral("qrc:/main.qml"));
     QObject::connect(&engine, &QQmlApplicationEngine::objectCreated,
                      &app, [url](QObject *obj, const QUrl &objUrl) {
         if (!obj && url == objUrl)
         {
             QCoreApplication::exit(-1);
         }
     }, Qt::QueuedConnection);
 

 

     engine.load(QUrl(QStringLiteral("qrc:/main.qml")));
 

     QList<QObject *> objs = engine.rootObjects();
 

     if (!objs.empty())
     {
         QObject *root = objs[0];
         QObject::connect( root, SIGNAL(deleteRow(int)), sqmodel.get(), SLOT(deleteRow(int)));
         QObject::connect( root, SIGNAL(copyRow(int)), sqmodel.get(), SLOT(copyRow(int)));
         QObject::connect( root, SIGNAL(updateRow(int, QString)), sqmodel.get(), SLOT(updateRow(int, QString)));
     }
 

     return app.exec();
 }
 
import QtQuick 2.15
 import QtQuick.Window 2.15
 import QtQuick.Controls 1.4
 import QtQuick.Controls 2.15
 import QtQuick.Controls.Material 2.15
 import Qt.labs.qmlmodels 1.0
 import Fred 1.0
 ApplicationWindow
 {
     id: root
     title: qsTr("SQL Demo")
     width: 640
     height: 480
     visible: true
 minimumWidth: 640 minimumHeight: 480 signal deleteRow(int rowNo) signal copyRow(int rowNo) signal updateRow(int rowNo, string txt) TableView {     id: myView     anchors.fill: parent     TableViewColumn{ role: "Q_NO"  ; title: "Primary Key" ; width: 80 }     TableViewColumn{ role: "Q_TEXT"  ; title: "Question" ; width: 400 }     model: MyModel     Menu     {         id: contextMenu         property int rowNo: 0         property string rowTxt         MenuItem         {             text: qsTr('Delete')             onClicked:             {                 root.deleteRow(contextMenu.rowNo)             }         }         MenuItem         {             text: qsTr('Edit')             onClicked:             {                 console.log("Edit clicked for row " + contextMenu.rowNo)                 promptQuestion.rowNo = contextMenu.rowNo                 promptQuestion.rowTxt = contextMenu.rowTxt                 promptQuestion.visible = true             }         }         MenuItem         {             text: qsTr('Copy')             onClicked:             {                 console.log("Copy clicked for row " + contextMenu.rowNo)                 root.copyRow(contextMenu.rowNo)             }         }         MenuItem         {             text: qsTr('Cancel')             onClicked:             {                 console.log("Cancel clicked for row " + contextMenu.rowNo)             }         }     }     rowDelegate: Item     {         Rectangle         {             anchors             {                 left: parent.left                 right: parent.right                 verticalCenter: parent.verticalCenter             }             height: parent.height             color: styleData.selected ? 'gray' : styleData.row % 2 ? 'white' : 'cornsilk'             MouseArea             {                 anchors.fill: parent                 acceptedButtons: Qt.RightButton                 onClicked:                 {                     mouse.accepted = false                     if (mouse.button == Qt.RightButton)                     {                         console.log("caught right button")                         mouse.accepted = true                         contextMenu.rowNo = model.row                         contextMenu.rowTxt = MyModel.getQuestion(model.row )                         contextMenu.popup()                     }                 }             }         }     } } Dialog {   id: promptQuestion   title: "Edit Question"   standardButtons: Dialog.Ok | Dialog.Cancel   modal: true   property int rowNo: 0   property string rowTxt   TextField   {       id: userTxt       text: promptQuestion.rowTxt   }   onAccepted:   {       console.log("Accepted")       root.updateRow(rowNo, userTxt.text )       promptQuestion.close()   }   onRejected:   {       promptQuestion.close()   } }
 }

Commentary

Too many of you will take this code and paste it into your own projects without even looking at it. You really need to read the comments found in sqlitemodel.cpp at lines 150 through 194.

I didn’t know the models were this busted because I never use them. I only use QML at gunpoint and even then I have to think about it.

The models are a failed architecture because they must run in the main event loop. Never do significant I/O in the main event loop. Yeah, this works nice with a few dozen records. Probably work nice with a few thousand on a fast machine that has plenty of RAM for disk caching and no other users needing to do I/O.

If your database is accross the Internet where you could have upwards of 20 second network timeouts your UI is locked up for that entire time. If your source data is stored on something like an IBM Data Cell;

Or its modern day replacement, the tape library

Tape library robot in action

your UI could be locked up for over half an hour waiting its turn in the I/O request queue. You don’t know the definition of “Big Data” until you are writing applications designed to use that amount of data. Despite what you may think, there are a lot of these still around. Chapel Hill didn’t retire Roberta until 2019.

Roberta at Chapel Hill

The “Data Appliance” which replaced Roberta isn’t going to be oceans faster. Faster, yes. Oceans faster, not really. Has to do with the compressing and de-duplicating.

Here is a tape robot library that just got installed in 2019.

The extra capacity needed to be at least 60 petabytes (PB) – the equivalent of storing more than 10 million DVDs – with the potential to increase again to meet future data demands.

From the above link
The Minimum You Need to Know About Qt and Databases

CopperSpice Experiments Pt. 1

CopperSpice is a fork of Qt 4.8 sans the icky nasty bug infested (and infesting) QML. Lots of bad things have been cleaned up, like replacing UTF-8 under the hood with QChar32 so every character requires same amount of space. Yes, it wastes a bit of RAM, but there is a lot less overhead.

Qt Company has offended the bulk of the Qt developer community with its new licensing stance. Not to mention its attitude about things to be released back to the OpenSource community which provides much of the Qt Application Framework in the first place. From the emails I’ve seen it appears many in the KDE community want to kick Qt Company to the curb without so much as a “horse you rode in on.”

QML – The Hand Polished Turd

Years ago when the current owners of the commercial version of Qt released QML into the universe I declared it a hand polished turd that should be cut from the product. When this idea was originally pitched to me, we were going to get a scripting language that would run through a MOC-like process and generate all of the widget code. That would have been good and righteous.

We already had XML files with .ui extensions created via the Designer. During that time the Designer had some severe issues. It would hose up your XML file so you had to use some other editor to edit the XML and make the Designer happy. It only seemed to do this when it was something you had been really working on. Never seemed to happen on simple screens you didn’t care about. Most of us had gotten pretty good at editing the XML by the time QML came out.

The Reasoning

The owners of the commercial version of Qt wanted to make the product accessible to unqualified developers. There is a great cesspit of developers who read half a Teach Yourself How to Be Totally Useless in 24 Hours or Less book and hung a shingle out as an expert freelance consultant. There language of choice was JavaScript. Yes, this allowed more unqualified developers to work with the product. It also started causing more tragedies out in the industry. These tragedies weren’t blamed on their cause, but on use of Qt itself.

The Problem

The problem is you now have burdened your embedded system with running two virtual machines along with native compiled binary. All three of them are absolutely certain they are in charge of the lifespan of objects. Can you say “random crash?” Knew you could.

One of the big appeals of Qt was the fact it provided garbage collection. One of the big detriments of Qt is the fact it provides garbage collection. One of the big reasons Rust is getting noticed is that it doesn’t have any garbage collection. When you have two different VMs, at least one of which providing garbage collection, combined with a compiled binary language API that provides garbage collection and you pass objects between the three, you have a &^%(*&^ing catastrophe on your hands.

I have seen code run just fine on under powered machines only to die a horrible death as soon as it is loaded on a better machine. All of those ->deleteLater() calls never happened on the box wheezing hard just to run the application. “Idle” never came around. On the properly sized machine your program didn’t even keep one core busy. ->deleteLater() happened almost instantaneously. That’s when you found out the hard way about some shallow copy or reference one the VMs was holding onto.

Those of you who are really lucky will have this happen with your very first demo to upper management using the production hardware.

My snide comments here have just saved you months of trying to track the problem down. You solve it by getting rid of QML and JavaScript.

Birth of CopperSpice

Someone not only listened to me all of those years ago, they did something about it. (Other than ranting where I was heard, I had nothing to do with the creation of the something.)That something is CopperSpice. It started as a fork of Qt, sans all of that worthless QML and JavaScript. Along the way they got rid of the MOC process as well.

Diamond

What really got me interested in CopperSpice was the fact they used it to create the Diamond editor. Is the Diamond editor some fabulous everything to everyone editor like so many try to be? No. It is modeled after Crimson and Emerald. Both of these editors were Windows only. Back when I used to allow the virus known as Microsoft Windows to infect computers I owned I used to really like these editors. They weren’t full blown IDEs, just handy and nice looking.

Next-Part>

Experiments with IUP Pt. 1

It seems rainy days and rabbit holes go hand in hand. I keep trying to get back to writing my GnuCOBOL book (having finished the first draft of the Emacs book) and I keep finding rabbit holes. In particular, I got involved in the GnuCOBOL GUI discussion.

The main version of GnuCOBOL is a transpiler. It translates the COBOL code to C then uses gcc to compile it. There is a C++ fork being worked on that I haven’t played with yet. Given all of the work I’ve done using Qt, I was interested in hearing there was consideration being given to using Qt for the GUI of the C++ fork. At first it sounds cool to have COBOL generate your Qt application, but the more I thought about the massive footprint and “adopt it as a complete religion” view of the framework, I advised against looking at Qt. You can’t “just sprinkle in” Qt. I get phone calls about projects from people/companies who tried to do that. At some point I’m going to play with NanoGUI and probably recommend they use that. It claims to be just a UI without the massive overreach of Qt.

Here is the other reason I would recommend they avoid Qt, even if I didn’t foresee technical problems. Qt Company has really honked off the OpenSource community with their licensing. As such, I expect KDE will be kicking Qt Company to the curb. I suspect they are a bit too heavily invested in Qt to kick it all the way to the curb. Rumblings I’ve heard is something I suggested a while back. A fork of 4.8, ripping out all of that worthless QML then splitting the packaging up. We shall see what becomes of it. There are too many people looking to get involved in such a project, potentially renaming the fork to avoid confusion. Last I heard, all that FLOSS needs is a sponsor. Exactly what a sponsor does or would be on the hook for, I have no idea. I honestly don’t care to know. Whether it is FLOSS or someone else, a fork of Qt is imminent at this point. Maybe KDE will kick Qt to the curb completely and just start with NanoGUI? Instead of one massive overreaching application framework, have a lot of little cooperative frameworks. At this point Qt definitely needs to be a contestant on “The Biggest Loser.”

Anyway, I pulled down iup from SourceForge.

sourceforge screenshot

I extracted the file into a subdirectory under Downloads. In the new subdirectory you find two files of importance:  install   install_dev

The first installs the shared libraries (.so). The second installs the header files and statically linked libraries for development. I ran them both just for grins, running install_dev first.

Like most OpenSource projects, the tutorial is out of date. It seems the only time people are willing to “do the paperwork” is in the bathroom after taking a good Trump and voting at least twice. Between email interruptions and Stevie Wonder leading Ray Charles through the woods, I got the first example to compile and run.

#include <stdlib.h>
#include <iup.h>

int main(int argc, char **argv)
{
  IupOpen(&argc, &argv);
  
  IupMessage("Hello World 1", "Hello world from IUP.");
  
  IupClose();
  return EXIT_SUCCESS;
}

Where the documentation went a bit off the rails was the command line to compile. It appears someone split the libraries into finer groupings. Here is the command line.

gcc -I/usr/include/iup example2_1.c -o example2_1 -liup -liupimglib

For whatever reason the tutorial skips adding the iup library. I’m guessing someone had an environment variable set or what is in iup got split off from iupimglib.

The problem with all GTK based applications is they look like an alien invader or something that stood too close to a nuclear reactor for too long. Then again, so did a quick test of a Qt application on this KDE system.

#include <QApplication>
#include <QMessageBox>

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);
    QMessageBox msg(QMessageBox::Information, "HelloQt", "Hello World!", QMessageBox::Ok );
    msg.exec();
    return a.exec();
}
QT       += core gui

greaterThan(QT_MAJOR_VERSION, 4): QT += widgets

CONFIG += c++11

# The following define makes your compiler emit warnings if you use
# any Qt feature that has been marked deprecated (the exact warnings
# depend on your compiler). Please consult the documentation of the
# deprecated API in order to know how to port your code away from it.
DEFINES += QT_DEPRECATED_WARNINGS

# You can also make your code fail to compile if it uses deprecated APIs.
# In order to do so, uncomment the following line.
# You can also select to disable deprecated APIs only up to a certain version of Qt.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0

SOURCES += \
    main.cpp

HEADERS +=

FORMS +=

# Default rules for deployment.
qnx: target.path = /tmp/$${TARGET}/bin
else: unix:!android: target.path = /opt/$${TARGET}/bin
!isEmpty(target.path): INSTALLS += target

Changing the icon requires just a touch more finesse under Qt.

#include <QApplication>
#include <QMessageBox>
#include <QIcon>

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);
    QMessageBox msg(QMessageBox::Information, "HelloQt", "Hello World!", QMessageBox::Ok );
    msg.setWindowIcon(QIcon::fromTheme("system-run"));
    msg.exec();
    return msg.exec();
}

It isn’t fair to say I cannot find one in iup. I haven’t really dug into anything yet.

Compiling Qt 5.14 Under Msys2

I went down this road because I wanted to compile one simple test program so I could file a bug with GnuEmacs about how they don’t catch NumLock. This is a really old bug and they seem willing to let it rot until computers cease to exist. My test program was a KeyEvent example I stole from online then added support for NumLock. It took longer to scrape from the Internet than it did to test under Linux.

keypress.h

/*
 * stolen from http://programmingexamples.wikidot.com/qt-events
 * and modified for Qt 5.14
 */
#ifndef KEYPRESS_H
#define KEYPRESS_H

#include 

class QLabel;
class QVBoxLayout;

class KeyPress : public QWidget
{
    Q_OBJECT
public:
    KeyPress(QWidget *parent = 0);

protected:
    void keyPressEvent(QKeyEvent *);
    void keyReleaseEvent(QKeyEvent *);

private:
    QLabel *myLabel;
    QVBoxLayout *mainLayout;
};

#endif // KEYPRESS_H

keypress.cpp

/*
 * stolen from http://programmingexamples.wikidot.com/qt-events
 * and modified for Qt 5.14
 */
#include "keypress.h"


#include 
#include 
#include 

KeyPress::KeyPress(QWidget *parent) :
    QWidget(parent)
{
    myLabel = new QLabel("LABEL");
    mainLayout = new QVBoxLayout;
    mainLayout->addWidget(myLabel);
    setLayout(mainLayout);

}

void KeyPress::keyPressEvent(QKeyEvent *event)
{
    if(event->key() == Qt::Key_Escape)
    {
        myLabel->setText("You pressed ESC");
    }

    if (event->key() == Qt::Key_NumLock)
    {
        myLabel->setText("NumLock was pressed!!!");
    }
}

void KeyPress::keyReleaseEvent(QKeyEvent *event)
{
    if(event->key() == Qt::Key_Escape)
    {
        myLabel->setText("You released ESC");
    }

    if (event->key() == Qt::Key_NumLock)
    {
        myLabel->setText("NumLock was released***");
    }

}

 

main.cpp

 

/*
 * stolen from http://programmingexamples.wikidot.com/qt-events
 * and modified for Qt 5.14
 */
#include 
#include "keypress.h"
#include 

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);

    KeyPress *keyPress = new KeyPress();
    keyPress->show();

    return a.exec();
}

It puts up a tiny little window. When you press the [Esc] key it displays a message. Another message when you release the key. I added a pair of messages for NumLock. The reason I went through all of this pain is I wanted to prove to these people that Qt can do it under Msys2/Windows and Linux so why can’t you just look at the code in Qt to see how they are intercepting key events. Since Qt is OpenSource, the code is there for all to see.

I had hoped there was a pre-compiled Qt development package in the Msys2 environment. After installing C/C++ compilers and a few other bits, my hopes were dashed. Adding insult to injury Qt currently only has a mkspec for 32-bit mingw.

Note: I don’t install the virus known as Windows on a bare machine as the primary OS. If I have to do anything with Windows I install it in a VM. Nobody should let that be a primary computer operating system. At least not on a computer they care about.

Open a terminal in Msys2 and type the following:

pacman -S base-devel git mercurial cvs wget p7zip
pacman -S perl ruby python2 mingw-w64-i686-toolchain mingw-w64-x86_64-toolchain

pacman -S clang
pacman -S mingw32/mingw-w64-i686-clang
pacman -S mingw64/mingw-w64-x86_64-clang
pacman -S mingw32/mingw-w64-i686-clang-analyzer
pacman -S mingw64/mingw-w64-x86_64-clang-analyzer
pacman -S mingw32/mingw-w64-i686-clang-tools-extra
pacman -S mingw64/mingw-w64-x86_64-clang-tools-extra
pacman -S mingw32/mingw-w64-i686-compiler-rt
pacman -S mingw64/mingw-w64-x86_64-compiler-rt
pacman -S mingw32/mingw-w64-i686-libblocksruntime
pacman -S mingw64/mingw-w64-x86_64-libblocksruntime

(The clang stuff is mostly to get the llvm stuff Qt is looking for.) Exit out of that terminal and open a new terminal and type

pacman -Syu

You will need to crash out of the terminal rather than typing exit once this is complete. Open a new terminal and type

pacman -Su

to complete installation. Depending on the quality of your Internet connection and the time of day, this could run a long time. You may want to add

 

 --disable-download-timeout

to your pacman command. It will take even longer because it won’t be skipping the slow/sucky mirrors, but it should not fail to download. The timeout feature of pacman seems to have been written by Twitter users. It fails with a message about no bytes within 10 seconds, but only takes 2-3 seconds to do that.

If something happens and pacman crashes (happened to me once) don’t panic. There is probably a dump file, but you don’t care about that. Wait a few seconds and retry the command. If you see something like this

rolan@DESKTOP-HTIGNPK MSYS ~
$ pacman -S mingw-w64-i686-qt5
error: failed to init transaction (unable to lock database)
error: could not lock database: File exists
  if you're sure a package manager is not already
  running, you can remove /var/lib/pacman/db.lck

Do this

rolan@DESKTOP-HTIGNPK MSYS ~
$ rm /var/lib/pacman/db.lck
rm: remove write-protected regular empty file '/var/lib/pacman/db.lck'? y

Theoretically you are now ready to configure Qt. I unzipped Qt in a shared directory so I had to cd here:

msys2 terminal prompt

Paste this in at the prompt.

windows2unix() { local pathPcs=() split pathTmp IFS=\;; read -ra split <<< "$*"; for pathTmp in "${split[@],}"; do pathPcs+=( "/${pathTmp//+([:\\])//}" ); done; echo "${pathPcs[*]}"; }; systemrootP=$(windows2unix "$SYSTEMROOT"); export PATH="$PWD/qtbase/bin:$PWD/gnuwin32/bin:/c/msys64/mingw64/bin:/c/msys64/usr/bin:$PATH"

You need more of the path than you typically get. That takes care of it.

 

./configure -opensource -confirm-license -release -skip qtwebengine -skip qtvirtualkeyboard -skip qtlocation -opengl desktop -qt-sqlite -qt-zlib -qt-libjpeg -qt-libpng -qt-freetype -qt-pcre -qt-harfbuzz -nomake examples -nomake tests -prefix /usr/local/bin/qt-5-14-1 -platform win32-g++ -silent 

Keep in mind I’m working with 5.14.1. You may want a different install directory. Neither qtvirtualkeyboard nor qtlocation will compile on msys2 so you definitely need to skip them. You might want to skip the 3d too if you have no plans on using it. In a VM assigned 2 CPUs and 8 Gig of RAM it took something like 6 hours to compile.

If configure spits up about some missing dependency, don’t panic. Simply find the dependency and install it. Remember, you need the 32-bit version for this build of Qt, but you might as well install both the 32 & 64-bit versions if you have the disk space.

Before running configure again you should delete these files.

$ rm config.summary
$ rm .qmake.cache
$ rm .qmake.stash
$ rm config.cache
$ rm config.log

A nice clean configure will end like this without any messages about missing things or errors.

Note: No wayland-egl support detected. Cross-toolkit compatibility disabled.

Qt is now configured for building. Just run 'make'.
Once everything is built, you must run 'make install'.
Qt will be installed into 'C:\msys64\usr\local\bin\qt-5-14-1'.

Prior to reconfiguration, make sure you remove any leftovers from
the previous build.

Note: If your build fails and you decide to configure differently, you need to run “make clean” before you delete the files mentioned above.

mingw32-make -j 4

I only had 2 CPUs assigned to this VM but I still used 4. Please note that you need to use mingw32-make. Hours later, if your build seems to go fine

mingw32-make install

If all goes well, not a given since make install seems to build a bunch of stuff the actual make skipped, you can exit that terminal window. Don’t try to use it for anything else because you’ve jacked with the path.

path after install

Note that Msys2 automatically puts /usr/local/bin in the PATH. It’s just not polite though.

msys2 path image

You have to edit your .bash_profile

bash_profile image

Things get better after you close your terminal window and open another.

image after qmake

after make image

msys2 success image

It runs really really slow launched from the command line like that. (Probably doesn’t help being in a 2 CPU VM!) Yes, I could probably fix my Windows environment to make things better, but that wasn’t the point of this exercise. There for everyone to see is the release message from NumLock having been pressed.