THE GUI SHOULD BE BETTER

I know this thread is about the desktop GUI and not about individual applications, but I thought I mention blender nonetheless. A lot of people hate on blender because it dares to look different, which I don't understand. I love blender's UI. I wish more software would be like that.

 

These kind of programs have a huge amount of functions and it can be cumbersome to search for them in the menus, especially when you know the name of the function, but not in which nested menu it might be. In blender there's a way to search for commands (in the context of the current window). It will give you a list of results that also shows where in the menus you can find it and the hotkey (if there is any). (Note: This is in the current alpha version, so the contrast issue there might be fixed until it's released.)

 

You can scale everything or separate panels. You can squeeze it all into a small margin or five it as much space as you want.

 

Blender is basically like a tiling window manager and you can organize the tiles in whatever way you want. It comes with a few presets (the tabs at the top), but you can add your own (and delete the default ones) yourself as you like. You can also create new windows and put in whatever views you want. You know, for multi monitor setups. (Here I just show the integrated Python shell that has of course auto-completion.)

 

For developers you can activate tooltips that show you how to access the hovered function/value via the Python API. A really quick way to find that out without having to read a programming reference.

 

You can "paint" over checkboxes. In other UIs if you click a checkbox and then drag, well, nothing much happens. In blender you can mass select/deselect checkboxes easily by clicking one checkbox, holding and dragging the pointer over all other checkboxes that you want to toggle.

 

Context action help. In this screenshot I pressed "g" (for grab) and now it shows me all the more refined things I can do from here by pressing another key.

 

Of course there are also a lot of things that can be improved, but I just thought to mention these things as an inspiration of what an UI can be. In particular I hate how clunky Inkscape is. Certain side panels take away so much screen space and you just can't scale them down! WHY!?

 

Oh and in the image with the Python window you see my KDE window manager. I've put two extra buttons into the window bar: Pin on all desktops (the pin) and keep always on top (the double arrow up). These are very very useful buttons. I can just open a small image viewer and put it always on top so I can have it hovering over a maximized Blender/Inkscape window to work from the reference. (Or a video player to watch something on the side. I don't have a multi monitor setup.)

I'm not sure if anyone's suggested this yet but Would it be possible to build your new UI on top of even the restrictive modern versions of windows using some sort of remote desktop app? Like, if you were to just put a Raspberry Pi or arduino between your monitor, mouse and keyboard and your desktop, you could have whatever shell you wanted installed in that that interacts with windows on its level but lets you interact with it however you configure it? 

1 minute ago, mattihase said:

I'm not sure if anyone's suggested this yet but Would it be possible to build your new UI on top of even the restrictive modern versions of windows using some sort of remote desktop app? Like, if you were to just put a Raspberry Pi or arduino between your monitor, mouse and keyboard and your desktop, you could have whatever shell you wanted installed in that that interacts with windows on its level but lets you interact with it however you configure it? 

Only at serious speed costs.

11 hours ago, Ross Scott said:

Also, your desktop is a perfect example of the problem I notice visually with a lot of themes.  Your terminal windows look pretty slick, dark theme, but readable colors, looking nice.  Then BAM your web page is bright as hell in comparison and just overpowers the rest of it.  My solution was to go for something in between, but maybe forcing everything to be dark on the web with plugins could work too.

It was my desktop he referenced; and the nice thing about tiling WMs on Linux (aside from the fact that dwm + dmenu resolves all your issues) is that I can segment tags by color if I like then only merge them if I need, or I can just switch between them. I'd recommend you check out tiling WMs though, since they *can* be configured to be all light mode and additionally allow you to resolve essentially all of your issues (with the caveat that learning minimal POSIX shell is necessary)

shocked i didnt already make an account here ages ago as ive been following ross since around 2013 but whatever, this vid was particularly cathartic for me as someone that's been poking at the windows shell's guts for years and i still feel like going on a tangent

 

maybe its just my tism acting up again but i dont actually agree with a lot of your views regarding the *perfect* desktop gui - i dont need mouse gestures and so on, i actually like the standard windows 9x desktop paradigm. it isnt perfect and it can absolutely be improved upon (for a short time i used bbzero/bblean for windows and that comes with the right click start menu-esque thing most linux window managers do and i actually liked it a lot, although i still just open a lot of programs straight from explorer... which i should probably fix), but ive stuck to it since i was a squirmy little embryo and it generally works for me. i even love the raw beveled "classic" windows 9x theme you derided really early on. however every single thing you said about large software manufacturers (coughmicrosoftcough) gradually removing customization options and crippling their products in the name of """accessibility""" was absolutely spot on. this has been a widespread practice for the last decade at least but im going to illustrate this with an example thats been killing me from the inside for yeeeears:

 

 

and multiple people in this thread and the youtube comments and so on have already alluded to this (grrr...) but the "circle launcher" concept you brought up is called a pie menu and theyve been around in some form since the late 60s, and yes, don hopkins has advocated for them for yeeeears and he was the dude at maxis responsible for adding them into the sims. this is what i get for always being late to the party..

 

edit: whoops!! i forgot about stardock and all of their windows customization tools and that ross explicitly mentioned them several times in the video. i really really dislike them and i wouldnt recommend their software at all to anyone, not just because its paid proprietary software that barely works (well, so is windows, but... y'know...) but as far as i can tell, they actually changed their eula last year. if you buy ANY stardock product, you get to activate "one (1) copy of the Software on one (1) unique personal computer" and thats it! im not even going to try explaining why this is a vile fucking business practice, it should be blatantly obvious. just dont bother with stardock.

I don't have time at the moment to read the other comments, but I will mention a couple things. Apologies if they've already been brought up!

 

Part of what makes Windows (and the other commercial OSes) GUI shell experiences so bad is a philosophy: that there is a distinction between the system author and the system user. The Xerox PARC Smalltalk system that originated a lot of the still-modern ideas about the UI did not make this distinction: everything was editable by the user, in realtime, constantly. If you didn't like the menus, the buttons, the layout of the program, all of it was up for grabs by you. You can experience this today in Squeak or Pharo, but of course, this is niche technology.

 

Another forgotten mouse concept from Plan 9 is mouse chords. If you click button 1 and then add button 2, that creates a 1-2 chord; there is also a 1-3 chord. I believe on Plan 9 you can do cut/copy/paste just with the mouse using these chords. Plan 9 in general has a lot of weird and interesting ideas about computing, but this seems like the only one that would resonate with Ross, based on the video.

 

This actually reminds me of chord keyboards. Even back in 1968 in Doug Englebart's famous "mother of all demos," he's using one of these. Most of these, unlike the keyboards we use for text entry, there usually are not too many other ways to put your hands on the device, so there shouldn't be as much "n"/"m" confusion.

7 minutes ago, Daniel K Lyons said:

Part of what makes Windows (and the other commercial OSes) GUI shell experiences so bad is a philosophy: that there is a distinction between the system author and the system user. The Xerox PARC Smalltalk system that originated a lot of the still-modern ideas about the UI did not make this distinction: everything was editable by the user, in realtime, constantly. If you didn't like the menus, the buttons, the layout of the program, all of it was up for grabs by you. You can experience this today in Squeak or Pharo, but of course, this is niche technology.

The Genera operating system has similar ideas, although it was written in Lisp.

So, I am not a GUI Wizzard, nor I want to be. Programmable mouse and second monitor allow me to perform most of the task as fast as I can process them anyway. But, if I were to design "perfect" GUI, it would not require lifting your hands most of the time. All thanks to using all three of modifier keys,  mouse gestures and radial menus - because all of these are options multiplayers, and they reach their true potential only when combined.

 

First - modifier keys. ALL they do is modify and allow other stuff - if you press one by accident, nothing happens. Two of them should be enough - but you can have more, as long as they are in comfortable positions. I choose two because that's how many legs most of us have. Legs that, you guess it, could press these two extra keys. One at a time, or both - all comfortable. That gives us 4 possible states - one for "normal" use and 3 for all the fun.
 

Second - we use gestures. Because each gesture is actually 3 gestures depending on modifier keys, we don't need fancy ones. Just simple directional A-B and A-B-A, approximation to one of 8 cardinal directions, is 48 very unique, very easy gestures. We could have more, but we don't need them. That's because even after we reserve some for things like copy and and paste and other always useful stuff, we still have plenty to bring up...

 

Third - radial menus. Even after mapping most used functions, we still have plenty of gestures to use. Instead of spending them on single thing, we instead use each one to bring up different radial menu. Because that gives us so many combinations we can keep the menus small - especially if we allow nesting and scrolling and other types of menus when there are too many options. And they will be all annotated at all times, with nice icons and all that. Because we can do that - they will only show when we want them to show.

 

Now, the problem can be memorizing what radial menu has what. There are two solutions - clickable sidebar that you can toggle any time, that also functions as hub for easy modification of these menus. And... simply not using that many at first. Sure, this is basically total replacement of normal GUI, but nothing stops you from having both installed and slowly switching as you learn. You don't need quick access to every program and every file on your computer.

 

So, this is nice GUI with minimal modifications to hardware (you can just use mouse with 2 programmable buttons). But these two buttons can not only be used for a gestures - they can also be used with keyboard. Not so you can have over 400 keys (I mean, if you want, sure). It allows you to START smaller. People are typing with just a few fingers every day - One (supported) hand, after some practice, will suffice for most but especially typing-heavy applications, like programming. But even there, modifiers under your feed allow you to always keep your hand where you want them - or to make even more shortcuts, for even faster computer use. And if you hate mouse so much, you can use other input device. The implementation is not so important - it's all about amplifying already existing techniques.

Also have to mention the KDE file chooser. I really like it. It looks quite normal, but has many nifty little features that I'm missing in Gtk programs. I use Firefox for several reasons and am really missing how Chrome integrates in the Desktop. Firefox just doesn't do that. There is a thing, forgot it's name, some common desktop service thing that you can install that makes Gtk applications use the KDE file dialog, but it messed up seemingly unrelated parts of my system and crashed a lot of things, so I removed it again.

 

The KDE file chooser has several nice options to view your file system. A tree view and a thumbnail view where you can zoom the thumbnails. (Under Gtk you only get teeny tiny icons that are totally useless as thumbnails. Sometimes you get a preview in the right panel when you click a file, but randomly sometimes you don't. Whyyyy?)

 

A program can define a preset of file filters (just like in Windows), but you can also just type there and filter for whatever globbing expression you want.

 

You can edit the bookmarks on the side easily. (Last time I used Windows (XP) that wasn't possible at all.) You can set this as global bookmark or only as bookmark for the given program only! Of course you can set a custom icon if you want.

 

You can navigate folders in the address bar by clicking, nothing special there.

 

But you can also click after the last path entry and start typing! E.g. if you're quicker by typing or so you can paste a path. (Can't do that in the Gtk file chooser.)

 

Have you noticed the question mark in the title bar? When you click it and the hover certain UI elements your mouse pointer changes to this.

 

This means there is a help text for that and with another click you get that help. Sadly not enough KDE programs make use of that feature and those that do aren't rigorous enough.

 

And yes, there is auto-completion in the filename input field. Nothing special, or so one would think: The Gtk file choose HAS NO file name input and as such no auto-completion. You can just start typing there and it starts to search for files recursively, but I don't want that. I don't want to be overwhelmed by all the matching files in some subdirectory. Most of the time I want to search for a subfolder to navigate to and I get all these files from wherever!

 

Sorry, started with praising the KDE file chooser and ended up bashing the Gtk file chooser. But to be fair, there is one feature I'm really missing in the KDE file chooser: recently used files. Gtk has that! And it is even cross toolkit! So KDE should have the ability to access the list of recent files too, but doesn't show it in its file chooser. Welp. Rant over.

More things I like about KDE:

 

You can force your will on program windows. First you match windows by certain properties.

 

Then you force other properties.

 

And more properties.

 

And even more.

2 hours ago, sulupickles said:

Only at serious speed costs.

True, that would be a significant problem. But still, I feel like there ought to be a way to get a second OS working over windows running your custom shell but deferring to the Windows OS when it comes to the matter of running software. I guess it'd have to be more involved than that to avoid slowdown. I feel like there's bound to be a software solution too here... 

 

 

On 6/10/2020 at 8:04 PM, Ross Scott said:

If you ever wanted to know all my thoughts on the GUI, here you are!  This has honestly been brewing in my mind for decades and while this video took way too long to make, it's an accomplishment for me that I was able to put this into something coherent.  I'm really hoping this leads to somebody bestowing GUI enlightenment upon us, though I'm not betting on it.

This post also doubles as a thread for people to post any helpful information regarding my GUI quest at the end of the video.  Thanks in advance for anyone who finds some answers!

 

This is a blog post. To read the original post, please click here »

 

 

 

These are really good ideas. Ross, or someone who's willing, please write exact design white-paper with visual examples for FreeDesktop.Org, ESPECIALLY radial menu concept that would work wonders for mice, gamepads, touchpad, gesture-controls, all point-type devices, so they could be expressed in Wayland protocol specifications and implemented in common ways among W-compliant WM/DEs that are supposed to replace entire GNU/Linux & F/OSS GUI infrastructure of the past, that is based on X11. It can be posted https://gitlab.freedesktop.org/wayland/wayland/-/issues  for programmers to figure out. Points about proper palettes should also be relayed to GUI design committees of common DEs, such as KDE and Gnome, they indeed deliberately copying stupid crap from Windows and MacOS. Campaigning for them to stop that and start thinking again would be nice.

 

Although, it's possible that all underlying infrastructure already exists and the problem is visual design and implementation. By the way, Wayland allows creating fully 3D DEs, video-game style, not just window layers. But we would need actual experience VG UI designers to achieve that, like ones working on economical strategies with big data-sheets and complex controls. There were notable 3D DE experiments, such as Project Looking Glass that Steve Jobs smothered, SGI's IRIX and Metisse.

 

And here's some obligatory F/OSS DE goodness (if you can call degradation of KDE 3 to 4 and 5 that) bragging (wallpapers are upscaled with legendary waifu2x-converter-cpp with OpenCL GPU acceleration):

* KDE3 with amaroK audio manager on Qt3:

* KDE3 on 16:9 display with Quake-style terminal:

* LXQt with jamin audio processing on JACK:

* KDE4 with Conky with "now-playing" menu for Clementine, Qt4 amaroK fork:

* KDE5 on Qt5 featuring Calf & LSP JACK plugins and Strawberry, Clementine fork:

I love the GUI of Haiku, for example it has a couple of cool features

 

Tabs:

 

 

Right click menus everywhere:

 

 

E-Mails, contacts, documents or anything you want in your filesystem (and not just in any silly extra program):

 

Dragging and dropping elements (like system monitors, calculator, volume control etc.) from other programs to the desktop and pinning them there:

 

And more tools and themes to rice your desktop to your likings:

 

It reached Beta 2 two days ago and it's definitely worth trying out (even if it needs some polish). But I think this one beats any operating system GUI-wise so far (and it isn't heavily terminal-dependant like Linux). And yes, it's its own little thing, no Linux, Windows or Mac thing (hence driver support is a little bit spotty right now).

Haven't read the thread so I don't know if anyone has posted this yet, but I was looking into how to make my phone and PC look more like TNG era Star Trek computers. I found these:

http://lcarsx32.org/ - Looks like development ended on this one a few years ago. Features "Computer" as a voice activation line to preform different functions with the ability to add your own. Very clunky. Star date is set as the default for the clock, and that's in the negatives right now. Looks like the guy also had a youtube channel.

I was also really interested in what kind of results I could get with my phone given what I see online.

EDIT: Some people have even done this with their cars. I don't know if I would go that far on a vanity interface. It has potential but suffers from some of the same issues as modern GUIs.

There's a major problem with GUIs as they exist today: a lack of composability, or the ability to connect multiple programs together or assemble larger blocks of functionality from the smaller ones you already have and understand. This is something the CLI gets right, and it's one of the big reasons people tout CLIs as having superior efficiency.

 

In a composable interface, I might only ever need to learn a few searching/scanning/data extraction tools really well. Instead of learning yet another search interface, or worse yet resorting to a program's builtin find-and-replace dialog, I could just plug my favorite tool in and use that. For a less text-oriented example, imagine taking a video lecture, filtering it through a video-processing program to extract the slides, and sending the images directly into another program to assemble the resulting images into a PDF. No saving intermediate files to disk or manually dragging things around, just wiring them together and letting the data go from one program to another. Finally, once you've got it perfect, it'd be trivial to make the whole chain into a new "tool" you could use later on - maybe right-click a Youtube link, run it through the "get video, filter, make PDF" chain, then print the output.

 

In most GUIs I've seen, these functions are all separate programs that can't really talk to each other aside from saving and loading files. There's certainly no easy way to package up a collection of programs, their options, and the connections as a new building block. Essentially, the problem is that applications are special - you can't connect them together to make a new application without programming and a lot of work.

 

CLIs are great at composability. Connecting the output of one program to the input of another is so fundamental to the Unix paradigm that there's a character dedicated to it. They're by no means perfect - discoverability is not one of the strong points - but the ability to put together small, easily-understood parts like Lego to get the functionality you want outweighs the drawbacks for a lot of people, myself included.

 

You could have a perfect GUI by all the standards Ross sets out in the video, but if it didn't solve this problem it'd still end up being necessarily inefficient for a large class of tasks. He mentioned in passing that he sometimes spends large amounts of time searching for and evaluating programs to accomplish a specific task; the fact that this is common enough to mention is a perfect example of what I mean. With a composable system it wouldn't go away entirely, but it'd be less frequent.

 

I don't have as much direct investment in improving GUIs as some people since I'm a developer and live in the terminal or browser 99% of the time, but I figured I'd contribute my two cents anyway. Besides, if there *is* a solution to GUI composability I wouldn't object to a nicer computing environment .

Hello, I'm a KDE developer. Just wanted to pop in, share a bit of my workflow and say it's cool to see all the love for Plasma here.

 

Re: CLI:

CLI doesn't have to be as bad as cmd.exe. It's currently not as good as it could be, but it could get there and you can already do much, much better than cmd.exe or bash.

 

I actually like to combine CLI and GUI in one with Dolphin's terminal panel (Dolphin is KDE's main file manager):

 

You see all those `cd` (change directory) commands? I'm sure some of you know that from cmd.exe or Linux, but I'm not actually typing those myself! Instead, when I change directories in the GUI, the terminal changes to the directory I'm looking at. Pretty sweet, right? Also, see how my terminal looks fancy? It can do autosuggestions based on command history when I start typing, do completions when I press tab, do syntax highlighting and more. With all that, it's not 100% perfect, but using the CLI is actually a pretty pleasant experience for certain tasks. BTW, the CLI shell I'm using is called Zsh. There are tons of plugins for it, but it's not that nice OOTB. There are newer shells like nushell and elvish out there.

 

Could a lot of it be replaced with a GUI that does something equivalent? Yes and sometimes the only reason why I use the CLI is because there are no good GUI alternatives. For example, most GUI frontends to Git (source code version control system) have major flaws. There are still advantages to the CLI though.

1. Repeating the same tasks over and over is extremely easy. A GUI can't really ever match its efficiency at doing that.
2. Unlike GUIs, CLI tools often can be made to work together, allowing you achieve more with fewer programs. Info is often text that can be passed around and modified freely. It would be very difficult to design a GUI ecosystem that works as well with passing around and modifying text.

Ways CLI could be better:

• Better autosuggestions: why not make them context based? Why not make it possible to compose new combinations of commands only with autosuggestions? Think about what your software keyboard does on your phone. It could be like that, but even better because CLI isn't as complex as English.
• Why such a hard divide between CLI and GUI? Dolphin's terminal panel gets a bit closer, but it's still completely separate in every other way. Although, we did recently add the ability to see thumbnails for files when you hover over file names in the terminal!
• CLI and voice commands are not so distant relatives. Not sure if I'd actually want to use it, but I see no reason why you couldn't use voice commands to do what the CLI does.

To be clear, I'm not saying that everyone should just use the CLI and that GUIs don't need improving, just that you shouldn't dismiss the CLI as old and automatically bad. It's still in use for very good reasons and not just because of bad GUIs.

I love this kinda thing! I love to skin my programs and make them look and behave the way I want. I was happy to see foobar in the video since that's one of my favorite programs for this.

 

Someone mentioned Haiku--I also love Haiku!

 

My dream UI brainstorm session:

 

I love to use keyboard shortcuts for most of my computer usage. Mouse is inefficient from my point of view. One big issue I have with the way we usually design keyboard shortcuts is that they require you to press multiple buttons at once, which a keyboard isnt designed to do, and also is often hard to do one-handed.

 

So my solution to this problem is key sequences. I remember playing an FPS game that had many voice commands bound to sequences on the numpad. This allowed the game to have dozens of options that were all accessible in 2 or at most 3 key strokes. Not only that but as you were playing you would learn the most used commands by heart VERY quickly. (I thought it was Return to Castle Wolvenstein that had this but I can't find any confirmation online.)

 

Perhaps this principle could be used in UI design as well?

ie. press numpad 0 to get a list of commands: 

1 -> firefox (browser)

2 -> foobar (music player)

3 -> directory opus (file explorer)

4 -> photoshop (image editor)

5 -> all programs

6 -> settings

...

9 -> [current program context, for example 1-minimize, 2-maximize, 3-close]

 

by doing this, you can start to do super quick one-handed shortcuts. they could even be context/program-sensitive so for example:

0->2(music player)->3(next song)

or

9->3 (close)

 

alternatively each number could contain a number of programs, so:

1(browsers)->1 firefox, 2 vivaldi, 3 opera

2(music players)->1 foobar, 2 winamp, 3 xmplay

this could be configurable!

 

it's really fast to type those kinda codes in and they will become second nature. It could be accompanied by a visual as well that shows a list of items, perhaps it could be scrollable with page up/page down or arrow keys in case you don't know what number you want. You could even include a search option as well.

 

If I think about it, you could also do this with a numpad on the side of a mouse, if you didn't want to take your hand off the mouse.

 

I hope this makes sense...

 

... Does anything like this exist? I really want this now.

I'd be really interested to see you try out tiling window managers for Linux. They're basically exactly the idea of creating a new type of GUI from the ground up without worrying about looking or feeling like Windows. I think the result is different than the direction that you would be looking to go, but they're still amazingly huge steps forward over Windows, and I think seeing how other people approach the problem would be helpful in coming up with more concrete ideas.

13 hours ago, Eisen said:

Hey, i'm an amateur pixel artist. While i can't chime in too much, I did do something kind of strange that may seem interesting to some folks here.

 

 

Hey, a pixel art theme could be pretty awesome for the OS.  I'm a fan of that style.  When editing the video, I found myself wishing my OS looked like I was living in the racing game world at the end.

I can drop a bit on Dvorak and for anybody who's interested.

 

If you're really enthusiastic about keyboard layouts and keyboard design, then you should probably learn Dvorak and/or other layouts like Colemak and Workman and such simply because you'll find it fun -- hard work, but fun. Similarly, if you care a lot about the feeling of typing (like you're collecting keyswitches and such), then it's almost rediculous to not be well-versed in a second layout (if for anything but the fun of it). Otherwise.... there's not really anything in it for other people.

 

Personally I started using Dvorak because I needed to break a lot of bad habits really quickly. My home row was Shift+WAD and JIOP/arrow keys, one of my thumbs anchored itself awkwardly on the front edge of the keyboard, and over the course of a year I was starting to get a really burning in my wrists after typing (noting here that I'm 20 years of age and an aspiring programmer, go-figure). Learning a proper, clean way of using the keyboard doesn't get rid of the build-up of pain, but over the course of two years it's far milder. (Because of this, I also use a 42-key Atreus keyboard so I don't have any far-reaching stretches when I type.)

 

If you want to type more productively, then learning a new keyboard layout isn't for you -- unless maybe you want to get into stenography, but that's a whole separate skill. Faster typing is a product of practice, and a new layout is going to set you back with a 100 hours of awkwardness. Then there's the technical problems: changing your keyboard layout is a native function of Windows and Linux and BSD and such, but all the default shortcuts for programs are meant for QWERTY, not all of those shortcuts are re-mappable, and some programs are going to ignore your OS's selected keyboard layout and will try and parse the keyboard scancodes with their own half-assed methods. Depending on whether or not you're using a native Dvorak keyboard especially messes with this stuff.

 

The final note I'd like to make about this is that after using Dvorak full time for a month, I completely forgot how to type in QWERTY. To other people I'm like this big computer expert, but I sit down in front of their computers and I have to stare face-down into the keyboard and type with one or two fingers.