THE GUI SHOULD BE BETTER

[Im_CIA]

18 minutes ago, ekket said:

Lastly, I do appreciate you fighting the fight against entropy on this; don’t let chaos win out!+

 

But it has already won. Your only solace is in those hippy-dippy open source communes.

[AtomicPurple]

I created an account just to post in this thread because Ross's video made me realize why CLI is more efficient than GUI for some tasks.

I'm on Linux and prefer to use a GUI for most things, but there are a few thing that I prefer the command line, not because it's particularly fast, but because it's faster than navigating through multiple menus/folders. That's because the terminal is the only program that has a system integrated working directory model.

 

If I have some files in a folder I want to convert, I can right-click in the file manager, click open terminal here, and type the command to batch convert everything in the folder. If I use a GUI interface for the same tool, I have to click to open the program, click the open directory button, then click a bunch of times to navigate to the directory I already had open, and THEN I can start converting.

 

Dang-and-drop between windows helps with this problem, but not all programs support it (looking at you filezilla). A much better solution, I think, would be to have every program open with  the working directory set to the path of whatever file browser window is open and in focus at the time it's launched, if there is one. For example, if a flash drive open in your file browser and you launch a word processor, then clicking save would save the document directly to the flash drive. For my file conversion problem I could just open the program when I'm, browsing the files I want to convert, and hit one button, it would be way faster than CLI.

 

Of course something like this would require applications to coordinate across the entire system in a way that they currently can't/don't, and I can see lots of potential of misplaced files, so maybe make the behavior optional, and display the working file path in all window titles when it's active. I don't really know, just my two cents that maybe someone can use as inspiration.

 

Edited June 10, 2020 by AtomicPurple (see edit history)

[FuzzyCandle]

Not sure if a lot of people here are talking about the ideal future IU. The conversation is mostly on Linux superiority (natch) and improving UI with our current hardware. That's healthy, we shouldn't stop that. I'd just like to dump a long feverish description of my ideal future UI (likely with notes and edits as I go) right here for some consideration. Mostly, I really think the problem isn't with software. It's the mouse, which we created in the 20th century and just never decided to change for whatever reason, alongside the keyboard which is pretty much ideal for data entry but shortcuts and rapid navigation is just not in its wheelhouse.

 

The idea of haptic gloves has been around for basically forever, but there are a few things I think no one has ever considered with it. Basically, we have gloves that can navigate in full three-dimensional space and can have your fingers in literally any configuration yet we're still tied down to the idea of having a long flat rectangle that you need two hands to type on. Imagine, though, a keyboard that _is_ that wheel around the mouse, except there's one around each finger with haptic feedback for which key you're hovering over.

 

Spoiler

 

I wish I was artistic enough to make diagrams but the gist is that one finger (in this example your index finger) would be hovering near the obligatory desk (since desks are useful for hundreds of things, not least of which is drinks and food, the desk would still be there) You'd have a neutral position that you calibrated relative to the position of your thumb in space. Not any pre-set virtual space! Wherever your thumb was near the desk your keyboard would be there. Turned slightly to the side? Mid-gesture with your hand on the wrong side of the screen? No problem, just lower your hand near your desk and you are now in keyboard mode. Now your index finger might be hovering over an invisible 'a' key tuned to your hand's neutral position. Press it down on the desk, feel that haptic click, you've just typed the letter A. Let your finger bounce off and now you need to type 't' for 'attack'. Extend your finger out until you feel the haptic rub of the edge of the key, tap it down on the desk again, instant T. Of course, that only covers half the keyboard... which is where your thumb comes in.

 

The thumb, on -both- hands, controls the 'state' of your commands. You have three buttons right there on your index finger, just tap your thumb on the first, second or third knuckle. When you do that your state changes, so you can switch your index finger from being ATEI etc. to being WYZQ or .!?* or somesuch. Capital letters would be done by double-tapping a letter (harder) whereas double letters (like 'Mississippi) would be done with a tap-slide (easier and faster, but that can be reversed by those who find it too unintuitive to tolerate) Spaces could be input by tapping with your thumb once the virtual keyboard has been created (thus necessarily after a word) or by pressing your palm to the desk. I picture that resting your hand on your desk or on your armchair is still possible because if more than three keys are active at the same time you're either drunk, anxious or resting your hand and in none of those cases are those keys actually worth typing as you pressed them (but this behavior could be changed too) so if you put your whole hand down at once it just won't input any keys at all.

That changes with game mode. You don't want your hand hovering over WASD, you want to be able to rest your hand down on them and control them at will. Awesome, now allow the OS to auto-set game mode or, if it's not working right, set it manually with a quick full-pressed left-right-left gesture with the virtual keyboard. You'll get a notification on your screen, an out-of-the-way corner box which should disappear before the intro loading is even done, and you can disable it by repeating the gesture. Now if you rest your whole hand it will do nothing, but if you slide your index finger forward you are now moving forward. Slide it right? Now you're movingback (because you'll never want to move both of those directions at once). Slide your middle forward? Strafe or move right. Ring finger? Strafe or move left. Pinky? Slide it left to activate the shift key.

You probably still think this is less versatile than a real keyboard. You don't just need WASD (or your regional equivalent), you need Q, E, R, Z X C F CTRL etc etc etc. definitely. So now you can pull your fingers _off_ the desk for other shortcuts. Need to reload? Raise your index finger and you got it. Need to run forward or back and reload? Great, move your finger forward or right, then lift it. Or lift it and move it forward or right. Your fingers are still being tracked up there! Grenade? Index. Crouch? Raise your pinky to toggle. Oh heck, you need to quickly drink a potion and all your slots are already full? Here comes that thumb again! Raise it off the desk to change shortcut category, hold it against the side of your index finger for a second shortcut category, double-tap the desk for another, double-tap your index finger for another, the openness for versatility is endless and most games won't need to use even half of that. All of these gestures can be rebound and modified, like, what if you want to pull your index finger back for move_back instead of to the right? You can set that there! Deadzones are also configurable in a settings menu, in a clearly marked section.

 

And that's all just the keyboard, which could be so much better with technology we already have, just optimized a little bit. Computer mice are garbage and always have been, changing these was my first priority thinking through this. The best we have after decades of mice is just adding more fiddly buttons that are easy to press on accident? Pah! Another glove for your other hand should work, particularly when combined with eye tracking.

 

Spoiler

 

Now you don't just have your hands on the mouse and keyboard, you -are- the mouse and keyboard. While your (let's say) left hand is on your desk or pressed lazily to your cheek or having a drink or whatever, your right hand (for example) is totally dedicated to shortcuts and ease of use. This is the most complex and technical bit, I have no idea what the most ergonomic of these sorts of gestural systems would be like because we've never had a good one made, so bear with me.

 

Start with the neutral position. This is the position your hand naturally falls into when you're not telling it to do anything else. Nothing is bound to this, so you can move your hand freely between gestures. Just let your hand return to resting. You start doing things by pressing your thumb against the base of your index finger, which brings up the specific shortcut wheel for your application. Press your middle finger to that pair as well and now you're at the global application shortcut wheel, that is, your shortcuts for launching every application you own. Gesturing in the air is never going to be as precise as using a mouse so you'll be able to draw vague shapes to access parts of the shortcut wheel rather than have to make very precise short-sharp movements.

Start where you pressed your fingers into shortcut mode. Sweep left, then up, make a right-facing half-circle back down and you have opened your editing software. This, of course, was a shortcut you already set. Sounds too complicated? Want even quicker shortcuts? Turn your hand around and press your thumb against the tips in a specified order. If your index is 1 and pinky is 4 maybe 1-1-2-1-2-3 would open Rimworld, while 1-1-2-1-2-4 would open World of Warcraft. Each program would be given a UI ID when it has finished installing, one you could change manually. To set a shortcut quickly... hm. You can't set a shortcut with the hand that will tell the system to set a shortcut, can you..? You could, but it would require an extra gesture, slowing things down and making it easier to mess up something so important. But you didn't really think the left hand would be regulated to only keyboard duty, did you?

Lift it off your desk. It is the grand window controller, and it will wrangle your UI and shortcut customization. Change the active window by making a fist with your left hand and swiping along a left-right ribbon of everything you have open. Grab and toss up to store a window for easy recovery by pulling it back down like a projector screen at any time. Splay out your hand to see all windows you have open. Make a claw and swipe it to select them all, while your other hand can pick and choose the ones you'd rather have not selected. To close them, make a fist and toss them down into the trash where they belong. You should feel like the god of your UI! Want to tile multiple windows? Select them with your right hand, raise two/three/four fingers with your left hand (thumb against your palm), swipe them top-to-bottom. Reopen the window/s you just closed? Graciously lift them from the trash again. Quick-access your nice clean desktop? Wipe everything off your screen with your palm flat and your fingers pressed tightly together.

Try this now: Even if you can't snap you probably know how to. Imagine your right hand is doing this gesture menu thing, you have it with your middle and index fingers pressed together, your thumb is pressed to the base. Your left hand, place it in the snapping position, thumb-to-middle-fingertip. Do a gesture, then snap. That's how you choose to actually open something. You do a gesture and what you got isn't what you intended? Consider making the gesture easier, changing it, and meanwhile quickly swipe your left hand left, to discard it. Return to neutral. If it is what you wanted, snap, and keep gesturing. Picture how high a combo of opening things successfully you could get to if all of these three-gesture sets opened something you wanted. Need a full editing suite? Your music software? Games and game-specific tools? An emulator with specific settings? Etc. etc. etc... Just open -everything- that way. Imagine having one software be the bridge to other software. Up-left-fullcircle is FL Studio? Snap. Keep your fingers held, return your left hand to snapping position 'cause you're going to need it. Do up-left-fullcircle again, snap. You just opened the specific project you wanted to work on but last made progress with three months ago. Want to start from that further-along snap bridge after doing other work? Add your ring finger in, it will bring back those shortcuts from the focused program just like before. This could be set, so maybe only three fingers will save your snap, two-fingers never will, to let you return to start every snap and do six programs in a row from there. Of course, you could always just set those up in a chain. Left-snap-right-snap-left-snap-right... (Note: You don't actually have to snap, it just sounds and feels cooler. You could just slide one finger off the other and let the haptic snap take over)

 

There's so much more I could say about the potential of combining this with eye/head tracking, the possibilities with gesture modes and the fact that this would actually allow 3D interfacing with 3D space, but I'd like to leave a few mysteries and this is getting long enough as-is, just as a taste of near-future UI that's fun to think about. Most VR companies already make tech capable of this and it's only going to get cheaper, easier and more accessible in time, the biggest obstacle is tying it to a bulky head monitor when the monitors we have are perfectly fine for our needs. If I think of more I'll add it in other posts in this thread. Hoping to get feedback, but if you're planning on bashing it for being too different from what we have, don't bother. This is a rough draft and it's already worlds better than what we have now, and learning it would likely be easier than learning to use anything else from both a newbie's knowledge and pre-existing experience.

[ekket]

22 minutes ago, Im_CIA said:

But it has already won. Your only solace is in those hippy-dippy open source communes.

It won’t die without bringing us down with it.

[Im_CIA]

12 minutes ago, ekket said:

It won’t die without bringing us down with it.

All of Ross's apps need to be ported to React and run entirely on Chrome. Problem solved

[Icebox]

You made a video about GUIs that is over 1 hour long and still made it interesting, good work.

[Jellidrone]

A sort of GUI I always wanted is like the ones from Serial Experiment Lain. They seemed clean visually but like you could pack in a lot of information while keeping things running smooth, sort like the mouse menu idea. Though I'm a bigger fan of things that clack so having everything navigated with a keyboard would be better, that way you could seamlessly move through menus and type. Something like just having a button to bring up a general navigation menu and then hitting several followups to get to what you want, sort of like the process Dwarf Fortress does. Just you know, with visual aids to help things not suck.

Which I guess is technically hot keys but I'm thinking more like this: Say you start at the desktop, you press your menu button which should be one your thumb can immediately reach, a menu wheel pops up where you can either use the arrow keys to select one or a dedicated button. So like you choose media, it changes the options but keeps the keys and the placements the same. So like if you had up arrow for media, then you can just have up arrow again set for whatever you use the most. Just two quick taps without needing to worry about precision or timing.

[frog_0]

While this is for mobile phones and not desktop, I think a lot of inspiration on GUI could be drawn from this website, which is a collection of custom phone homescreens made using various apps that allow for that sort of thing. Some of them are WebMs, too, since they have custom animations.

[Ross Scott]

Just a quick check-in, I'll try and go through all this a little later, but thanks for all the feedback on this.  I may also edit the OP later to add links to any particularly good examples people bring up in different categories also.

[Panthercat64]

Something I would like to have in my GUI utopia is to have the option to make a desktop folder. Not what your thinking. It's a folder where you can move the icons around so you can have layers on top of layers of miniature desktops so you can move things around how you like. Oh, and it doesn't have that stupid grid system like default windows does. don't like it: don't use it. But it would make navigating my projects so much faster while also helping me be more organized. maybe have it so you can have folder backgrounds too.
and speaking of desktops: that multi-dimensional desktop thing at 1:13:09 would also be on my list. it would be just cool and nice because I use three different desktops at all times: my main, school, and stuff I want to deal with later. maybe have it so the shape depends on of many desktops you have. so having four would get you a pyramid, six gets you a cube and so on.
having icon themes, this only apples to default system icons; but it would be nice to have a default theme, maybe a prehistoric or sci-fi theme that you can easily change to help make the computer look nicer. It's like if the icons had different font.
maybe a dues ex style (different colors possible) windows for file explorer (except for those desktop folders I was talking about) would be cool to have as an option.
maybe also have a transparency effect like in windows 7, because that was just cool looking for me.
I'll add on to my list if I come up with anything else.

[The Old Reverence]

Using VR for a while now and it's definitely let me appreciate using motions and gestures to interact with our workspaces, but it's definitely something that isn't accessible by the major public.

A little addition I think would be a good idea is to cut out the task bar all together and give more use to the mouse wheel click. When working normally you can click your mouse wheel and while holding it drag up to access programs to boot up, drag down to view all active windows (going into a full screen mode similar to alt + tab), right to access a search function and finally left to access file explorer. Users could then these access path with top right perhaps leading to command line or a specific set of programs.

 

Edited June 11, 2020 by The Old Reverence (see edit history)

[FixesComputers]

I haven't seen any Mac users comment so I'll put in my opinion.

 

For reference, I've been using a Mac as my main desktop for well over a decade. I've generally enjoyed how the MacOS works as a GUI compared to Windows. Usually when I've purchased a new Mac, it's been relatively trouble free for several years.

 

Now comes the opinion: Don't. Just...don't.

 

As I've tended to use the OS as it appears, I've done little to customize. There are some utilities that allow you to "unlock" features the standard GUI hides by default. I use a utility call Onix to do this. Onix has some other features as well, like giving you easy access to running maintenance utilities, etc. (these trash collection utilities generally run automatically on a daily/weekly/monthly basis anyway so actively running them is generally not necessary). Other utilities exist.

 

What I've noticed over the years is Apple either hiding or making it more difficult to do things. Secure Erase used to be a thing I could do when emptying the trash with a mouse click. Now I have to go into the Terminal (the MacOS CLI) to do that. Setting up a hard drive in Disk Utility to have multiple partitions was a fairly simple process. Now we're back to Terminal or setting up the drive on a different computer with a different OS (I'm sure there are third-party GUI utilities for this, I haven't sought them out).

 

Occasionally I've seen other utilities here and there, but since I don't frequent Mac forums, I don't see any of them mentioned to be able to suss them out. Again, I've largely been OK with the default with just a few minor tweaks I like to make.

 

Finally, don't get a Mac. I've decided my current Mac is the last one I'm likely to own. My previous Mac suffered from a problem with the video card which caused the system to be unstable. I've noticed my current Mac to increase in instability. It frequently locks up for up to several seconds (and sometimes several minutes) before allowing me to continue. I've been noticing video glitches several times an hour as I use it. It probably has some kind of underlying hardware issue.

 

I've never had these kinds of problems with PCs. I've had them run for YEARS before any kind of hardware problem occurs. If one does, since I've built them myself, I can competently replace any failing component.

 

If you really want to know why not to get a Mac, check out Louis Rossmann's YouTube channel. He fixes MacBooks for a living and he goes into great detail as to why you shouldn't buy a MacBook. If everyone followed his advice, he'd be out of a job. Some of his rants go on to how Apple has made it increasingly difficult for their computers to be serviced. As an example, current MacBooks have an SSD that's integrated onto the logic board. If you have a logic board failure, there is a strong likelihood your data will be inaccessible. Because of the T2 security chip and defaults in the current MacOS, your data will be encrypted by default. Replacing a failed CPU or T2 chip is effectively useless as your data can no longer be accessed. Previous models had a "Lifeboat" connector so there was a way to access your data should the rest of the logic board be toast. Models before that had SSDs using a standard slot (M.2/NVMe). Again, newer models are increasingly worse for maintenance. And for that, you pay a premium price for the product.

 

A decade ago, I would have had no problem recommending a Mac over a Windows box. Generally easy to use and got the job done and nearly universally worked. Now, I'll tell people to stay far away. There has to be a major change in how Apple does things before I'll consider going back. At this point I hope I can afford a new PC before my current Mac becomes unusable. 

[krickit]

I have typically kept to standard windows and just dealt with most of the inadequacies in exchange for not dealing with incompatibility. I'd rather lose 30 seconds a day for 10 years than spend a day every few years trying to fix old broken GUI customisations. However...

 

When I first got my own PC back in 2009 I noticed a workflow problem that was fixed by a script called AlwaysOnTop, which ran in its own executable. (Looked and still looks innocuous to me.) After a year of solid use, Windows Defender started to slow my PC to a crawl for the first 10 minutes from startup, which I traced back me running AlwaysOnTop.exe on boot. So I decided to give up the functionality and get rid of it. Even now every time that it could be useful I get annoyed that I *had* the tool for it, which got broken by Windows being Windows.

 

Making this little mini player ModArchive has stay on top of the main window would be a perfect example.  (I can scroll down to see rest of the lists behind it)

 

Also, apparently I changed something in Firefox and some folks get annoyed when they see it. I don't know, it's easy to get used to a UI change if it simply doesn't break any functions, so I've forgotten what it was that I changed. So I appreciate your chunky ass set up Ross, even if it's ugly at first glance.

[coockieclickerisagoodame]

If you're looking for something that's purely function, there's ratpoison WM for linux.

 

It's the most barebones tiling WM there is. It's designed to never touch the "rat" (mouse), and drive your PC entirely through the keyboard. This isn't for everyone, but if you type a lot (if you're a writer, or a programmer), then you have the opposite problem of what Ross has described: it takes time to touch the mouse, because both your hands are on the keyboard.

 

What I also like about it is that it practically has no UI. What I mean is that the UI is your keyboard, so there's no trays, no tabs, no icons (that is, by default). The only thing that appears on your screen is a small notification bar and only when needed. This has the trade off of not being able to see that info all the time, but you don't really need that, do you? With it, your content will fill 100% of the screen, which is all that matters if you ask me.

 

It's also quite easy to customize, and it's very hacky. In fact, the default keybindings suck, if you ask me. But they're really easy to change to whatever your heart desires. There's also plenty of scripts and external programs that can add all sorts of things on top of it, including fancy graphics, though I don't use those, I don't know how good they are.

 

The only problems are that the apps are all designed with the rat in mind, so they might require additional modification. And, of course, browsxers are entirely rat-based, so no luck there. At least everything else can become easily accessible from the keyboard. It takes me less than a second to open any app I frequently use because I added it to the config file, and I have plenty of commands to organize stuff on the screen and perform various automatic actions.

 

I don't see it discussed a lot, so I felt obligated to bring it up here.

[ws_]

I'm surprised Ross hasn't realized windows floating on top of each other is kind of dumb to begin with.

[krickit]

6 minutes ago, ws_ said:

I'm surprised Ross hasn't realized windows floating on top of each other is kind of dumb to begin with.

Whenever I open a new program that requires a lot of screen real estate, I just buy a new monitor. No need to switch windows, tabs or desktops, I just need to move my eyes.

[Ross Scott]

Christ, I can't even use my own forums formatting properly.  I tried using the quote tags and everything went to hell, so please excuse the awkward formatting instead:

 

jacquie48th There's a utility that's similar to that called waifu2x.

Althought it was made originially for anime-style images, it should also work for photos.

I'll give it a shot sometime, thanks.

 

Im_CIA:  Many people don't use computers for "work", and those who do, use excel. For software developers/engineers (computers ARE work), hostile GUIs don't pose as big of problem since most of the time they work in a specific scope that can be fine-tuned. True power users are a dying breed.

 Yeah, for me, computers are a means to an end.  I'm willing to put the time in to get it working how I want to, but then I want to not have to deal with that again for years at a time if I can.

 

Also, your desktop is a perfect example of the problem I notice visually with a lot of themes.  Your terminal windows look pretty slick, dark theme, but readable colors, looking nice.  Then BAM your web page is bright as hell in comparison and just overpowers the rest of it.  My solution was to go for something in between, but maybe forcing everything to be dark on the web with plugins could work too.

 

hj You, Ross, yourself, complained about "learning all the hotkeys", the very same principle applies to mouse gestures, learning how to use console et cetera. 

It's a fair point about me complaining about learning the hotkeys, that maybe was a cheapshot on my part, however it's ALSO coupled with that not being an ergonomically great system.  So we're talking about memorization, but it ALSO not feeling great once it's memorized!

 

hj Windows 8 wasn't hated because it was slower or because it has tablet interface, biggest reason was because they drastically changed things there. 

This is true, HOWEVER, Metro ALSO LITERALLY COULD NOT DO ALL THE FUNCTIONS the regular UI could!  I don't think there was a Metro version of device manager, for example.  So even if you liked it, it was OBJECTIVELY NOT AS FUNCTIONAL.  It was like my example in the video where I tried to change the resolution in a VM using the GUI and it was LITERALLY IMPOSSIBLE.  No wonder those users prefer the CLI then!  If I hand you a new interface, but then it can only do 60% of what your old one did, of course you're going to hate it.  It's the same story with Windows 8.

It's true the public fights change.  However, if something is far and away BETTER, then a lot of people can end up coming around over time.  For example, I don't see a lot of people going back to typewriters over computers these days.  Regardless, it's possible to have OPTIONS, so the people who do want change can have it, while the holdouts can keep using what they're used to.  Don't hand me false dichotomies man, I get enough of that from Microsoft.

 

hj However if we really focus on efficiency and forgo the "easy to understand" and other things, I'm afraid we'll have to also forget the mouse.  

I'm not saying the mouse is great, but for some things, the keyboard is WORSE.  I tried to point out some examples in that video, how would you rapidly select specific files from a line-up like I did in the video with just the keyboard?  How about rapidly changing DOSBOX profile settings on-the-fly (not preconfigured)?  I mean if you explain in detail how that is JUST AS  FAST with the keyboard with the right interface I'll listen, but I have a suspicion it's that some people want work things to work one way, so they'll forgo other methods when that doesn't fit the mold.

 

As for the look, I like a lot of elements of old-school Linux themes, but they could use some modernizing.  I'm not thrilled about flat with lots of black and / or white.

 

testman We need a central knowledgebase for everything related to the GUI Quest™.
A place for proposed ideas, design guidelines for existing interfaces, links to experimental interfaces, list of abandoned concepts, etc.

Someone on Reddit made a repo that I assume will be used for this purpose. https://github.com/HawaiinPizza/Ross-Good-Gui

By all means, the forums here were just my  stab at bringing order to chaos.  I can maybe migrate things to that later.

 

NightNord But honestly I don't think a lot of people really care.

Hence the reason I couldn't stop my urge to create an hour-long quixotic video on the subject.  

 

danm36 For Ross's comment about the Windows shell being embedded so far in the OS that it's a pain to mess around with/replace, there's some good-ish news. Microsoft have apparently finally decided to separate the Windows shell from the OS layer, which should hopefully mean that shell replacements become much easier to implement in the future. Their main reasoning is so that they can push out shell updates without needing a full OS update, but dummying it out and replacing it with a custom shell should become trivial. How long this will take though... that's yet to be seen. As a Windows developer myself, the shell is pretty deeply nestled in there. 

That's fantastic news, I'm glad I made the video when I did then, so I didn't have to throw out a whole chunk of what I was going to say.  I look forward to some of the video becoming obsolete.

 

AtomicPurple If I use a GUI interface for the same tool, I have to click to open the program, click the open directory button, then click a bunch of times to navigate to the directory I already had open, and THEN I can start converting.

I really don't think it's an either / or thing.  Even though I showed a couple examples in the video where I think the GUI is faster, I still don't think it's GOOD there.  Organzing files isn't particular pleasant at all on the mouse and keyboard, I see losing the mouse part of the equation not making it much nicer for me, I think we just need better software there to be honest.

 

FixesComputers Now comes the opinion: Don't. Just...don't.

Duly noted!

Yllia Yeah I think the motion tracking gloves + keyboard are the "everybody wins" solution for getting away from mouse switching.  
I tend to be more visually oriented and I actually think the GUI

 

LochNessMobster AR is one of those things I could see evolving to the Iron Man level over many years, or it may sputter out.  I'm not waiting for that, I can be happier with more primitive means, but not what Windows is handing me.  Also, you may find this interesting:
https://www.pcgamer.com/this-8k-holographic-monitor-has-me-dreaming-about-the-future-of-gaming/

 

FuzzyCandle Yeah, you're beyond my level, I'm just at a point where I can see the potential of haptic controls.  A problem that's unlikely to change for decades though (maybe ever) is software isn't going to be designed for it, so we'll have to focus on motion controls as a translation layer for more conventional interfaces for programs.  By all means, figure out how things should work though.  It's really not even a joke talking about us having to develop a martial arts style to interact with a more advanced GUI.

Edited June 11, 2020 by Ross Scott (see edit history)

[Ross Scott]

For Linux people, none of these are the perfect look or anything, but here are some example of Linux themes where I liked a lot of what was going on.  I think they could use some cleaner lines, but I liked some of the shading, textures, fonts, icons, etc.  In general, I want to avoid pure white, but aim for some lighter shades, plus a few normal ones with a focus on readability.  I'm open to other ideas too, but I wasn't able to find anything like this in modern Linux themes.  Everything felt flat and / or way too much white or black.

 

 

 

 

 

 

Edited June 11, 2020 by Ross Scott (see edit history)

[Rbon]

11 hours ago, hj said:

I don't have to aim with mouse to click "save", i don't have to memorize that "Ctrl-S" is "save", i just hit (Space), a menu pops up, i see "f - file", i press "f", then i see "s - save" so i press "s", eventually it becomes a second nature to me - space - (f)iles - (s)ave.

Hey, another spacemacs user here. I want to elaborate on what hj said. This system is intuitive, fast, and crucially, context sensitive. Depending on what file you're working on (the major mode), you are offered different key sequences, in addition to a list of global ones.

 

But that's not the good part. The real beauty is that in emacs everything you to is a function, and every function can be bound to a key sequence. Which means that, when you find yourself doing the same series of actions over and over, you can just write a function that does all those actions in order, and bind it to just one key sequence.

 

Now, I'm not trying to say that spacemacs is the future of GUI (it's really not). The point I've been working towards is that the mouse radial menu system would be really really good if more applications had a way to receive signals from external programs. Imagine you're watching a video, and with one mouse motion, you pause the video, switch to your preferred email client, reply to the most recent email with "I'll get right on it," switch back to your video, and resume playback.

 

This could very easily be a reality. It is platform agnostic. It doesn't matter how locked down Windows is, as long as an application can be told to act by receiving a signal. The only reason this hasn't already happened is because devs by and large just don't implement it. If you are a developer working on an application right now, you can make this future a reality. You have the power to make the future of GUI amazing.

[UsefullPig]

Well I guess I'll throw my hat into the ring. I've watched the new video twice now so I have some thoughts.

 

Infinity New Tab

I love the Chrome/Firefox Extension "Infinity New Tab". It takes a bit of tweaking to look good, but it's absolutely pleasant for me to use with my current setup. These website icons are massive landing strips, as opposed to the tiny helipads bookmarks are.

 

This is my setup:

 

This is what it looks like by default:

Spoiler

 

The Sims 2
TS2 has my favorite game UI that I can think of. The circle launcher Ross envisioned in the video is present here, and it's contextual too. You can even Shift+Click for debug options if you enable testing cheats.

 

Also look at the main part of the GUI:

Notice that that buttons are sized relative to importance. And I love that it's laid out from left to right in sperate section that can grow or generate new sections next to them. Look at it in a collapsed view:

 

Hotkeys

I largely hate hotkeys. Ross's sentiment resonates with me. "Better learn those hotkeys. Hup hup hup" is what a lot of Adobe Premiere tutorials look like. I wish the Sonder Keyboard existed already but it looks like it's never going to come out.

 

I use these on the regular:

• Win+Shift+S to take screenshots. I used this one a lot while making this post.
• Alt+Tab to switch programs. Holding down Alt+Tab is actually why I don't have that big of a problem with the taskbar. 
• Alt+\ is what I use on Discord to deafen myself.
• Ctrl+Shift+T to open up the tab I just closed.

Mobile UI/UX

Google copied iOS 11 when they made Stock Android 9 and they made it better. I honestly hate using Android 8 and lower thanks to Google's overhaul to the UI. Android 10 made it even better. I'll say this: I am just less frustrated when I use my Pixel than when I use my iPad. There are so many little touches that Android 10 has that it sometimes makes me feel like a "GUI Wizard" when I use it.

 

Alright lads I'm too tried to keep writing this. I look forward to any replies.

Edit: Spelling. Also, I'd be happy to expand on the tiny details Stock Android 10 has that make it a treat to use, and how good TS2's UI functions for what its trying to accomplish. But later...

Edited June 11, 2020 by UsefullPig (see edit history)