I suggest expanding the test range to include the open source nouveau driver for nvidia hardware, geforce 770 is kepler architecture so it should be supported by nouveau, what makes nouveau special is the ability to run DX9 native directly in hardware, no translations to GL or VK, you may have to refer to your distro manual to figure out how to switch the display drivers to nouveau, but it's possible distro builds of mesa may not include nine support so you may have to compile the graphics stack yourself (recommended, and worth it ), one good thing about this option specifically for you because you wanne play less than DX9 games is boosting the games from pre-9 to 9, so for example you take say a DX7/8 games and using DX1-8 to 9 convertion you get "native" DX9 for pre DX9 games by boosting libraries
I am also looking for ways to have SS/MS in WINE/Proton+Nine
so far I only have the general nvidia operational phrase: "you gete blur and you get blur and everyone gets more blur"
but im getting there slowly one hack at a time
I would also recommend to try with the open source radeon stack, that is get a radeon card and see if anything changes (radeon is VASTLY superior to nouveau in open source driver stack terms, which could lead to improved results), I know this is not exactly what people usually whant to hear since this is money going out, but if it fixes the rendering problems...is worth it
I know you mentioned you don't like post "AA" but...it can look good when set to highest, I had very good results even at 2K(native) resolution with FXAA set to 39 in reshade (simply have to edit the shader code and uncomment), one more thing: a 770 can only go so far, performance is very limited, performing translations+applying exotic post shaders or AA not to mention supersampling and friends....can't be done, not at any normal framerate, you could mitigate this by compiling a fully preemptive kernel and set it's clock to 1000 hz, but even the lowest latencies cannot give you throughput...you just need more graphic resources if you really prefer to stick to nvidia I highly recommend a 780 Ti, a good one with high clocks like 1200 mhz on the core (such as the EVGA classified kingpin), it's expensive around 500$ (which is the same price since 2014) but it's the best there is. the problem is that radeon cards fail badly in windows and nvidia cards fail badly in linux, so if you buy a radeon card for linux and linux fails you, your windows experience will be compromized by radeon bugs (AMD is a small low-budget company and they cannot afford software development...) and if you buy a modern nvidia card you won't be able to drive it (by open source drivers) in linux at all. I cannot recommend using nvidia closed source drivers, not realy so much for the principle of it but more because nvidia has proven in the past to do dirty things, one such thing is proactively withholding firmware images to drive their modern hardware by open source nouveau driver, they are actively trying to force linux users to use their closed drivers...
I also recommend to test some winecfg DLL overrides for various DLL's and don't forget to install directX fully (download directX june 2010 and install it)
try xrandr --output yourdisplay --scale 2x2
but I think it might look bad, I haven't tested this yet myself