Skip to content

Only reroute stdout etc. to new console if not already redirected

AnyOldName3 requested to merge AnyOldName3/openmw:redirection-fix into master

This should fix the issue where Windows Release builds (compiled as /SUBSYSTEM:WINDOWS instead of /SUBSYSTEM:CONSOLE) can't have their output redirected.

Basically, a console application creates a console if not given one, so you get a console window behind OpenMW while it's running. It was decided that this was ugly, so we set Release builds to be windows applications, which don't get an automatic console and don't automatically connect to a console if given one anyway.

Of course, we still wanted to actually be able to print to a console if given one, so we manually attach to the parent process' console if it exists, then reopen the standard streams connected to CON, the Windows pseudo-file representing the current console.

This is a little like connecting a second wire into a dumb terminal in that you're pumping characters into the display rather than onto a pipeline, so output can't be redirected.

It turns out, though, that if a /SUBSYSTEM:WINDOWS application has its standard streams redirected by the calling process, it still gets its handles as normal, so everything starts off connected just how we want it and we were clobbering this good setup with the straight-to-console fix.

All we need to do to fix that is check if we've got valid standard handles and that they go somewhere useful, and if so, avoid reopening them once the console is attached. Simples.

Resolves #5650 (closed)

Merge request reports