Skip to content

Guacamole Support

Chris Martin requested to merge cmart/exosphere:issue-358-guacamole into master

Fixes #358 (closed) (see that issue for more context) by implementing the following:

  • Exosphere to know about which OpenStack clouds have TLS-terminating proxy services to support use of Guacamole
  • By default, Guacamole is deployed on instances that are created on clouds on above list
  • User can click one button to get a graphical shell or (if server image has a GUI set up) a remote graphical desktop to their instance

When this is merged, it will make Exosphere useful enough in a web browser that we can start pointing people to try.exosphere.app instead of requiring them to download the Electron app.

This functionality uses the cloud-specific TLS-terminating proxy built in #381 (closed). Some of this work will follow the basic pattern of !308 (closed), but learning from mistakes made there.

Our goal is to deprecate/remove the Cockpit integration, this will be done in a subsequent MR.

Also fixes #384 (closed) and #392 (closed), and (likely) #293.

Design tasks

  • Decide how to indicate (in instance metadata) that Guacamole is deployed, which protocols (SSH and/or VNC) are available, and perhaps where it is accessible
    • Maybe some brief JSON like exo_guac={ssh:"true",vnc:"false",url:"https://http-1-2-3-4-8081.proxy-j7m-iu.exosphere.app/guacamole"}
  • Shared secret for HMAC auth: decide what to use and where to store it. Should we just use server password as this shared secret, or create another one?
    • Create another secret and store as a server tag (this way we aren't storing plaintext of server password on the server, regardless of filesystem permissions)
  • Don't use port 8080/8081, pick a high-numbered port that's unlikely to conflict with anything
    • 49528!?
  • Does proxy hostname need to be set in Guacamole config (in which case cloud-init needs to know about it), or can we do without?
    • Does not need to be set! From Guacamole Manual:

      Your servlet container is most likely already configured to listen for HTTP connections on port 8080 as this is the default. If this is the case, and you can already access Guacamole over port 8080 from a web browser, you need not make any further changes to its configuration.

  • Should we make the instance ignorant of the proxy URL, so that the proxy server address can change over time?
    • Depends if we need to set proxy hostname in Guacamole config, above
    • The answer is yes, because Guacamole config doesn't care about proxy hostname/URL.
  • Figure out how to deploy Guacamole to instances with HTTPS (self-signed certificate)
    • I believe this requires either changing the configuration of Tomcat (which the Guacamole Docker container doesn't support natively), or reverse-proxying Guacamole again (on the instance), so we would be double-proxying the connection which I'm not keen on. So, moving to out-of-scope.
  • How long lifetime to use for guacamole-auth-hmac tokens? 1 day perhaps?
    • Per the docs, default api-session-timeout is 60 minutes. Exosphere should either respect this (get a new token every hour), or configure Guacamole for tokens to last longer, and get a new token as often as that.
  • Can we set variables in cloud-init? (e.g. for the HMAC secret key)
    • Probably not, just template it out using Elm
  • Decide how to do Guacamole SSH auth, for the moment.
    • Probably ChallengeResponse, which means we need to enable ChallengeResponse SSH auth
  • Test basic auth with hand-configured instance
    • Ensure that Exosphere can POST to /guacamole/api/tokens and authenticate the user, so the user doesn't need to enter a password
    • If it works, redo plan to use basic auth rather than HMAC auth
    • It works!

Implementation tasks

(some of this is similar to !308 (closed))

  • Continuous deployment to "dev" branch, a la !308 (closed)
  • Host compiled build of guacamole-auth-hmac somewhere Not doing anymore, using basic auth
  • Store list of clouds that support TLS-terminating proxy
    • Either in globalDefaults like !308 (closed) did, or as a flag at startup (which means it can be overridden without recompile)
    • For each cloud, store:
      • Hostname part of their Keystone auth URL for that cloud
      • Hostname of TLS-terminating proxy server
    • Start with just IU, add TACC later
  • Bump exoServerVersion (but do we need to? yes, so we know when to stop looking for cockpit in instances)
  • "Deploy Guacamole for easy remote access" checkbox in advanced settings of create server view, determines which cloud-init to run
    • How to model this? New field in CreateServerViewParams
  • Figure out how to translate user's choice above to cloud-init
    • Maybe Helpers.renderUserDataTemplate replaces {guac-setup} with the actual code to set up Guacamole
      • I don't love this, maybe there's a better way
  • When creating a server, generate random shared secret (for HMAC auth), store as server tag and use cloud-init Not doing anymore, using basic auth
    • How to pass to cloud-init?
      • Maybe generate the secret when we enter the create server view, store it in the CreateServerRequest, template it out in the cloud-init and specify in Nova API call to create server
  • When creating a server, set JSON in server metadata, maybe something like exo_guac={ssh:"true",vnc:"false",deploy_complete="false"}?
  • Build out cloud-init/docker-compose/etc code to deploy the following on new servers:
    • Enable ChallengeResponse auth for SSH, and restart service
    • Set up directory structure for Guacamole config/extensions
    • Create user-mapping.xml with username, password hash, and connection information
    • Populate guacamole.properties with HMAC secret key, probably passed in the cloud-init
    • Download guacamole-auth-hmac from wherever we host it, verify expected checksum
    • Set correct file permissions for user-mapping.xml?
    • Initialize database
    • Rotate guacadmin password, perhaps use HMAC secret
    • Run Guacamole docker-compose.yml, as daemon
      • Ensure containers survive instance reboot
    • Upon successful completion of Guacamole setup, maybe tell Exosphere (e.g. by logging to console)
  • Notice when Guacamole is ready
    • TODO do we check this by POSTing to get login token, or by looking for confirmation of successful deployment in the console?
    • Set deploy_complete="true" in server metadata
  • Exosphere code to sign HMAC requests for login token -- basically what this Python script does Don't need anymore with basic auth
  • Model Guacamole login tokens (for now just shell, later also VNC) as a server exoProp
    • Probably wrap it in a RemoteDataPlusPlus so we can track its age and loading status
  • Orchestration code to get Guacamole login token
    • If:
      • exo_guac is set
      • Either we don't have a login token, or we have a login token that is too old (beyond api-session-timeout configured in guacamole.properties, default is 60 minutes?)
      • We are not already loading a login token
    • Then:
      • POST to obtain login token(s)
  • If we have Guacamole login tokens in the model, then display "Launch shell" / "Launch desktop" buttons to user
  • Bring in 8c77348c from !308 (closed)
  • Make it all also work on TACC cloud
  • Document how to configure Exosphere to modify list of clouds that have a TLS-terminating proxy
  • Troubleshoot and fix this bug when starting a Guacamole session
  • Set up automatic cert renewal for both UAP servers

Out of Scope / future work

  • Create issues for these before/when this MR is merged

In subjective descending order of priority:

  • Start deprecating Cockpit support
    • Don't require Cockpit availability to show that a server is fully active
    • Stop deploying Cockpit to new instances
    • Stop looking for Cockpit on instances of exoServerVersion at least 3
    • When showing an instance with Cockpit enabled, warn that it will be disabled in a future version of Exosphere
  • Implement deploying Guacamole Desktop (GUI) support
    • Will need to pick a VNC server; Guacamole manual seems to recommend RealVNC or TigerVNC.
    • GUI desktop additionally needs:
      • At server deploy time:
        • Exosphere (probably in cloud-init code) needs to determine if instance has a GUI desktop environment, figure out how to do this
        • If so, Exosphere should set up a VNC server on each instance, exposed only to guacd
      • After deployment:
        • Exosphere needs to know if instance is set up for Guacamole Desktop, which determines whether to expose functionality to user
      • Shared secret between VNC server and Exosphere (to be passed in Guacamole token request)
        • Maybe same as HMAC shared secret, or maybe just use server password.
  • Support Guacamole on CentOS 8 instances (see comment on this MR, CentOS 8 doesn't officially support Docker)
  • Make it crystal clear to users how to copy and paste in and out of Guacamole shell/desktop
  • Guacamole supports file transfer! and we should enable it!
  • Document and announce, far and wide, that core features of Exosphere now work in a web browser, for clouds that have our TLS-terminating proxy (i.e. Jetstream)
  • Figure out HTTPS support for connections between TLS-terminating proxy and Guacamole server on cloud instances. Will likely require configuring Tomcat (inside Docker container), or reverse-proxying Guacamole connections on the instance.
  • Deprecate Electron app
  • At some point, deprecate Cockpit support for existing instances (launched before this MR)
  • If Guacamole support significantly increases the amount of time it takes to deploy an instance (spoiler: it will), consider reporting progress to user while cloud-init is running, e.g. ("70% done, setting up remote access"). Use /dev/console to pass this data to Exosphere, and expose it in server detail view.
  • #187 (closed), set up GUI desktop environment at deploy time, regardless of whether it's baked into the image
  • Better communication to user if Guacamole failed to deploy for some reason, perhaps use the console log, perhaps persist the error in server metadata
  • For users of clouds which do not have a TLS-terminating proxy, support direct connection to Guacamole, somehow?
  • Better infrastructure-as-code for TLS-terminating proxy, may already be tracked elsewhere
  • Does Guacamole support something better than MD5 to encode user passwords? Might need to move to database authentication: https://sourceforge.net/p/guacamole/discussion/1110834/thread/0e8d9291/
  • Decide whether to prevent connections to Guacamole from IPs other than known IP of TLS-terminating proxy (e.g. via firewall ruleset, or OpenStack security group)
    • To prevent malicious parties from trying to hack their way into Guacamole. This is Java, after all.
    • To prevent users from accessing it insecurely? Browser would warn them anyway
  • Audio support in the Guacamole session
  • UTF-8 support for clipboard, requires compatible VNC server
  • See if we can remove requirement of MySQL/MariaDB from Guacamole, since we aren't using it for anything

How to Test

  • Test with CentOS 8
  • Test with CentOS 7
  • Ensure Guac shell works in browser (most important) and also Electron
    • at IU
    • at TACC
  • Ensure new app behaves appropriately when showing servers launched before these changes (i.e. they have Cockpit deployed but no Guacamole)
  • Ensure old app (before this MR) behaves appropriately when showing servers launched with new app?

Screenshots

(If visual changes are introduced, show what they look like)

Edited by Chris Martin

Merge request reports