Fixes #39645 - Replace Fog::SSH/SCP with system SSH

[lzap]

Rubygem net-ssh does not provide all features for PQC and overall, we would like to simplify crypto stack in Foreman. This patch refactors the SSH service class so it shells out to ssh and sshpass instead.

The previous workflow was also quite complex, the main goal of the SSH finish script is not super-resilient SSH scheduling, but to bootstrap other components like Remote Execution or Ansible and for this, there is no need to establish a long-lasting connection via sockets.

Furthermore, the old solution copied the script using scp which is also unnecessary. Instead, there are simply two connections made using ssh command:

• The initial wait for the VM is done via ping method, previously this was named estabilish_connection! but this would be confusing. I made the method explicit so constructor does not perform this - this was confusing behavior.
• The final call is then done via deploy! method, instead of scp the script is simply passed via STDIN - no need of temporary files.

Other major changes to further simplify the code:

• For sshkey authentication, the temporary file with the private key is no longer created in the constructor relying on the cleanup method call. Instead, it is created just for the minimum possible time in the run_ssh method and cleaned immediately.
• Logging was moved to run_ssh method (DRY).
• Unit test was written by Claude, a bit heavy to my taste I will happily drop some cases.

I thought for a moment that we could even get rid of the ping and simply try to execute the template directly in a loop, but I discarded the idea. It is safer to be explicit.

I tested this via rails command in the proper SELinux policy with a small change (see the linked PR):

systemd-run --pipe --wait --service-type=oneshot \
  -p SELinuxContext=system_u:system_r:foreman_rails_t:s0 \
  -p User=foreman \
  -p WorkingDirectory=/usr/share/foreman \
  -p Environment=RAILS_ENV=production \
  -p PrivateTmp=no \
  /usr/share/foreman/bin/rails runner /tmp/test_ssh_provision.rb

This is a followup for previous work in #10518 and #10615 which was reverted.

TODO

• Create PR to SELinux policy with the changes.
• Add ssh and sshpass dependencies to foreman package.
• Drop ssh-ssh/scp packages from our repos. (After we drop the library from ReX)

Relevant PRs:

• Add openssh-clients dependency to foreman foreman-packaging#14041
• Add openssh-client and sshpass to foreman Depends foreman-packaging#14042

[ogajduse]

Packaging ACK, as I see that you have the packaging changes on your radar.

Reading from the PR description:

Drop ssh-ssh/scp packages from our repos.

[lzap]

Packaging ACK, as I see that you have the packaging changes on your radar.

For the record, the packages can be only removed after ReX also drops the dependency, it still needs it for relatively trivial reason and there is a ticket for that. I created https://redhat.atlassian.net/browse/SAT-49367 and I will move it to the ReX epic if core is merged earlier.

[stejskalleos]

Few comments from the first look, after the fix I will test the workflow

[stejskalleos]

From Ruby 3.0.z docs:

This method is not recommended and exists mostly for backward compatibility. Please use Tempfile.create instead, which avoids the cost of delegation, does not rely on a finalizer, and also unlinks the file when given a block.

[stejskalleos]

I assume the file is created with 0600 permissions, but what if it isn't?
I'm not sure if it is possible, but could we explicitly set the permissions after the key file is created?
Just to be sure.

[lzap]

Both functions has 0600 hardcoded in them, but I wrote test just to be sure. Not sure where your comment aims?

[stejskalleos]

Suggested change

	@template = options.delete(:template) || raise("must provide a template")
	@template = options.delete(:template) || raise ::Foreman::Exception.new(N_('must provide a template'))

[stejskalleos]

Suggested change

	raise "Failed to connect to #{@address} after #{Setting[:ssh_timeout]} seconds"
	raise ::Foreman::Exception.new(N_("Failed to connect to #{@address} after #{Setting[:ssh_timeout]} seconds"))

[lzap]

Because of exception extractor, I will use placeholders instead. But thanks for the catch.

[stejskalleos]

Comment from AI:

ssh_options missing -oBatchMode=yes

Not setting -oBatchMode=yes means SSH could prompt for a passphrase interactively if key auth is configured but the key is encrypted (or if sshpass fails for password auth). Since these are machine-generated ephemeral keys in a provisioning context this is low-risk, but worth noting.

[lzap]

Thanks for the review, all remarks done except not sure what you meant by the 0600 comment. I just added the test there for extra coverage - it is hardcoded in (both) functions).