🔐 Palo Alto SSH Proxy – Configuration, Troubleshooting & PCNSE Guide

1. Overview

SSH Proxy enables Palo Alto Networks firewalls to decrypt and inspect SSH (Secure Shell) traffic, preventing the use of SSH for tunneling malicious applications and content. It supports both inbound and outbound SSH sessions and operates by acting as a "man-in-the-middle" (MitM), establishing separate SSH sessions with the client and the server. This allows the firewall to gain visibility into the encrypted SSH payload for security inspection.

Key Function: Decrypt SSHv2 traffic for inspection and policy enforcement.

Core Benefit: Prevents abuse of SSH for unauthorized tunneling (e.g., X11 forwarding, TCP forwarding) and allows threat inspection within the SSH channel.

Figure 1: Conceptual SSH Proxy Man-in-the-Middle Flow

2. Configuration Steps

Configuring SSH Proxy involves creating a Decryption policy rule and, optionally, a Decryption profile.

1. Ensure Interface Types: SSH Proxy decryption can be performed on virtual wire, Layer 2, or Layer 3 interfaces.
2. (Optional but Recommended) Create/Modify Decryption Profile:
   • Navigate to Objects > Decryption > Decryption Profile .
   • Create a new profile or modify an existing one.
   • Under the SSH Proxy tab, you can define controls for:
     • Block sessions with unsupported versions (e.g., SSHv1, which is not supported by SSH Proxy).
     • Block sessions with unsupported algorithms .
     • Block sessions if resources not available .
     • Block sessions if a hardware security module (HSM) is unavailable .
   • This profile allows granular control over what SSH parameters are permitted when decryption is active.
3. Create Decryption Policy Rule:
   • Navigate to Policies > Decryption .
   • Add a new rule.
   • Define match criteria (Source, Destination, Service, etc.).
   • Under the Options tab:
     • Action: Decrypt
     • Type: SSH Proxy
     • Attach the Decryption Profile created/modified in the previous step.
4. Commit Configuration: Save and commit the changes to apply the configuration.
5. (Optional) Create Decryption Exclusions: Exclude specific traffic from decryption if necessary (e.g., for hosts requiring key-based authentication that cannot be altered). Navigate to Policies > Decryption , select the rule, go to the Destination tab, add IPs, and select Negate .

3. Impact on Key-Based Authentication

SSH Proxy fundamentally breaks standard public key-based authentication. This is a critical consideration.

Here's why:

• Man-in-the-Middle Nature: The firewall intercepts the SSH connection. The client establishes an SSH session with the firewall, and the firewall establishes a separate SSH session with the server.
• Host Key Verification: When the client connects, it is presented with the firewall's SSH host key, not the actual server's host key. This will likely trigger a "host key changed" warning on the client, or connection failure if the client is strict about host key checking.
• Client Public Key Authentication: In standard key-based authentication, the client proves possession of a private key corresponding to a public key stored in the server's authorized_keys file. When SSH Proxy is active:
  • The firewall would need to authenticate to the server. If the server expects key-based auth, the firewall doesn't possess the client's private key to perform this.
  • The client is authenticating to the firewall, not directly to the end server in the initial key exchange.

As a result, direct client public key authentication to the end server through an SSH Proxy setup will fail.

Figure 3: Sequence Diagram Illustrating Key-Based Authentication Impact

Solutions and Workarounds for Key-Based Authentication:

1. Username/Password Authentication: The most common solution is to use username and password-based authentication for SSH sessions that are being proxied. The firewall will proxy these credentials.
2. Decryption Exclusions: For specific servers or clients that must use key-based authentication and cannot switch to password authentication, exclude their traffic from SSH Proxy decryption. This means their traffic will not be inspected. Policies > Decryption > (select rule) > Destination > Add IP addresses > Select "Negate" checkbox.
3. Jump Server/Bastion Host: Consider using a bastion host that enforces strict access controls and logging. Traffic to this bastion host could potentially be excluded from decryption if key-based auth is paramount, though this reduces visibility. Alternatively, proxy connections *to* the bastion host (if it supports password auth) and then allow key-based auth *from* the bastion host to internal servers (this segment would not be decrypted by the perimeter firewall).

4. Common Issues, Challenges, and Caveats

Implementing SSH Proxy can introduce several challenges:

• Unsupported Algorithms/Ciphers:
  • Issue: If the client or server attempts to negotiate SSH versions (e.g., SSHv1) or cryptographic algorithms (host key types, ciphers, MACs, Key Exchange algorithms) not supported by the firewall's SSH Proxy implementation, the connection will fail. The firewall typically supports common, secure algorithms but may not support everything, especially older or very new ones.
  • Indication: Decryption logs may show "decrypt-unsupport-param" or similar errors.
  • Solution:
    • Use a Decryption Profile to "Block sessions with unsupported versions" and "Block sessions with unsupported algorithms."
    • Ensure clients and servers are configured to use compatible, strong algorithms. Check Palo Alto Networks documentation for the list of supported ciphers for your PAN-OS version.
    • Exclude traffic if alignment isn't possible and connectivity is critical.
• SSH Tunneling Limitations:
  • Issue: While SSH Proxy is designed to decrypt the main SSH session and identify/block common tunneling types (X11, direct-tcpip, forwarded-tcpip), it might not detect or block all forms of sophisticated or custom SSH tunneling if the application itself is not correctly identified. The firewall identifies tunnelled traffic using App-ID within the decrypted SSH stream.
  • Solution:
    • Ensure your Decryption Profile and Security Policy correctly identify and block the `ssh-tunnel` application.
    • Maintain up-to-date Application and Threat definitions.
• Authentication Failures (beyond key-based):
  • Issue: Even with password authentication, issues can arise if there are mismatches in supported authentication sub-methods or if RADIUS/TACACS+ interactions are affected.
  • Solution: Simplify authentication methods for testing; check authentication logs on the firewall and end-server.
• Client Host Key Warnings:
  • Issue: Users will receive warnings that the remote host key has changed because the firewall presents its own SSH key.
  • Solution:
    • Educate users about this expected change when connecting to resources subject to SSH decryption.
    • For internal, controlled environments, users might need to accept the new firewall host key. In some SSH clients, the old host key for the server will need to be removed from the `known_hosts` file.
• Performance Impact:
  • Issue: Decrypting and inspecting traffic consumes firewall resources (CPU, memory), which can impact performance, especially with a high volume of SSH sessions or high-throughput SSH data transfers (like SCP/SFTP).
  • Solution:
    • Monitor firewall resource utilization ( show system resources follow , show session info ).
    • Scope SSH Proxy policies to only the traffic that requires inspection.
    • Ensure the firewall hardware is adequately sized for the decryption load.
• Unsupported SSH Features:
  • Issue: Some advanced SSH functionalities might not be fully supported or may behave differently when proxied. For example, while X11 forwarding and TCP forwarding are typically identified as `ssh-tunnel` and blocked, the specifics of agent forwarding support through the proxy should be verified for your PAN-OS version. Palo Alto Networks documentation states X11, forwarded-tcpip, or direct-tcpip are identified as SSH tunneling traffic and blocked.
  • Solution: Test specific features thoroughly. If a critical advanced feature is incompatible, consider exclusion.
• QoS Limitations:
  • Issue: Proxied SSH traffic does not support DSCP code points or QoS marking Passthrough.
  • Solution: Be aware of this limitation if QoS is critical for SSH traffic. Traffic will likely be re-marked based on policy post-proxy.
• Logging and Visibility:
  • Issue: While the goal is increased visibility, SSH Proxy traffic itself is not captured in Decryption logs according to some documentation. However, the outcome of policy (allow/deny of tunneled apps) will be in Traffic logs, and threat logs if signatures trigger.
  • Solution: Primarily rely on Traffic, Threat, and URL Filtering (if applicable to tunneled content) logs for visibility into the *content* of decrypted SSH sessions. System logs might show issues with the proxy process itself.

5. Troubleshooting SSH Proxy

When SSH connections fail or behave unexpectedly with SSH Proxy enabled, follow a systematic troubleshooting approach.

Figure 4: SSH Proxy Troubleshooting Flowchart

5.1. Firewall Logs

• Traffic Logs ( Monitor > Logs > Traffic ):
  • Check if the session is matching the Decryption policy.
  • See if traffic is allowed or denied after decryption (e.g., `ssh-tunnel` application being blocked).
  • Look for `session end reason`. Reasons like `policy-deny` or `threat` can indicate what happened after decryption.
• Threat Logs ( Monitor > Logs > Threat ):
  • If threats are detected within the decrypted SSH stream, they will be logged here.
• System Logs ( Monitor > Logs > System ):
  • Look for errors related to SSH proxy processes or decryption resource issues.
• Decryption Logs ( Monitor > Logs > Decryption ):
  • Important Note: Some Palo Alto Networks documentation states that SSH Proxy traffic specifically is not captured in Decryption logs. However, if there are errors related to the decryption module processing SSH, such as unsupported parameters if not explicitly blocked by a profile, they might appear here with error codes like `decrypt-unsupport-param`. It's always worth checking.

5.2. CLI Troubleshooting Commands

Use these commands via the CLI for more in-depth analysis. Access CLI via SSH or console connection.

Show Commands:

Command	Description
show system info	Displays basic system information, uptime, PAN-OS version.
show session all filter type ssh proxy	(Or similar filters) Tries to display active SSH sessions being handled by the proxy. The exact filter might vary or need application ssh and inspection of flags.
show session all filter application ssh count yes	Shows count of sessions for application 'ssh'.
show session id	Displays detailed information for a specific session ID identified from logs or other commands.
show counter global filter aspect ssh	Displays global counters related to SSH processing. (The exact 'aspect' might differ, use '?' for options).
show counter global filter delta yes packet-filter yes	Show global counters that have changed, filtered by current packet capture filters (if any set). Useful for seeing specific counter changes during a test.
show running decryption-policy	Displays the configured decryption policies.
show running decryption-profile	Displays the configuration of a specific decryption profile.
show system setting L7-debug	Shows if Layer 7 debugging is enabled (relevant for debug commands).
show system setting ssh-proxy	May show specific system-level settings or status for SSH proxy (availability may vary by PAN-OS version).
show system resources follow	Real-time view of CPU and memory utilization, similar to `top` in Linux. Helps identify resource exhaustion.
show system statistics	General system health statistics.
show jobs all	Shows status of recent jobs like commits or software updates.
less mp-log sshd.log	(If available and applicable to the proxy component) Check management plane SSH daemon logs. Specific log file names may vary.
less mp-log devsrv.log	Device server logs on the management plane. Might contain information related to decryption key generation or proxy service status.
less dp-log pan_l7_proxyd.log	(If this specific log exists for SSH proxy) Dataplane Layer 7 proxy daemon logs. The actual log name for SSH proxy might differ; explore `dp-log` options.

Debug Commands (Use with Caution):

Warning: Debug commands can generate a large amount of output and consume significant CPU resources. Use them sparingly in a controlled manner, preferably during a maintenance window or under the guidance of Palo Alto Networks support. Always remember to turn off debugs after troubleshooting.

Command	Description
set L7-debug on severity debug	Enables Layer 7 debugging. Specific sub-modules for SSH might exist. (Older command, may be replaced by dataplane debugs).
debug L7-debug	(Older PAN-OS) Toggles Layer 7 debugging.
debug decryption L7-debug on	Enables debugging for the L7 decryption process. Check if this provides SSH Proxy details.
debug dataplane packet-diag set filter match source destination destination-port 22	Sets a filter for packet diagnostics. Add protocol 6 for TCP.
debug dataplane packet-diag set capture stage ?	Shows available capture stages (e.g., `receive`, `firewall`, `transmit`, `drop`).
debug dataplane packet-diag set capture stage firewall file ssh_capture.pcap	Captures packets at the firewall stage matching the filter to a file.
debug dataplane packet-diag set capture on	Starts the packet capture.
debug dataplane packet-diag set capture off	Stops the packet capture. (PCAPs can be viewed in Wireshark).
debug dataplane pool statistics	Shows statistics for various dataplane resource pools, which can indicate exhaustion.
debug user-id log-all on	If User-ID is involved with policies applied to SSH traffic. Remember to turn off.
debug device-server show	(May require specific context or role) Show debug information for device server processes.
debug log-receiver statistics	(May require specific context or role) For troubleshooting log forwarding issues, if logs about SSH proxy are expected elsewhere.
clear L7-debug	Clears the L7 debug settings.
debug decryption L7-debug off	Turns off decryption L7 debug.

To view the logs generated by some debug commands (especially older ones or management plane logs): less mp-log or less dp-log . For live tailing: tail follow yes mp-log .

5.3. Packet Captures

Packet captures are invaluable for understanding network interactions:

• Client-Side Capture: See what the client sends and receives, including SSH handshake messages and host key presentations.
• Firewall Ingress/Egress Capture: Use debug dataplane packet-diag or the GUI ( Monitor > Packet Capture ) to capture traffic as it enters and exits the firewall. This helps verify NAT, routing, and what the firewall does with the packets.
• Server-Side Capture: See what the server receives from the firewall and what it sends back.

Compare captures at different points to isolate where the SSH negotiation is failing or data stops flowing.

Figure 5: State Diagram of an SSH Proxied Session

6. PCNSE Exam Insights & Gotchas

For the Palo Alto Networks Certified Network Security Engineer (PCNSE) exam, understanding SSH Proxy is crucial, especially its capabilities, limitations, and configuration nuances.

Key Areas for PCNSE Takers:

• Purpose of SSH Proxy: To decrypt and inspect SSHv2 traffic, primarily to prevent SSH tunneling of unauthorized applications or threats.
• No Certificate Requirement: Unlike SSL Forward Proxy or SSL Inbound Inspection, SSH Proxy does not require administrator-managed certificates . The firewall auto-generates its proxy key. This is a frequent exam point.
• Policy Configuration:
  • Action: Decrypt
  • Type: SSH Proxy
  • Attaching a Decryption Profile is a best practice to control unsupported versions/algorithms.
• Impact on Key-Based Authentication: Fundamentally incompatible. Solutions are password authentication or exclusion. This is a major gotcha.
• Tunneling Detection: The firewall uses App-ID on the decrypted traffic to identify channel types like `session`, `X11`, `forwarded-tcpip`, or `direct-tcpip`. The latter three are typically identified as `ssh-tunnel` and can be blocked by Security policy.
• SSH Version Support: Only SSH version 2 is supported for decryption. Sessions attempting SSHv1 will be blocked if the Decryption Profile is configured to do so.
• Decryption Logs: Be aware that SSH Proxy traffic might not appear in Decryption logs. Focus on Traffic and Threat logs for insights into inspected content.
• QoS: Proxied SSH traffic does not support DSCP/QoS.
• Comparison with `ssh` application vs. `ssh-tunnel` application:
  • Security policy allowing `ssh` permits the SSH protocol itself.
  • SSH Proxy decrypts this allowed `ssh` traffic.
  • If tunneling is detected (e.g., X11, port forwarding) within the decrypted SSH session, it's identified as the `ssh-tunnel` application.
  • A separate Security policy rule is typically needed to explicitly Deny the `ssh-tunnel` application.

Critical Information & Gotchas Summary:

Common PCNSE Exam Questions Themes:

• Prerequisites for SSH Proxy (e.g., are certificates needed?).
• How SSH Proxy handles key-based authentication.
• The purpose of a Decryption Profile in an SSH Proxy context.
• How the firewall detects and blocks SSH tunneling when SSH Proxy is enabled.
• Differences in logging between SSH Proxy and SSL Decryption.
• The type of interfaces SSH Proxy can be configured on.

7. Summary

Palo Alto Networks' SSH Proxy is a powerful feature for gaining visibility into encrypted SSHv2 traffic. Its primary goal is to prevent the abuse of SSH for tunneling unauthorized applications and to enable threat inspection. Configuration is relatively straightforward, involving a Decryption policy rule with the "SSH Proxy" type and an optional, but highly recommended, Decryption Profile to handle unsupported SSH versions and algorithms.

The most significant challenge and "gotcha" is its incompatibility with public key-based authentication due to its man-in-the-middle architecture. Administrators must plan for this, either by using password authentication for proxied sessions or by strategically excluding traffic that strictly requires key-based methods.

Troubleshooting involves checking firewall logs (Traffic, Threat, System), using CLI `show` commands for status and statistics, and cautiously employing `debug` commands for deeper insights. Understanding the packet flow and the distinct roles of the `ssh` application versus the `ssh-tunnel` application is vital for effective policy enforcement. While SSH Proxy enhances security by decrypting and inspecting SSH, awareness of its limitations, such as the impact on key-based logins and QoS, is crucial for successful deployment and for PCNSE exam preparedness.