Prisma Access: Operations, Monitoring & Troubleshooting

Effective operation of Prisma Access relies on proactive monitoring, robust logging and reporting capabilities, efficient troubleshooting methodologies, and a clear understanding of the update processes. This document covers these critical operational aspects.

Part 1: Monitoring Prisma Access Health

Regularly monitoring the health and performance of your Prisma Access deployment is crucial for identifying potential issues before they impact users and for ensuring optimal service delivery.

Using the Prisma Access Insights Dashboard

Key Metrics to Monitor

Alerting Configuration

Part 2: Logging and Reporting

Prisma Access leverages Cortex Data Lake (CDL) for centralized, scalable logging. Accessing and interpreting these logs is vital for troubleshooting, security analysis, and compliance reporting.

Querying Logs in Cortex Data Lake

Conceptual Log Query Process

flowchart TD
    A[Admin Needs Info] --> B{Choose Access Method};
    B -- Panorama --> C[Panorama Monitor Tab];
    B -- Cloud UI --> D[Prisma SASE Monitor];
    B -- XDR --> E[Cortex XDR Explore];
    B -- Hub --> F[CDL App Query];

    subgraph Query Interface
        C --> G{Build Filter / Query};
        D --> G;
        E --> G;
        F --> G;
    end

    G -- Query Sent --> H((Cortex Data Lake));
    H -- Log Results --> I{Display Results};

    subgraph Results Display
        C --> I;
        D --> I;
        E --> I;
        F --> I;
    end

    I --> J[Admin Analyzes Logs];

     

Using Predefined and Custom Reports

Interpreting Different Log Types

Part 3: Troubleshooting Common Issues

A systematic approach is key to resolving issues efficiently.

Mobile User Connectivity Problems

Troubleshooting Steps:

  1. Check Client Status & Logs: Start with the GlobalProtect client itself. Check the status panel for error messages. Collect logs from the client (Settings > Troubleshooting > Collect Logs).
  2. Verify Network Connectivity: Can the user reach the GP Portal address from their current location? Check local firewall, ISP issues.
  3. Authentication Failures:
    • Check credentials, account lockouts (AD/IdP).
    • Examine GP System Logs on Panorama/CDL for detailed authentication failure reasons (filter by user, `subtype eq globalprotect`).
    • Verify SAML IdP configuration and status. Check IdP logs.
    • Check RADIUS/LDAP server connectivity and configuration if used.
  4. Gateway Connection Issues:
    • Check if Portal assigned gateways correctly.
    • Look for tunnel negotiation errors in GP System Logs.
    • Verify IP pool availability for mobile users.
    • Check Security Policies blocking GP traffic (less common).
  5. Split Tunnel Problems:
    • Verify split tunnel configuration (include/exclude lists, applications, domains) in the GP Agent config (Portal).
    • Check client's effective routes (e.g., `ipconfig /all`, `route print` on Windows). Is traffic destined for the tunnel or direct?
    • Confirm DNS resolution is working correctly for split-tunneled domains.
  6. HIP Failures:
    • Check HIP Match logs on Panorama/CDL (filter by `subtype eq hip-match`). See which object/profile failed.
    • Verify the HIP configuration on the client matches the requirements (e.g., AV running, disk encrypted).
    • Check Security Policy rules using the HIP profile – are non-compliant users blocked as expected?
  7. Performance Issues: Check latency to the connected gateway (GP Client Status panel). Test throughput. May indicate local network issues or needing to connect to a closer gateway.

Remote Network Tunnel Down Issues

Troubleshooting Steps:

  1. Check Status (Prisma Access & Branch): Verify tunnel status in Prisma Access Insights / Cloud Services Status / Monitor tab. Check status on the branch device CLI/UI.
  2. Verify Configuration Parameters: Mismatched parameters are the most common cause. Double-check meticulously on both Prisma Access (Panorama/Cloud UI) and the branch device:
    • IKE Version (v1/v2)
    • IPSec/IKE Crypto Profiles (Encryption, Auth, DH Group, Lifetimes - ensure *exact* match)
    • Pre-Shared Key (check for typos, complexity issues) or Certificate validity/trust.
    • Local/Peer IDs (IP Address, FQDN, Key ID - must match expectations).
    • Tunnel Interface IP addresses (if numbered).
    • Proxy IDs (if used - ensure match or use route-based 0.0.0.0/0).
  3. Check Network Connectivity: Can the branch device's public IP reach the Prisma Access public IPs for the assigned PoP? Check intervening firewalls, routing, ISP issues.
  4. Examine Logs:
    • Prisma Access:** Check System logs (`subtype eq vpn`) on Panorama/CDL. Filter by the branch peer IP. Look for IKE Phase 1 / Phase 2 negotiation errors (e.g., "No proposal chosen", "Authentication failed", "Invalid ID").
    • Branch Device:** Check the corresponding VPN/IPSec/IKE logs on the branch device. Error messages often provide specific clues.
  5. Debug Commands (Branch Device - Examples): Commands vary by vendor.
    • Palo Alto NGFW (On-Prem):
      debug ike global on debug
      debug ike pcap on 
      less mp-log ikemgr.log
      less mp-log vpn.log
      show vpn ike-sa gateway 
      show vpn ipsec-sa tunnel 
      clear vpn ike-sa gateway  
      clear vpn ipsec-sa tunnel  
                              
    • Cisco IOS Router:
      debug crypto isakmp
      debug crypto ipsec
      show crypto isakmp sa
      show crypto ipsec sa
      clear crypto isakmp 
      clear crypto ipsec sa peer 
                              
  6. BGP Issues (If Tunnel is Up):
    • Check BGP neighbor status on both Prisma Access (Monitor tab or CLI if accessible via support) and branch device. Look for Idle, Active, Connect states (should be Established).
    • Verify BGP ASN, neighbor IPs, timers match.
    • Check route advertisements and received routes. Are expected prefixes being sent/received? Apply BGP filters if needed.
    • Examine System logs (`subtype eq routing`) on Prisma Access and BGP logs on the branch device.
    • Palo Alto NGFW BGP Commands:
      show routing protocol bgp summary
      show routing protocol bgp peer 
      show routing protocol bgp rib-out neighbor 
      show routing protocol bgp rib-in neighbor 
                              

Service Connection Problems

Policy Enforcement Issues

Troubleshooting Steps:

  1. Identify the Specific Flow: Source IP/User, Destination IP/URL/App, Port/Service.
  2. Check Traffic Logs: Filter logs by the specific flow parameters.
    • Rule Match:** Which Security Policy rule did the traffic match? Is it the expected rule?
    • Action:** Was the action allow/deny as expected?
    • Session End Reason:** If denied/dropped, why? (e.g., "policy-deny", "threat", "url-block"). If allowed but app doesn't work, could be "tcp-reset-from-server" or other application-level issue.
  3. Verify Rule Order: Remember top-down evaluation. Is a rule higher up matching unexpectedly?
  4. Check Policy Objects: Are Address Objects, Service Objects, App-IDs, URL Categories, User/Groups defined correctly and accurately matching the traffic? (e.g., typo in IP, wrong App-ID, user not in expected group).
  5. Verify User-ID / Group Mapping: Check User-ID logs. Does Prisma Access have the correct IP-to-User mapping? Is the user correctly mapped to the group used in the policy?
  6. Verify HIP Profile Match: Check HIP Match logs if the policy uses HIP profiles. Is the device passing/failing the check as expected?
  7. Check NAT Policy: Is NAT being applied correctly (especially Source NAT for outbound)? Check NAT policy rules and Traffic logs (NAT source/destination fields).
  8. Check Decryption Policy: Is traffic being decrypted/not decrypted as intended? Is an exclusion rule matching unexpectedly? Are clients trusting the forward-trust CA?
  9. Use Policy Optimizer (Panorama): Helps identify unused rules, rules with port-based services that could use App-ID, shadowed rules.
  10. Global Find (Panorama): Search for objects (addresses, services) to see where they are used in policies.

Performance Issues (Latency, Throughput)

Troubleshooting Steps:

  1. Isolate the Scope: Is the issue affecting specific users, sites, applications, or destinations? Is it consistent or intermittent?
  2. Check Prisma Access Insights: Review latency, bandwidth, packet loss metrics for the affected users/sites.
  3. Check Local Network: Crucial first step. Is the user's local Wi-Fi, ISP connection, or branch network experiencing issues? Run speed tests, ping tests to external sites *bypassing* Prisma Access if possible (e.g., using split tunnel).
  4. Gateway Selection (MU): Is the user connected to the optimal (lowest latency) gateway? (Check GP client status).
  5. Bandwidth Limits: Is the user/site hitting their allocated bandwidth limits? Check Insights. Consider QoS.
  6. Application Performance: Is the issue specific to one application? Could it be the application server itself, not the network path?
  7. Decryption Impact: While generally well-handled, complex decryption scenarios could potentially add minimal latency. Test bypassing decryption for the specific flow if suspected (for testing only).
  8. Path Analysis: Use tools like `traceroute` or `mtr` (from the endpoint, if possible) towards the destination to identify potential high-latency hops along the path (which may be outside Prisma Access).
  9. Engage Support: If local network and Prisma Access metrics seem normal, but performance is poor, Palo Alto Networks support may need to investigate internal routing or SPN performance.

Panorama / Cloud Services Plugin Issues

  • Plugin Not Connecting/Authenticated: Verify Panorama has internet access, DNS resolution works, and the Panorama service account can authenticate to the Hub. Re-authenticate via `Cloud Services > Service Setup`.
  • Configuration Push Failures: Check Task Manager in Panorama for detailed error messages. Common causes: configuration errors (validation failures), plugin communication issues, transient cloud service problems. Review the configuration pushed for errors.
  • Status Not Updating: Ensure Panorama can reach the Cloud Services infrastructure. Check plugin logs (`less mp-log cloud_services.log` or via `Technical Support File`).
  • Plugin Version Compatibility: Ensure the Cloud Services plugin version is compatible with the Panorama software version and the Prisma Access backend version. Check release notes.
  • Resource Issues on Panorama: Ensure the Panorama VM/appliance has sufficient CPU/Memory/Disk resources, especially during large commits or log queries.

Part 4: Software Updates

Keeping components updated is essential for security patches, bug fixes, and new features.

Prisma Access Infrastructure Updates

  • Managed by Palo Alto Networks: The underlying infrastructure (SPNs, CANs, Gateways, Portals) software (PAN-OS version running in the cloud) is updated and patched by Palo Alto Networks.
  • Maintenance Notifications: Customers are typically notified in advance via email and/or Hub notifications about scheduled maintenance windows for infrastructure updates.
  • Version Selection: Customers usually select a preferred PAN-OS version for their Prisma Access instance from a list of qualified versions provided by Palo Alto Networks (configured via Cloud Services plugin or Cloud UI). Updates to new major versions might require customer scheduling/approval.
  • Hitless Updates (Goal): Updates are designed to be hitless or minimally disruptive, leveraging infrastructure redundancy.

GlobalProtect Client Updates

  • Deployment Methods:
    • Via GlobalProtect Portal:** Upload desired client versions to the Portal configuration (Panorama/Cloud UI). Configure agent settings to allow/prompt/force upgrades. Clients check the portal periodically.
    • Manual Deployment:** Distribute client installers via software deployment tools (SCCM, Intune, Jamf), email, or download links.
  • Version Control: Configure minimum allowed client versions in the GP Portal/Agent settings.
  • Testing: Recommended to test new client versions with a pilot group before broad deployment.
  • Release Cadence: Palo Alto Networks releases new GP client versions regularly with fixes and features. Check release notes.

Panorama Cloud Services Plugin Updates

  • Manual Process: Updating the plugin on Panorama is a manual process.
  • Steps:
    1. Download the new compatible plugin version from the Support Portal.
    2. Upload and install via `Panorama > Plugins`. (Requires Panorama restart).
  • Compatibility: Crucial to check release notes for compatibility between the plugin version, Panorama software version, and the Prisma Access backend PAN-OS version.
  • When to Update: Update when required for compatibility with new Panorama/Prisma Access features, for bug fixes listed in the release notes, or as advised by support.