This article focuses on explaining the meaning of 'error subcode 5 (Connection rejected)' while establishing BGP between two firewalls.
If a BGP speaker decides to disallow a BGP connection ( e.g., the peer
is not configured locally ) after the speaker accepts a transport
protocol connection, then the BGP speaker SHOULD send a NOTIFICATION
message with the Error Code Cease and the Error Subcode " Connection
Rejected ".
This means that after the initial TCP handshake between the BGP peers, when peer A receives an OPEN message from peer B, and peer A does not recognize peer B, it would send a Notification message with Subcode " Connection Rejected "
Assume the following topology :
PA-1 (192.168.30.1) ----- (192.168.30.2) PA-2
PA-2 has a misconfigured peer IP address : (instead of 192.168.30.1 it is configured as 192.168.30.3)
As soon as PA-2 (192.168.30.2) receives a OPEN message from PA-1, it sends a Notification message :
PA-1 shows this notification message being received and error code in routed.log: