Zero Downtime Upgrade between major versions WITH/OUT dynamic routing

Good news:Can be done possible
Bad news: This is work in progress, hope to update with pictures. If you call CP support, they might be able to fish up the document.

Overview:

  1. Go through CP steps for zero time upgrades. But don’t take them toooooo seriously or you will have surprises. Make sure you do these steps.
  2. Run the upgrade on the standby – DO NOT REBOOT
  3. If you have to copy fwkern.conf from the ACTIVE member ..do it now
  4. control_bootsec – install initial policy and makes sure that the default filter (bricks the firewall) is not loaded. Run from UPGRADE file system, not old file system.

    cd /opt/CPsuite-R77/fw1/bin

    bash

    control_bootsec

  5. Reboot standby
  6. Standby comes back up “Active Attention” – no problem has no cluster policy
  7. In dynamic routing, if you have “Wait for Clustering” enabled. Disable it. Let the routed startup without a cluster
  8. Start/Stop routed:
    tellpm process:routed
    tellpm process:routed t
  9. On mgt server change policy to latest version  R77.10/20/30 and push to upgraded member (uncheck mark in policy install for cluster push). Upgraded member now knows it has to be part of a cluster. It will go to READY state, waiting for the failover
  10. Use this script to export the routes off the ACTIVE firewall onto the Standby firewall. It will turn them into STATIC routes. NOTE: There is no ‘save config’ at the end. This are only temporary until the system reboots and get real OSPF routes. Make sure you differentiate between dynamic routes that will go away on reboot and real static routes that will be kept on reboot.
  11. Reboot the READY firewall just to clear out the cobwebs.
  12. Run the ospf script on the READY firewall. This will load all the OSPF and STATIC routes onto the firewall. NOTE: YOu will have to decide if you want to keep/delete the STATIC routes. You might have to SAVE CONFIG on the static routes if you want to keep them.
  13. Do a netstat -an | wc -l and fw tab -t connections -s to metric the routes and states
  14. Do a ‘cphaprob stat’ to get the IP and ‘number ID’ of the ACTIVE member.
  15. Now on the READY member PULL the state table from the ACTIVE member.cphaprob stat   –
    Retrieve the cluster NUMBER and sync IP of the ACTIVE membercphacu start <Active Member IP> <Cluster member Number>  –
    So if active was 1.1.1.1 and number 2 in cluster:
    cphacu start 1.1.1.1 2
    Will pull the state table from the ACTIVE onto the READY member. This is like the OLD fcu command…but snazzier somehow.
  16. Do a netstat and fw tab -t connections and make sure the numbers are about the same on both members
  17. On the ACTIVE member – drum roll.
    cphaprob stop
  18. On the DOWN member STOP the routing daemon because you don’t want it to fight with the new ACTIVE member. This is where the checkpoint cluster and routing teams never broke bread and coordinated cluster & routing activity and you have to do it manually.tellpm process:routed
  19. The READY member will now go to ACTIVE
  20. On ACTIVE member check out the state tables and network tables again. OSPF should be populating. Check the neighbor status to see if OSPF neighbors are negotiating. If not, they are stuck, then stop and restart. No worries you have static entries until you reboot.clish> show ospf neighbors
    clish> show route ospftellpm process:routed         ##### stop
    tellpm process:routed t        #### start
  21. You are over the hump, congrats
  22. Upgrade the OLD system
  23. Copy fwkern from the standby if required
  24. Reboot
  25. Push policy to both members
  26. Reboot both (to clear out static network entries and cobwebs)
  27. Done
Advertisements
Post a comment or leave a trackback: Trackback URL.

Comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

blog.lachmann.org

Michael Endrizzi's - St. Paul MN - CheckPoint blog on topics related to Check Point products and security in general.

%d bloggers like this: