How to SIC without losing state

We recently migrated firewalls between domains without dropping a packet. I am still a little bit in shock. This is a very very delicate operation, and considering all the problems with R75.4X, we didn’t run into a single bug.

Anyways maybe I’ll write that up in a future blog. But here was the magic for re-SIC’ing the DMS and the gateways without have to restart or reboot or lose any traffic, for both clusters and standalone gateways.

SIC primer. When a DMS wants to talk to a Gateway it does it over an encrypted tunnel. The encrypted tunnel can be called the SIC tunnel. There are two components to SIC, the mgt station and the gateway. Each component has its own private keys and certs. When you SIC then they exchange their keys and setup a mgt tunnel. (OK, that is the quick version, more detail here).

Normally on the the gateway when you re-SIC, you have to restart services.  OK for a cluster (do one and fail over do the other), not so good on a standalone…you lose traffic when you restart. Well when migrating firewalls between domains you have to re-SIC because the mgt station has a new IP address so you have to SIC the gateway to the new IP address of the domain. Its almost like you are taking Member A out of the cluster and putting into the another cluster. Why is this a big deal? Because when Member A is re-SIC to the new domain, it loses policy and reverts to initial DROP ALL policy. So now you cannot failover from Member B to Member A because they have different policies. With this process (below) you can re-SIC and remain active/standby throughout the process without dropping a packet. Pretty cool ‘eh?

CPD is the process on the mgt and gateway that maintain SIC. So the magic juju here is to only deal with CPD and not disturb the other processes on the gateway.

SOOOO. Our Diamond Engineer Taylor came up with a way to do this without losing state HERE is some of the basics and below I will comment on what they do.


  1. mdsenv MDS_NAME
  2.  fwm sic_reset  – This will destroy the cert authority on the DMS and create a new one. Obvious you will lose communication with gateways if any exist.
  3. $MDSDIR/bin/mdsconfig -ca <DMS_NAME> <DMS_IP> – This will restart the CA in the DMS

So at this point the DMS MGT has destroyed old keying material and created new keying material and is ready to re-SIC with gateways.


  1. cp_conf sic init abc123 norestart – This will reset the private keys on the gateway without restarting anything.
  2. cpwd_admin stop -name CPD -path "$CPDIR/bin/cpd_admin" -command "cpd_admin stop"       – THIS WILL STOP CPD
  3. cpwd_admin start -name CPD -path “$CPDIR/bin/cpd” -command “cpd”    —– THIS WILL START CPD with the new keys installed
  4. cpwd_admin list —- will list the CP processes that are watched by the watcher daemon (restart in case they die)
  5. cpwd_admin -?  — will give you tons of choices to choose from

So at this point the gateway is ready to be re-SIC’d with the management using the secret password “abc123” (only used one time and not used again, can be simple just used for setup and exchanging key material). AND – bonus points is…. the gateway has not reloaded the initial policy. It is still running with the old policy so if it was a standby member you can fail over to it if you wanted.


In the communication window for the gateway, enter the secret key ‘abc123’. This will allow the MGT and the GATEWAY to exchange key information and setup the SIC tunnel.







Post a comment or leave a trackback: Trackback URL.


  • John Fleming  On December 22, 2016 at 11:25 am

    Have you tried this yet? Might as well stick with cli on mgmt 😀
    mgmt server
    push_cert –s CMAIP –u admin –p adminpw –o examplegw –k test123


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Helen's Loom

"Peculiar travel suggestions are dancing lessons from God." - Kurt Vonnegut

Life Stories from Dreez

These are stories from my travels. Generally I like to write stories about local people that I meet and also brag about living the retirement dream with my #1 wife Gaby. She is also my only wife.

%d bloggers like this: