RADIUS Lock Out – Warning

So I was deploying my superuser RADIUS solution to our R75.46 gateways and locked myself out of one box. Could not even log in at the console. Turns out it was a R75.40 unpatched system and RADIUS was broken and ONLY did RADIUS auth and nothing else. Not even local authentication. Something went wrong with the PAM module and bypassed the PAM_UNIX processing.

The secret to get in was to pull the network cable(another guy Dan figured this out). Some sort of race condition between the cable and the console. Geez louise.

Make sure you have these patches.

pam-0.99.6.2-3.26.cp986008001
CPshell-1-986008001

dreez

 

How to migrate firewalls between domains

This is in rough shape, I haven’t sanity checked it. Just did this and slammed this together so I wouldn’t forget. Hope it is helpful.

1.  mdsenv <DOMAIN>
2. mcd bin/upgrade_tools
3. ./migrate export /var/log/tmp/givenameofexportfile.tgz
4. Create new DMS, do NOT start it
5. Import the tgz file
6. You will get warnings that the ICA is the same as another domain. Ignore for now.
7. Do NOT start the DMS
8. Make backups of the objects_5_0.C file!!! Can’t lose this
9. Make sure you delete the VPN cert or it will complain when you import into the new domain. You can put your cursor by the :certificate and do a ‘d%’ and vi will delete 25 lines withing the enclosed brackets. Do this for every VPN cert. You will have to recreate if you are using VPNs.
10. If you import global objects for some reason you will get duplicates when you re-import. You can make them local by
    mdsstop_customer DOMAIN
    vi objects_5_0.C
    :.,$s/global_level(1)/global_level(0)/g
    mdsstart_customer DOMAIN
11. You can then delete them en-masse in dashboard. You can select multiple at a time and delete:
    
12. Continue on my other blog  to create ICA and resic

R80 gossip – Fixing licensing again?

 

I”m dying to see the new R80 mgt station at CPX. Expectations are running very high because at the core Check Point is all about Management and Logging — That is their strength. SmartLog/SmartEvent are a fantastic team on the logging side.  Management/MDM/P1 has been a leader in the enterprise market for years…but recently has lost some ground to competitors as CheckPoint has been spending R&D on ancillary products. So this is the year for CP to regain the lead at its core.

1. One piece of gossip I’ve heard from two sources is that they are taking yet another run at fixing licensing. NO unfortunately they are not going to kill it totally. Supposedly in R80 they are going to finally tie together the user center and SmartUpdate. So now you may even will be able to compare what you bought (in user center) and compare it to what you deployed (in SmartUpdate). Today it is a ludicrous bug ridden acronym laden beast of burden on humanity. You have to tie the two together using comment fields in User Center…except these fields are limited and can’t be exported and blah blah blah.  I have perl scripts that pull license keys from User Center and pull “cplic prints” from Mgt/Gateways and compare.  This is the only way we can tell if we are using what we actually paid for and what we have extra.
2. They have moved the Israeli product support team into R&D for R80. The management upgrade will be a HUGE job so they want the support team to understand all the details so they can support their customers.. YEAH!!!
3. Bone up on your perl and SQL. The whole backend will go from flat files into SQL database. So all our tools we’ve written over the years will be for not….but its about time.
4. Some of the virtual stuff sounds very interesting. In The Cloud VMs float as needed and policy and management has to follow them as they migrate. Want to hear more about this.

 

 

 

How to SIC without losing state

We recently migrated firewalls between domains without dropping a packet. I am still a little bit in shock. This is a very very delicate operation, and considering all the problems with R75.4X, we didn’t run into a single bug.

Anyways maybe I’ll write that up in a future blog. But here was the magic for re-SIC’ing the DMS and the gateways without have to restart or reboot or lose any traffic, for both clusters and standalone gateways.

SIC primer. When a DMS wants to talk to a Gateway it does it over an encrypted tunnel. The encrypted tunnel can be called the SIC tunnel. There are two components to SIC, the mgt station and the gateway. Each component has its own private keys and certs. When you SIC then they exchange their keys and setup a mgt tunnel. (OK, that is the quick version, more detail here).

Normally on the the gateway when you re-SIC, you have to restart services.  OK for a cluster (do one and fail over do the other), not so good on a standalone…you lose traffic when you restart. Well when migrating firewalls between domains you have to re-SIC because the mgt station has a new IP address so you have to SIC the gateway to the new IP address of the domain. Its almost like you are taking Member A out of the cluster and putting into the another cluster. Why is this a big deal? Because when Member A is re-SIC to the new domain, it loses policy and reverts to initial DROP ALL policy. So now you cannot failover from Member B to Member A because they have different policies. With this process (below) you can re-SIC and remain active/standby throughout the process without dropping a packet. Pretty cool ‘eh?

CPD is the process on the mgt and gateway that maintain SIC. So the magic juju here is to only deal with CPD and not disturb the other processes on the gateway.

SOOOO. Our Diamond Engineer Taylor came up with a way to do this without losing state HERE is some of the basics and below I will comment on what they do.

ON THE MGT:

1. mdsenv MDS_NAME
2.  fwm sic_reset  – This will destroy the cert authority on the DMS and create a new one. Obvious you will lose communication with gateways if any exist.
3. $MDSDIR/bin/mdsconfig -ca <DMS_NAME> <DMS_IP> – This will restart the CA in the DMS

So at this point the DMS MGT has destroyed old keying material and created new keying material and is ready to re-SIC with gateways.

ON THE GATEWAY

1. cp_conf sic init abc123 norestart – This will reset the private keys on the gateway without restarting anything.
2. cpwd_admin stop -name CPD -path "$CPDIR/bin/cpd_admin" -command "cpd_admin stop"       – THIS WILL STOP CPD
3. cpwd_admin start -name CPD -path “$CPDIR/bin/cpd” -command “cpd”    —– THIS WILL START CPD with the new keys installed
4. cpwd_admin list —- will list the CP processes that are watched by the watcher daemon (restart in case they die)
5. cpwd_admin -?  — will give you tons of choices to choose from

So at this point the gateway is ready to be re-SIC’d with the management using the secret password “abc123” (only used one time and not used again, can be simple just used for setup and exchanging key material). AND – bonus points is…. the gateway has not reloaded the initial policy. It is still running with the old policy so if it was a standby member you can fail over to it if you wanted.

ON THE MGT:

In the communication window for the gateway, enter the secret key ‘abc123’. This will allow the MGT and the GATEWAY to exchange key information and setup the SIC tunnel.

 

 

SIC ON!!

dreez

 

Mounting NTFS shares from GAIA

I lately have been archiving log files to Windows NTFS shares lately using this:

Mounting NTFS shares from Unix

GAIA has all the right drivers and seems to work great. Probably obvious but I find it useful instead of trying to tftp, ftp, ssh tunnel, curl files to/from GAIA. Use it when I want to do auditing, or archiving log files, or take backups I can just to a ‘cp’ instead of tftp.

dreez

 

 

Finally! be able to debug application control!!! New fw ctl debug flag sheet

Long time ago I asked Sergei if he could update this critical document. Sergei spent months updating the fw ctl debug flags. Then he spent more months getting the bureaucracy to release it to us unwashed peasants. Up to today it has been a bitch debugging things like application control because you can’t see inside the monster. Thanks to Sergei we now have a chance of fending for ourselves. If you get a chance please thank Sergei for his work (via linkedin) and provide comments. He will be glad to incorporate them.  I am being dramatic here because the more positive feedback we supply the the more internal information they will release in the future. This is a huge win-win for all of us.

HERE IT IS Sergei’s 2014: FW CTL DEBUG FLAGS.

 

1. 1) Run this command to see all supported/existing flags on your version:
   # fw ctl debug -m
2. 2) After enabling debug flags for a module, run this command to verify:
   # fw ctl debug -m MODULE

 

For those who live in SK land like I do, hopefully you all realize how the quality of the SK’s have improved this past year. Sergei Shir  and his crew:

1) Knowledge Center Manager – Uri Lewitus  (all the major changes in the quality and quantity of solutions are his initiative + he released my debugging document) – https://www.linkedin.com/profile/view?id=103729954
2) Knowledge Center Technology Leader – Ronen Zel (his knowledge of our internal systems is crucial to our team)
3) Knowledge Content Developer – Stella Shteinbuk – https://www.linkedin.com/profile/view?id=972764
4) Knowledge Content Developer – David Kornfield

are knocking them dead. Its a little bit sad that people like Sergei don’t get more credit when people like us are keeping 25% of the world’s economy online (well, CP is in all the top Fortune 100 so made that number up) thanks to people like Sergei having our back. Please make sure and provide comments on the SKs so they can show upper management how important they are to keeping CheckPoint on top of its game. Win-Win-Win.

Number of log events per day and second

Ever want to know how many log events per day and per second (in a day) you are generating??? Derived from this SK.

You can run this on your MLM MDS from the root mdsenv and NOT inside a domain context.

——————————————————-

#!/bin/sh
. $CPDIR/tmp/.CPprofile.sh
# Print log rate data on all Domains
echo DOMAIN , DAY , LOGSPERDAY , LOGSPERSECOND
for DOMAIN in $(ls -1 $MDSDIR/customers)
do
mdsenv “$DOMAIN” > /bin/null
mcd log > /bin/null
ls -al *.logptr | awk ‘{split($9,a,”_”);print a[1],$5 }’ | sort | awk -v mds=$DOMAIN -F ” ” ‘{s[$1]+=$2} END {for (i in s) printf “%s,%s,%d,%f\n”, mds,i,s[i]/4,s[i]/4/86400} ‘ | sort
done

exit 0

—————————————————————

Try this environment too

. $MDS_CPDIR/tmp/.CPprofile.sh
. $MDS_SYSTEM/shared/bin_definitions.sh

License mumbo jumbo

This is more of a “NOTE TO SELF”. I have a licensing brain block so I have to track these tidbits….

When firewalls have LOCAL licenses.

FIREWALLXXX  has two licenses generated:

1) FIREWALLXXX  has a local gateway license that obviously let’s it do firewall stuff
2) FIREWALLXXX has a management portion with 3 features

FIREWALLXXX license on CMAXXX/DOMAINXXX
——————————————–
# cplic print -D
1.2.3.4  never CPVP-SNX-5-NGX CPSB-SWB CPSB-ADNC-M CK-00-1C-7F-XX-XX-XX
———————————————
1) CPVP-SNX-5-NGX : Manage SSL VPN for 5 users – we can ignore not used
2) CPSB-SWB: Turns the archaic SNX-NGX license into a blade license – we can ignore
3) CPSB-ADNC-M : Allows the management station to administrate a ACTIVE-ACTIVE load balanced cluster – we can ignore not used

Summary: As long as a management station has the DMN-U license it can manage most functions of a firewall.

 

 

I”m a CCSE again

So I”ve gone dark again because I was in a panic studying for  my CCSE renewal. I totally suck at taking tests, freeze up like a popsicle.

Got an 85. Ugh. Test was easy and I swore I was going to get 100% because I knew every answer and could almost see the answer in the manual where the answer appeared. So I’m a little freaked that I got 13 wrong – is it me or the test?

Overall the practice test and real test I felt were 75% fair. I still think they test you on the minutiae like “what file VTI custom routes in” instead of “What are the sequence of steps that occur during cluster failover”…”How to debug cluster failure”…”How to copy rules and objects between Domains”.

Here are some hints:

1) Major topics to know

• Clustering modes
• Cluster commands – fw ctl pstat, cphaprob stat, list, if, set_ccp
• Cluster upgrade modes
• Upgrades and tools to upgrade and how they work
• Smartevent – The components and what they do
• VPN – How to debug, domain vs route mode
• PKI – The components
• Route based VPN and VTIs
• VPN command line –
• Smart center and HA mode
• Backups vs snapshot
• IPSEC
• SecureXL and commands – fwaccel
• CoreXL
• SmartWorkflow

2) One or NO questions on:

• QOS – 1
• LDAP – 2 – LDAP groups
• SmartLSM – 1
• IPS – 1 or 2
• DLP NONE
• SmartMonitor none
• SSL VPN none

3) Studying

• ALL the questions are online. I got a 600 question sampler from a testing firm and the first 300 questions covered the whole test. 90% of the answers were correct…so you never knew what was correct/wrong so I still researched each question.
• You will get a better score if you never touched CheckPoint. People in our group that barely touch CP were getting in the 90% because they just memorized things. Our experienced people were getting 70-80% because they know the real answers

 

I would say go for it and take the test. Great for your resume and if you study it should not be a problem.

CCSE ON!

dreez