Monthly Archives: December 2011

MDM and RAID config

In my latest class we had a discussion about recommendations for RAID drives in MDM environments. To be honest I haven’t thought through the pluses and minuses of each configuration, but these are my initial thoughts so please feel free to add/comment on them.

MDS: RAID 1 or clone, RAID10: Why: Performance is not an issue. Upgrades and redundancy are critical. When you upgrade you want to be able to go backwards and forwards quickly during 3 hour upgrade windows. No time for reverts and snapshots (besides they are broke in 75.20).
MLM: RAID 5 or RAID 10. Write performance is somewhat critical but not really I worked in a huge MDM environment with multi-gig a diay of logs…but all log servers were on a 100Mb link. Upgrades…who cares, if they screw up DMSs log locally and dump to the MLML later.
Gateways: RAID1/Clone. Performance not an issue. Upgrades are very difficult to want to be able to recover.. Redundancy and single drive failures are also critical.
Smartstations: RAID1/Clone, Same as above.

That’s my world…what is your world?


Basics for any SPLAT debug geek

I thought this was a great debug cheat sheet. I’m going to be taking the Advanced Training Jan 9th-Jan 11th in Austin and I’m sure I’ll learn this all in detail and of course will share.


R75.20 MDM Nomenclature vs Provider-1

For those that are migrating….I’m sure this is elsewhere but I sure can’t find it.!

Tufin Application Policy Generator

Hey There,

I’ve been looking at the Tufin APG. This is a cool tool that will auto generate a policy based on the traffic
it sees in the logs. You can use this for:

1) M&A’s: see what traffic is coming/going to new division prior to integration and build new policy for them
2) Rule Review: Compare current policy to recommended policy from APG
3) Fine Tune: Creating more permissive or more fine grained rule…use it on per rule basis

I only looked at some small logs and the note of caution is you HAVE to do filtering prior to loading logs
into APG. Drop all the DROP traffic or single hit traffic. Otherwise you will need a CPU cloud to generate
the new rulebase. Its exponential, so every minor log entry kill will probably buy you 1 minute (I exagerate).

Here is my admittedly quirky review:




Nov 15th, 2011 MDM/P1 Group Meeting Minneapolis MN

posted Nov 16, 2011 2:29 AM by Michael Endrizzi                                                                                               [

Once again our MDM user group meeting was well attended, 40 people, biggest attendence yet. Shawna and CheckPoint team really did a bangup job hittin’ it and herding the cats. Obviously the venue was a big hit (Fago Di Ciao), but also people love the technical content. We had Itai Greenberg come from Isreal to talk about VSX and Gaia releases coming out next year and some of their features. I didn’t write down the details, but here are some high points I can recall (sorry, there are lots of voids here)
GAIA: Release Q2? 64 bit. 1000 commands merge of IPSO and SPLAT.
           Note: No IPv6/IPv4 translation, all or nothing
VSX: Next version comes out on Gaia, they are providing VSX appliances.
VE: Architected so that any changes in VMware do not impact VE
MDM: Provisioning is up for a rehaul.
Thanks to Itai for his LONG travel in a short time…You could see he was tired but stuck with it just for us.
I gave a 5-minute talk on our MDM Enterprise Management class for newbies. Centralized management is a double-edge sword. It can decrease TCO, but doing an Assign-All Domains-Install by a newbiecontractor will quickly lead to multi-million $$$ shutdowns. So we feel #1 Training is the only answer and #2 we will be putting up a Cloud CheckPoint Playland so our students can safely practice their new skills after they leave class. Also wanted feedback on putting up a Cloud VSX environment for our MDM group so they can safely play with VSX in and MDM environment.
I think the best part of all our MDM meetings is the social/technical interaction afterwards. Here is the creme’ di creme’ of large enterprise CheckPoint users having the ability to talk about what works and what doesn’t work, the good/bad/ugly unfiltered. People stayed until 10pm (I left at 9:30 and they were still going) just talking shop. Discussions I picked up/listened into between organizations:
    1. How to use MDM in an M&A environment
    2. How to structure workflow with MDM
    3. Cool backup solutions developed internally
    4. Solutions to enterprise mobile iPhone rollouts in an MDM environment
    5. How to migrate from large SmartCenter to MDM environment
    6. How to reduce rulesets using Tufin Auto Policy Generator
Hope we can keep these going!

Red Alert: R75 Snapshots are broken

posted Nov 15, 2011 7:29 AM by Michael Endrizzi                                                                                               [

Just found out from 2 customers that R75 snapshots are broken..kinda. If you are upgrading from R75 to R75.20 and it fails (and it will), you can’t just interrupt the boot and revert the snapshot. For some reason you have to rebuild the box as R75 with any hotfixes and THEN do a revert. UGH.
NOTE: When I was at CP support in Irving one advanced support person was trying to fix a huge bank that had corrupt backups/snapshots. They couldn’t go forward or backwards.
NOTE: I took apart the snapshot .tgz and it had every file on the SPLAT disk. I really don’t know why they can’t restore the snapshot…..
Read my blog below on how verifying restores and pulling RAID drives.

If I was CEO – MDM would be my only focus

posted Nov 5, 2011 6:39 AM by Michael Endrizzi                                                                                               [

Dear Gil,
As you know I am a total Check Point techie addict (aside from licensing). You have personally guided the company to be a billion dollar pillar in the security marketplace and have provided me a good living through it. I couldn’t pack your lunch. I know you receive hundreds of unsolicited opinions on how to run your business from the peanut gallery every day and choosing a winner is very complex, but overall you have picked winners. Hats off to you.
So here is another know-it-all opinion from the peanut gallery. Take the gems, flush the turds and go make another billion for all of us.
I understand that Check Point’s goal is to grow into a multibillion shekel security behemouth. The current strategy seems to be by offering every flash-in -the-pan security widget known to personkind to the marketplace. Today I noticed you bought another company Dynasec specializing in Governance and Compliance. So I ask myself how does Check Point or any company avoid the black hole of getting a sales force to sell so many disparate products. A PRODUCT oriented sales force which already has problems selling too many SKUs now has to learn to sell a SERVICE in a very competitive market. Will Check Point become the next Cisco? Losing focus and having a sales force running in circles selling routers, home firewalls, security services….blenders and who knows what.
MDM management and logging is the core of Check Point that has kept mama and its babies fed through the Ciscos, Junipers, and Palo Altos. WHY? Centralized management reduces TCO…less administrators, higher reliability. Easy focused sell. History has shown us that any attempt to create ancillary products that are not focused on MDM management and logging result in tepid sales at best. WHY? Sales force now has to learn a new product and compete against the best the marketplace has to offer. Very difficult pushing a rope uphill based on price alone.
BEGIN Peanut Gallery:
Any new product/service should be focused on MDM. That way MDM pulls that sale with the TCO pitch vs. pushing a rope uphill in a crowded market as a standalone product with the only competitive selling point is price. Cisco was giving ASA away for free and they still have problems displacing the mighty MDM.
In addition: Quit canabalizing your OPSEC partners. They usually have awesome products and Check Point is usually 5 versions behind the technology curve. Rather encourage them to integrate into MDM with an API. Or purchase the best OPSEC partner that integrates cleanly into MDM. Or license a portion of the technology (perhaps the low end) and brand it as CP and integrate into MDM.
Example: IPS is the perfect example.  IPS is your only product that integrates (weakly) into MDM. Now look at all your products and SERVICES and do the same. MDM will pull IPS product sales so a sales force does not have to push it uphill based on price alone.
END Peanut Gallery:
OK so we both started in the security business at the same time and you wound up with personal jets and bodyguards and I’m driving a 2006 Scion xA (most reliable car in USA) living in a 900sf condo. I’m just hoping you read this and take the best parts to make another billion so I can move up to a 2009 Scion xD (most reliable car in USA) and maybe buy this 1200sf condo I’ve been looking at.

MDM Architecture – Part II

posted Oct 28, 2011 5:41 AM by Michael Endrizzi                                                                                               [

So I think I’ve established you can’t design your MDS architecture just thinking about 1 parameter…security…or your job security. You have to think about the whole environment over the whole MDM life cycle. Ease of administration is a huge criteria here.
Global sharing is a great concept, but it does have pitfalls (can’t rename globals,  can’t delete some global objects (policies), can’t delete global objects in local policies, hard to locate global objects, can’t nest global policies, global database locked out by single administrator, few global tools like global provisioning/monitoring/strict IPS/logging), subnet hell. Until MDM is implemented in a true SQL database I don’t feel that they will fix many of these problems.
SOOOOO Until that day happens we have to live with what we have…which is still the greatest enterprise management tool on the planet.
1) Common Policies: Think about it from a policy point of view. Is there a group of local policies that are 80/90% similar? Group similar local policies into one domain and use either the INSTALLON field or separate saved policies to install on groups of firewalls. Now these are a bit tricky because you need procedures to make sure the a rule is installed on the correct firewall. If you have maverick gunslinging administrators, you may want to think twice about doing this.
2) Mavericks: Do you have administrative mavericks that need their own playpen? Do you do a lot of M&A’s with their own administrators? Separate them into their own set of isolated domain(s).
3) Decentralized administration: Are you like Berkshire Hathaway Company – the 8th largest company in the world but only 50’ish employees at headquarters in Nebraska (look at their web page once). I can assure you they have decentralized administration. A decentralized firm should have (maybe even multiple MDM environments) a global policy (or set of global policies) for each of the subfirms, then delegate administration to sets of domains for that firm. Note that this will require strict procedures so that sub-firms respect the integrity of other sub-firms global policies if they have write access.
4) Global VPNs: If you want to setup Check Point VPNs between Domains NOT under that same global policy…Research global VPNs before you do.
5) Ignore IPS: IPS really doesn’t impact MDM architecture. MDM can’t enforce IPS on domains, only offer IPS profiles to use if they want.
6) Separation of Duties: Remember that if you have separation of duties between security admins that create policy and NOC/Help Desk that REASSIGNS and installs policy, that the NOC will then have to have superadmin permissions and access to all global domains. If this is a problem, you may consider having multiple MDMs.
7) Security Zones: Let’s say you want to map PCI security zones into global policy or isolated Domains. The problem here is that it may be at cross-purpose with the other policies that are business based so now you are crossing administrative boundaries. Make sure you factor in the administrative overhead to do that. Look at who the administrators are and what the business rules are and if it crosses administrative boundaries. There are better ways to make the auditors happy then only doing this security mapping in MDM (like Tufin).
8) Business Risk: Remember that thing called ‘business’ that is suppose to function and write your paychecks. Well you can divy it up into microscopic chucks (i.e. firewall per server) so that malware outbreaks are contained as well as your paycheck. OR you can take an economic view of the organization and break it into economic risk categories. So if your Product A production line is 80% of revenue, then firewall it off and let Product B,C,D,…..E etc function behind a second set of firewalls. Marketing and sales….ugh put them together. I mean put operational staff together because they only suck money and do not produce it. I think you get the picture, think like a business economist, not a security geek.
9) Geographical Separation: I was wrong here. I couldn’t think of any reason but just heard a great one. Let’s say you have huge policies or lots of logging from Botswana. Do you need to backlog that over your WAN. Or doing installs with long delays and you get ERROR-Install Failed..but in reality it installed just timeouts? Locate a secondary HA mgt near where the firewalls are located so that policy pushes and logging is targeted to the shortest network connectivity to these sites.
10) The Risk – Let’s just say your patching, IPS, AV is stellar. But you have a slew of systems
that can’t be patched (3rd party systems, legacy systems, appliances that don’t autopatch). So place the well maintained systems behind one set of firewalls, and the unpatched ones behind a second set of firewalls. So I know the security geeks will scream “But what if unmaintained system A infects unmaintained system B??? Hereasy!!!” My comeback is “What if the rules get so complex and by the 10th firewall administrator (everyone else quit because it became so complex they couldn’t administrate it) your rules wind up being Swiss cheese. NOW try and remove the firewalls  and consolidate. Good luck, going backwards is a nightmare. If you DO have malware problems, then isolate those systems based on actual risk instead of potential risk which you can’t reverse engineer.
11) The Submarine – Think about how submarines are built. In water tight containers based on function. Now wouldn’t it be cool to build a container for each crew member!!! Think about all the doors you’d have to go through and the cost involved. YIKES! Instead why don’t we build containers based on function. The control room, the engine room, the torpedo room all have their own. Of course the crew all dies together in one container. Similarly break your business into business functions. As the ship is sinking which functions can you afford to kill off. Obviously most the staff goes first, marketing, suppliers, etc. But the execs, manufacturing and accounting have to stay alive at all costs. “The Golden Rule” I guess.
12) MDS separation – This category is somewhat overlapping on the others, and I got this from a CheckPoint SE. There is a customer that has XX MDS servers in XX countries. The point being you can have unique global instances for each of your “zones” no matter how you divy them up.
1) Never Never Never use global objects in local rules.
2) Create a written naming standard for global objects and enforce it. You cannot rename global objects.
3) As long as you are creating a naming standard, might want to create a object color standard too. And then
    use it to color your security zones. You can change colors, but not names.
3) Global policies are forever. You can’t delete them. Use gently
4) Warning. As of R75 there are no more MDS containers so the max domain count is about 250.
32-bit SPLAT just can’t handle more. Once GAIA and its 1200 commands come out you will probably be able to support more. Another area where the theory runs into the brick wall of reality.
5) There is this thing called budget. Believe it or not businesses can only afford so many $150,000/year CCSA’s. So your manager says “We have budget for 3 firewalls geeks”. That’s the upper limit. From that number determine how many domains you can humanely administrate with spending weekends and nights debugging. Perhaps assign 40 to each of the 3 and then 120 is your upper limit. The point is pick an upper limit of, people, domains, rules. If you go above that then go back to the well and justify based on metrics your keep on activity to date on existing infrastructure.
I’m sure I’ll think of more, but that’s it for now. This should get you started…

MDM Architecture

posted Oct 26, 2011 10:21 AM by Michael Endrizzi                                                                                               [

So these are the MDM ultimate questions I’m not sure anyone can answer (art vs science) but I will attempt to anyways.
1) When do I switch from SmartCenter to MDS?
2) How many global policies should I have in MDS?
3) How many domains should I assign to a Global Policy?
4) How many firewalls should I assign to a domain?
5) When do I use global objects vs local objects?
6) When should I start using high availability?
7) When should I start using a multi-domain log server vs. DMS logging?
OK, its like asking “What color should my next car be?”. Not everyone is going to get the same answer.
These are the factors that should guide your decisions:
1) ADMINISTRATION ADMINISTRATION ADMINISTRATION: First and foremost MDM was built to ease administration. ISPs had a tough time keeping
all their managed mom-and-pop rulebases separated in a 24×7 shop with lots of administrators (both ISP and client’s)
over a period of years so administrators came and went. If you are designing an MDS architecture the COMPLICATES administration, then you are going in the wrong direction. HINT: always design for simplified administration.
2) Security: MDM has facilities for global vs. local separation. If you are going at it from a programming point of view, you might be disappointed because its sure not C++, but more like Pascal – version 1 of global vs local scoping rules. So if you are a security nut and you decide to separate everything into 300 CMAs…well on paper it may look good to auditors but look out operationally: backups won’t work, policies will take forever to install, upgrades will kill you if you use global objects wrong, etc. HINT: Only separate if you have too, and use multiple policies in a DMS too. Use global objects with caution.
3) Licenses: Check Point’s achilles heel is licensing. Just assumed its screwed up and go from there. The more complex your environment the more licensing will kill you financially, morally, emotionally, etc. Its broken and everytime they try and fix it, the problem gets more complicated and worse. : HINT: Keep it simple
4) Updates: MDS has a facility for updating your environment on a global basis. Right now its Version 1 so don’t bet your kid’s schoolbooks on it but it is a good first step. HINT: don’t use for now
5) Operations: Remember that the ivory tower architecture was not designed by the people doing 3am assign/installs. MDS will break if you get too complex. Example: Backups. Each domain replicates in entirety, binaries and everything. So 250 domains is 250 sets of duplicate binaries. Backups will fail when you have too many domains.
And when an upgrade goes bad good luck finding which of the 250 domains blew up on you. And then trying to find all locations of that global object you used: CROSS_CMA searches work better but still not quite there. HINT: keep it simple
6) Templates: What information do you want to share across firewalls? MDS has several facilities for sharing
global information, but you need to first figure out what you would like to share…instead of manually replicate over 300 domains. HINT: keep it simple
7) Changing environment: Does your company buy and sell other firms? Do you have to deal with sub-divisions
that are hostile but will be friendly when you fire all their administrators? HINT: Isolate trouble makers.
8) Upgrades: How will you upgrade your MDS environment? In pieces or all at one time. HINT: keep it simple.
9) IPS: remember that MDS cannot!!! enforce IPS policy on domains. It can only offer global templates to use
10) You will be replaced; Remember that the MDM has a life cycle outside of your employment contract. Someday you will move on and this monster you are creating has to be managed by someone else. Now you could ensure job security and also create a testament to your supreme god-like existence on this planet by making the MDM so complex that only you understand it and your organization would be foolish to fire such a god as yourself. But if you value your good name, I suggest you think about TCO and MDM life-cycle after your greatness departs from this earth.
11) Logging: Note that when you have a 1:1 firewall:domain, you can only monitor 1 log file per tracker session. If you merge multiple firewalls into tracker you can watch multiple logs in 1 SmartTracker sessions. If your environment has a lot of problems in 1 set of firewalls, you might want to merge them into 1 domain so all the logs are sent to the same log server.
12) Network subnets: Remember the more firewalls you create the more subnets you create the more traceroutes you have to do to debug. And then when the subnets change, you have to change the network objects in SmartDashboard. So increasing the complexity in the name of security also increases network operational security.
Now remember I’m an MDM addict. It is by far the BEST enterprise management solution in the market. But it has its limits.
So my #1 advice is KEEP IT SIMPLE!!!! And ask yourself, are the security nutjubs going to be there at 3am when backups fail?
Mikes Motto: A security system that can’t be managed is inherently insecure!!!!
So going to MDS will ease you administrative headaches if you have 200 firewalls all in 1 SmartCenter server. But 200 domains hosted in MDS will NOT NOT NOT ease your total overall administrative headaches (although on paper it seems more secure because of the separation), because you will kill yourself trying to manage the beast…and will probably make mistakes that make the entire architecture insecure because its soooo complex you can’t analyze it.
So its easy for anyone to sit and cry WOLF!!, but coming up with solutions separates the men/women from the boys/girls.
Hold on as I check my manhood in future editions

Tufin SecureTrack Review

posted Oct 26, 2011 9:49 AM by Michael Endrizzi                                                                                               [

I reviewed the basic functions of Tufin SecureTrack. Great tool for:
1) Keeping and comparing versions of policies
2) Keeping auditors away with best practice, PCI, management reports
3) Optimize rulebases with rule hit counts
4) Also can be used for rule reviews
Also the last section has video on how to setup SecureTrack for this purpose. The
book is somewhat vague on setup in MDS environment.
OK it’s rough and wish I had more time to do these but hope you get something out of it.
Helen's Loom

"Peculiar travel suggestions are dancing lessons from God." - Kurt Vonnegut

Life Stories from Dreez

These are stories from my travels. Generally I like to write stories about local people that I meet and also brag about living the retirement dream with my #1 wife Gaby. She is also my only wife.