FTP using username password at command prompt

OK this one really pisses me off because I searched high and low for something like this. And then I spent too much time trying to find different ways of doing this without resorting to SCP priv/public key distribution. I’m sure its been around since the dawn of time and I’m the last person on earth to figure it out.

GAIA includes a ‘curl’ application.

UGH!

Download:curl -u ftpuser:ftppass -o localfilename  ftp://ftp_server/public_html/xss.php
Upload:curl -u ftpuser:ftppass -T myfile.txt      ftp://ftp.testserver.com

http://www.thegeekstuff.com/2012/04/curl-examples/

Anyways this will simplify the distribution of patches in a large enterprise environment when using scripts.

NOTE: Only exists on gateways and NOT MDS

FYI curl SCP is NOT supported in GAIA (but it is in Unix)

[Expert@NOCgw1:0]# curl -u user:password scp://1.1.1.1/file.txt
curl: (1) libcurl was built without LIBSSH2, scp: not supported!

Geez I hate being the last one to the party.

dreez

Smartlog sucking space patch

R75.46 turns out SmartLog (My favorite product) was sucking 2-3 times the space of the actual logs for its index files.

WHY?

Because before each log entry they pre-pended the full path to the log file name. The longer the path name, the more space for the index.

Do you see where I’m coming from Vern?

As support for the patch.

Log on!

dreez

Finally found appliance model number description

Randomly came across this one while trying to fix a bug where monitord was consuming 100% CPU. (still searching)

You’ll love this.

We have 200+ firewalls and all I want to know is what the friggin model number is and how many we have. How hard can that be??? Well, when you have 200 firewalls the ‘ol spreadsheet tends to get out of date.  And MDM and SmartMonitor and Smart Provision are of little use. and the ‘ol dmidecode | fgrep -i product is a bunch of crap.

Well check this out.

/etc/appliance_config.xml

Database of stats from the WEBUI

The search is over! Now if they only included this in an MDS level SmartMonitor so we could manage the fleet better.

ADDENDUM: From Heather Lewis, this is even better!

dbget :appliance_configuration:value:/model/name. This is even better!

dreez

Routing and multiple sync interfaces

If you are using dynamic routing you probably know that clustering + routing have a symbiotic relationship..they need each other but don’t like each other.  I’m not sure how many lunches that the cluster and routing folks had with each other but it shows in the implementation. I have written about this several times.

So here is a way to enhance the reliability of clustering + routing to bypass a Linux kernel bug. If you have a single cable as a sync link directly between cluster members and not going through a switch, then routing+clustering will not work well if the sync interface goes down.

In order to avoid buying switches for all your sync needs, set the mgt interfaces as a 2nd sync interface. This will ensure that the routing daemon continues to participate in routing (very important!!) in case something happens to the sync interface. (There is a patch that is floating around because routing previously did not like the 2nd sync interface. Check with support).

Rather Safe than Sorry

Route On!

dreez

CoreXL: Tips and Tricks

So I’ve was working with a bunch of SE’s in Chicago and one of them says to me “I can tell if someone has no experience with setting up a system by typing sim affinity -l. That will tell me if CoreXL has been balanced or not, then the person either knows their stuff or not”.

I’ll admit, until this past month I was clueless. I took a survey at the last CP get together in Minneapolis here and turns out so are most my peers. In addition I work with some awesome CP Diamond engineers and they never mentioned it. So I didn’t feel quite as bad, but it was time to get busy.

I’ve been working on this big CoreXL project and hopefully can share with you someday. CoreXL is one of those hidden Check Point gems that Marketing should be shouting from the rooftops about in order to compete with the ASIC competitors. But as usual CoreXL has really bad and spotty documentation so not even the techies get it. The best bet is to read the SKs, but its like blind people feeling an elephant trying to guess what it is. Well over the past month I think I am finally starting to get the big picture and hopefully can share with you in the future. It is cool.

Anyways here is a cool tip I learned the other day. CoreXL balances 4 functions amongst processors:

1.  Interface processing
2. Firewall instances
3. Firewall helper processes
4. Linux processes

If you have a ‘busy’ box, you can tell if network I/O is one of the issues impacting performance. I had this box with default config and decided to SCP a 3gig file between the SYNC interfaces. This forced the CPU %SI (software interrupts)to 80% and the system started rebalancing.

If you see this:

you may have an issue. By default interfaces are set to ALL, which means all processors can be used when handling processing for that interface. If the system is NOT busy, CPU0 will typically handle all the interface interrupts. If the box is ‘busy’ (CPU and %SI is high > 50% approx) and is having problems processing network I/O, then the rebalancing starts and the interfaces are assigned specific CPUs. In addition, if this re-balancing continues to happen with the interfaces THEN you really may be having network I/O problems. Check your ifconfig for errors if packets are dropping.

You can also tell if the interfaces have been rebalancing since boot by looking at the interrupt handling. The /proc/interrupts tells you if an interface interrupts has been handled by multiple multiple CPUs. If so then the box has been busy and trying to keep up and has auto rebalanced by moving interrupt handling amongst several CPUs.

Hopefully I can share more with you in the future!

Have a balanced day!

dreez

SmartLog Architecture and Tuning

So I spent this week helping Dudi create a new SK that was badly need for SmartLog. Most sites just fire and forget when starting to use SmartLog, but as you will soon find out on huge sites (we get 26 gig/day), you start running out of disk space.  So when you go digging into the smartlog_settings.txt, there are some really confusing parameters you will find that may/not help you tune disk usage. The existing SK’s are not that great, but  Dudi did a great job explaining all the parameters related to tuning disk usage.

One thing I noticed as I was editing his text was that no where in the documentation is the overall SmartLog architecture described. Maybe this is pretty obvious to y’all, but it took me a while to figure this out.  On an MDS, there are logs at the MDS level (mostly MDS mgt info only) and the DMS level. A peer Smartlog process is then run for each MDS and DMS.  The SmartLog process builds and maintains index’s for the individual MDS/DMS log files. NOTE: These are NOT firewall log files, only management information about who logged in and modifications to the rulebase.

Then there are TWO different SmartLogs you can run.  The one at the MDS level which will search through your entire MDM environment (here at the MDS level you can see the individual index’s)

At the DMS level, you can see only information related to your DMS. The searches will be faster and more detailed usage information is returned.

You can see thisMDS/DMS split when you do a ‘ps -ef | fgrep -i smartlog’. All the SmartLog processes will dump out. Then try doing a ‘mdsenv; cd $SMARTLOGDIR; pwd’ and then  ‘mdsenv <DOMAINNAME>; cd $SMARTLOGDIR; pwd’. Compare the directory names.

I think I have convinced Dudi to merge them both into one where you will be able to see usage information at the global AND the DMS level. This would make agreat competitor to crappy SEIM tools like RSA Envision, Arcsite and cool but hard to use SNMP NetFlow tools.

Second thing. Smartlog performs 4 functions:

1) Indexes new logs
2) Indexes old logs – Upon startup looks backwards
3) Services SmartLog GUI
4) Deletes old index files by looking at its peer log files

So let’s say you delete all your log  files and restart SmartLog and put a GUI on it and start doing searches. SmartLog will be a REALLY busy little beaver because it has to do all 3 functions at once. SOOOOO when you purge your logs, you may want to look at sk73361  or dink with the num_days_restriction_fetch_all_integraded (3) to limit how far back SL does history indexes.

Once again, SmartLog is the coolest tool in Check Point’s suite. For the life of me I’m not sure why they aren’t pushing it as the messiah of security/network tools. Then again, I don’t have a personal jet like Gill….

SmartLog – Index’ing your life

What the hell is a “Required Interface”???

I know you and the rest of the world knows this but I just figured this out after an interaction with the SK people.

I was having flapping problems and was reading sk44268  and was pretty broad. I also googled “required interfaces” and came up blank.

So they fixed sk44268, referred me to this great SK on internal cluster functions ClusterXL ATRG sk93306 (tons of details) and I thought I would
summarize for the common peasants:

Required interfaces is the summary of

1) Clustered interfaces (VIP)
2) Sync interfaces

This CLI shows 3 Required Interfaces: 1 sync and 2 clustered

This (different config)GUI shows 4 “Required Interfaces”, 3 clustered, 1 sync.

Summary:

Better: Cluster requires X number of interfaces to be available and sending receiving CCP packets.

1) If Cluster does NOT see CCP packets on X required interfaces, it fails over.

2) If Cluster members have different number of X required interfaces it fails over thinking one went down.

3) VLANs are counted in this required count …… forgot how?? I think the first and last on the clustered interface

4) Required interfaces CAN be modified on the fly by modifying the topology and pushing policy….but sometimes it doesn’t work and you have to reboot

5) Required interfaces are listed as “cluster interfaces” and “sync” interfaces in the gui (above)

6) At boot time, these cluster/sync interfaces are counted and set to Required Interfaces. But they can be modified in SmartDashboard and topology when you push policy

Short and sweet …. I think?? I’m the first to described this!!

Wow, living large.

dreez