Random Reboots on HP Hardware - Error ID 57

We have a server (BL460c) that we upgraded the drivers and firmware on and now it is rebooting every couple hours. The Event Log has a warning with an ID of 57. The description is "NetFN 0x36, command 0x2 timed out" This is solved by downgrading both iLO drivers.

1. HP ProLiant Integrated Lights-Out Management Interface Driver for Windows Server 2003/2008 x64 Editions to version 1.13.0.0
2. HP ProLiant iLO 2 Management Controller Driver for Windows Server 2003 x64 Editions to version 1.8.0.0

Chuck Swindoll on lust and controlling the mind

I really appreciate Chuck Swindoll for several reasons. First and foremost is the fact that he preaches like it is, no matter how hard the subject and that he practices what he preaches. That is probably what I want most to be remembered as. Here is a lesson that he recently presented at DTS on lust and purity in the mind. A very important yet terribly difficult lesson that you won't hear even in most churches since it is so convicting. If this message does not effect you or you think that it does not apply to you then you really need to get help; it will make your life so much better.

http://www.dts.edu/media/play/?MediaItemID=7fd7e223-479d-4183-ac94-ec74649a6d7a

If you are looking for help feel free to contact them at 469-252-5200.

ARP Poisoning in a Production Environment

Here is a sanatized email the I sent last night that has a very interesting problem that we ran into at work. Never ran into ARP Poisoning before...
-------------------------

Sent: Thursday, April 16, 2009 12:21 AM
To: XXXXX XXXXX
Subject: ARP Poisoning

Ok, so here is 12 hours of work boiled down into a couple sentences of “what happened” …

Basic Topography: 10.70.0.1(hop1) > 10.50.0.1(2) > 10.50.0.9(3) > Internet(4) > Destination(5)

MAC Address for 10.50.0.1 = XX:XX:XX:XX:b4:23

Scenario: We isolated the issue (mainly by completely replacing 10.70.0.1 (ISA 2006) & 10.50.0.1 (Core Router) to no avail) so that we knew that traffic was going from 10.70.0.1 out to the internet, hitting the destination and generating response traffic. This response traffic made it to the firewall but died before reaching back to the 10.70.0.1 ISA server. This leaves 10.50.0.1 and 10.50.0.9 as the only possible culprits for the missing traffic. After replacing 10.50.0.1 we discovered that the traffic still exhibited the same behavior and we realized that the chances of 2 Routers both being bad was really remote so we focused on the firewall. Taking a packet trace with the network up and another when the network was dead we found a very subtle difference in the packets. While the network was operating normally the packets were flowing from the firewall to the core router using the level 2 routing address’ of:

Src: XX:XX:XX:XX:ea:b0 - dst: XX:XX:XX:XX:b4:23
When the network was broken the level 2 flow of inbound packets was like:
Src: XX:XX:XX:XX:ea:b0 - dst: 00:0c:29:a9:d2:25

So what we have at this point is ARP Poisoning where another machine on the 10.50.x.x is impersonating 10.50.0.1 which is the Core Router; the result of this is that all traffic coming inbound from the internet (hop3 > hop2) was getting redirected to the mystery machine (hop3 > hop_blackhole) with the mac of 00:0c:29:a9:d2:25. Going from switch to switch we tracked the mac to MachineNameX. After unplugging the machine from the network and clearing the ARP cache on the firewall traffic immediately started working normally and the network is happy. Check out the cool wireshark screenshot attached… now you know what ARP poisoning looks like.

Bottom line: Wireshark is awesome, the 10.50.x.x switches have a command “show bridge address-table” which shows you the mac address’ that are associated with each port on the switch, 2 heads and 4 eyes are better than 1 head and 2 eyes… and sleep is overrated.. :)

Backing up a MAC Server automatically

Scenario: You run a Windows environment and all of the sudden have a couple Apple XServes that get added and then need backed up.

You are the responder: I will be backing this XServe up to a Windows File Server where the files will then be picked up to go to tape.
Step 1: Create the appropriate file share on the Windows box and and assign the proper permissions to it. Give it a unique AD Service Account since the password is stored in plain text in the backup scripts.
Step 2: Create a folder on the Apple that will be the mount point for your smbfs to the Windows File Server
Step 3: Create a batch file on the Apple that looks like this:

#!/bin/bash
cd /Volume/FoldertoBackup
/sbin/mount_smbfs //ADuserName:ADPassword@WinFileServer/ShareToSaveFilesTo /MyAppleMountPoint
tar -cf backup-servername-`date '+%d-%B-%Y`.tar /Volume/FoldertoBackup
cp backup-servername-`date ' +%f-%B-%Y`.tar /MyAppleMountPoint
umount /MyAppleMountPoint

Step 4: Test the script and see if it works when launched manually. If you cut and paste it from here you will need to use dos2unix to fix the hidden EOL characters since they are different from standard unix EOL.

Step 5: Automate with Crontab
a. 00 04 * * 02 /var/backups/backup_script.sh
This will execute the backup_script.sh file every Tuesday at 4am.

Step 6: You're done... sort of... Now you need to create a script to clean up the backup files after X amount of days on the Apple so you don't lose too much disk space. You can also edit the tar command to do differential backups if you so choose...

Cisco ASA 5510 - Disable Specific Alerts

We get a ton of false positives on one specific alert on our ASA. The alert is a "No Translation Group Found" that happens when somebody brings a laptop from home and attempts to create a connection before getting the proper IP settings from DHCP. The alert looks like this:

<163>%ASA-3-305005: No translation group found for tcp src (InterfaceName):(IPAddress/Port) dst (InterfaceName):(IPAddress/Port)

We wanted to disable just this one alert so that we do not get so many false positives. After trying several things to no avail I finally opened a support case with Cisco and got a quick and easy fix. To acomplish this all that you need to do is type in:

ciscoasa(config)# no logging message 305005

and to re-enable it all you need to do is type:

ciscoasa(config)# logging message 305005

That was easy!