My initial assessment:
Here is the scenario that is most likely to cause this problem:
 1. Remove power from Radio.
 2. Wait for more than 1/2 hour
 3. This causes the battery on the Radio to provide charge to the MSP. One of the functionality of the MSP is Real Time Clock(RTC).
 3. Power on the Radio
 4. During boot-up the i2c read of the MSP returns error
 5. The MSP is reprogrammed. During the reprogramming all the interrupts are disabled.
 6. While the step 4 & 5 are running the wireless is detected and is set-up.
 7. Eventually the MSP is programmed, but the wireless does not function properly.

There are two problems that has to be addressed:
1. Why does the read of the MSP information through i2c bus returns error after power up.
2. Why is the sequencing of the MSP reprogramming and wireless setup are not synchronized properly.

The wlan gets started as part of the rcS script and after running the inetd.

# Start wlan
/etc/init.d/wlan start

The code to detect and initialize the msp is in msp430_i2c_probe function, which gets called as part of msp430 probe(baby-none-linux-gnueabi/linux-imx25-2.6.26+7.5+svnr7914-r7/linux-2.6.26/drivers/mxc/baby/msp430/ directory and in msp430_programmer.c file).

If I power off my Baby and wait for 30 minutes and then power it on I get the following sequence:

[    3.980230] UBIFS: mounted UBI device 0, volume 2, name "ubifs"
[    3.986216] UBIFS: file system size:   9418752 bytes (9198 KiB, 8 MiB, 73 LEBs)
[    3.993583] UBIFS: journal size:       1032193 bytes (1008 KiB, 0 MiB, 6 LEBs)
[    4.000858] UBIFS: media format:       w4/r0 (latest is w4/r0)
[    4.006725] UBIFS: default compressor: lzo
[    4.010869] UBIFS: reserved for root:  444870 bytes (434 KiB)
[   13.630221] i2c-adapter i2c-1: ACK not received
[   13.634923] Couldn't read msp ID register, resetting mcu
[   15.141132] i2c-adapter i2c-1: ACK not received
[   15.145843] Couldn't read msp ID register, resetting mcu
[   16.460908] msp430: upgrade (-1 to 1316)
[   27.166119] AR6000 Reg Code = 0x40000060
[   27.862362] msp430: upgrade complete (1316)
[   33.716510] channel hint set to 2462
[   33.752198] AR6000 connected event on freq 2462 with bssid 00:09:5b:6a:bf:70  listenInterval=100, beaconInterval = 100, beaconIeLen = 0 assocReqLen=35 assocRespLen =22

It is clear that the MSP430 is in the state that does not respond to i2c requests. We reset the device and then the MSP430 is reprogrammed(resetting causes the i2c interface to work properly).

[    1.833408] msp430 1-0010: rtc core: registered msp430 as rtc0
[    1.841119] Requesting msp430 firmware msp430-0005.txt
[    1.847631] firmware: requesting msp430-0005.txt

when the probe is called for msp430 the following functions get called:
msp430_i2c_probe (...)
{
.
.
.
data->rtc = rtc_device_register("msp430", &client->dev,
                                  &msp430_rtc_ops, THIS_MODULE);

.
.
.
        /* initialize the programmer */
        msp430_programmer_probe(client);

        /* request firmware upgrade (if needed) */
        msp430_programmer_upgrade(client);

        /* register power off handler */
        pm_power_off = msp430_power_off;

}

msp430_programmer_upgrade calls the request_firmware_nowait with the msp430_programmer_firmware as the callback.

The request_firmware_nowait functions is defined in the 
linux-2.6.26/drivers/base/firmware_class.c and is defined as:
 *      Asynchronous variant of request_firmware() for contexts where
 *      it is not possible to sleep.

The major problem I see with this approach is that we cannot be sure when the device initialization/upgrade is done.
Have to get a better understanding of this approach.
The sequence of initializing the MSP430 should be deterministic in the sense that we should know when the detection/initialization is completed before accessing the device.

In case of the log attached to this bug the probe of the msp430 i2c is done at:
[    1.841116] Requesting msp430 firmware msp430-0005.txt
[    1.847623] firmware: requesting msp430-0005.txt

The msp430_programmer_firmware is called way later on at:
[   15.461185] i2c-adapter i2c-1: ACK not received
[   15.465889] Couldn't read msp ID register, resetting mcu

The problem with this approach is that we disable all the interrupts while the programming of the MSP430 is in progress. In this case even though the MSP430 eventually is programmed properly, but it caused the problem in initialization of the wireless driver ( Atheros)
root: wlan: starting
Setting eth1 macaddr: 00:04:20:26:8E:EC
Load Board Data from /lib/atheros/calData_ar6102_15dBm.bin
Updating MAC address
BMI Write Memory (address: 0x502400, filename: /lib/atheros/athwlan.bin)
[   15.461185] i2c-adapter i2c-1: ACK not received
[   15.465889] Couldn't read msp ID register, resetting mcu
BMI Write Memory (address: 0x52d8bc, filename: /lib/atheros/data.patch.hw2_0.bin
)
BMI Write Memory (address: 0x500418, value: 0x52d8bc)
BMI Bit-Wise (OR) modify Register (address: 0x500410, orig:0x0, new: 0x1,  mask:
0x1)
BMI Done
[   16.760874] msp430: upgrade (-1 to 1316)
[   27.117572] AR6000 Reg Code = 0x40000060
bmiloader: eth1: Input/output error
[   27.233644] debug_hdr_ptr: 0x5018f0
[   28.162431] msp430:

I waited 24h before powering up again a Baby here and msp430 did _not_ get reprogrammed. It started up just fine.

Could we have a hardware issue with the super-cap or something else causing this msp430 reprogramming issue on some devices but not others?

The MSP430 firmware is on the flash, so the data will not be lost by the power
failure. Because the i2c transaction issued to the MSP430 fails, we reprogram
the MSP430.
There are two source of preserving power on Baby: The capacitor that is
supposed to provide power to MSP430 in case of power failures and the Battery
that is used to maintain the power for the Baby if the power is lost. 
This Bug can happen with or without the Battery. If the Baby is running with no
Battery and then power goes off the capacitor tries to provide power to MSP430
in order of preserving the hardware clock. This hardware clock will be used for
the Alarms in case of failure to connect to network in the next boot up.
The current theory is that the MSP430 goes to an state that does not respond
properly to i2c transaction.
This is the first issue.
The second issue is the way we are managing/initializing the MSP430.

baby_msp430/src/power.c contains the implementation of the low power mode. enter_low_power_mode gets called once the power is removed. This function does all the cleaning including i2c:

void i2c_close(void)
{       
        i2c_state = I2C_IDLE;
        UCB0I2CIE = 0; // disable all i2c interrupts.
        UCB0STAT  = 0; // clear any pending flags.
        BIS(UCB0CTL1, UCSWRST); // PUT I2C into reset state.
}
and once the power is on, it sets up the i2c to init state by calling i2c_init:
{       
        volatile int j = 0;
        int i;
        INTERRUPT_MX25_CLEAR;
        i2c_state = I2C_IDLE;
        // Reference section 17.3.1 of msp430f2xx user guide
        // 1. make sure i2c is in reset state
        BIS(UCB0CTL1, UCSWRST);
.
.
.

This problem will be fixed by my changes. I am creating a new bug to trace the other problem, which is related to MSP430 not responding to i2c commands and forcing initialization.

This bug is finxed and the necessary changes are checked into 7.5.1-rtm branch.

Hi Vahid, it looks like you should have the ability to mark bugs as 'resolved'.  When you check in a fix, please go ahead and mark it resolved so that I can know to test the fix.

I'll do it this time.

 == Auto-comment from SVN commit #9004 to the jive repo by fmueller ==
 == http://svn.slimdevices.com/jive?view=revision&revision=9004 ==

Bug: 16279 
Description: Make sure the change how msp430 is initialized is picked up when building locally. Else you can end up with a Baby where msp430 is not initialized / programmed at all.

All the related changes are checked into the 7.5 branch. Also I increased the PR number to make sure that they will get built for local builds.