Bug 16560 - Random reset to new battery condition
: Random reset to new battery condition
Status: RESOLVED FIXED
Product: SB Radio
Classification: Unclassified
Component: Battery
: Include FW version in comment
: PC Windows XP
: P1 normal (vote)
: ---
Assigned To: Vahid Fereydouny
: factory_mp
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2010-09-17 13:28 UTC by Ryan
Modified: 2010-12-20 11:34 UTC (History)
5 users (show)

See Also:
Category: ---


Attachments
Battery data log for test unit with msp code version 1351 (47.78 KB, application/octet-stream)
2010-09-17 13:28 UTC, Ryan
Details
excell file of battery charge and discharge data (587.20 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2010-11-15 13:15 UTC, Ryan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan 2010-09-17 13:28:42 UTC
Created attachment 6977 [details]
Battery data log for test unit with msp code version 1351

FW9089 and msp version 1351

With no external input the battery charge state was reset to the "as new" state
charge value 800. The unit was setup and playing overnight with battery
installed and AC power attached. At some point between 2:46 AM and 2:51 AM the
charge value was reset and the chargestate changed from idle (2) to charging
(8). Here is a segment of the battery data log.

 The full log file attached includes other events from performing the battery
testing. 

Fri Sep 17 02:41:41 PDT 2010
voltage: 13655 charge: 1954 temperature: 913 chargestate: 2
Fri Sep 17 02:46:41 PDT 2010
voltage: 13655 charge: 1954 temperature: 913 chargestate: 2
Fri Sep 17 02:51:41 PDT 2010
voltage: 14260 charge: 817 temperature: 916 chargestate: 8
Fri Sep 17 02:56:42 PDT 2010
voltage: 14663 charge: 844 temperature: 927 chargestate: 8
Fri Sep 17 03:01:42 PDT 2010
voltage: 14966 charge: 871 temperature: 952 chargestate: 8
Fri Sep 17 03:06:43 PDT 2010
voltage: 15167 charge: 899 temperature: 997 chargestate: 8
Fri Sep 17 03:11:43 PDT 2010
voltage: 14724 charge: 2000 temperature: 1078 chargestate: 2
Fri Sep 17 03:16:44 PDT 2010
voltage: 14582 charge: 2000 temperature: 1102 chargestate: 2
Comment 1 Vahid Fereydouny 2010-09-28 12:51:59 UTC
The condition that checks for the fully charged battery is as follows:

#define dTdt                    (15)   /* 15/32 = 0.5C / minute means charged. */
#define BATTERY_HOLDOFF_TIME    (SAFETY_TIMER_TIMEOUT >> 5)             /* SAFETY_TIMER_TIMEOUT / 32 = 20 mins - charge for at least this period */

holdoff_timer = BATTERY_HOLDOFF_TIME;
deltaT = current_temp - last_temperature;


if (battery_voltage > battery_stop_charging_voltage
    || (deltaT > dTdt && holdoff_timer == 0)
    || (safety_timer == 0))
{
       stop_charging(battery_cap /* fully charged */, CHARGER_COMPLETE);
       break;
}

In the logs the temperature went from 997 to 1078, which is more than 15.
It seems that if the battery is charging and after 20 minutes(holdoff timer) the temperature jumps more than 15 then we stop charging the battery.
I will check with Alan.
Comment 2 Vahid Fereydouny 2010-09-28 13:04:49 UTC
There are two bugs associated with this one:
1. Is the duplicate of: Bug 16580 -  Battery sets to charge state to soon
2. The other portion is related to resetting to 800 that I believe is fixed.
I mark this bug as resolved for the 2nd bug. We track the first bug in 16580.
Comment 3 Ryan 2010-10-05 11:38:27 UTC
This is still happening with the latest 1406 MSP code  I have also noticed that the reset to new battery charge value will occur during a discharge cycle which really messes up the charge value counter
Comment 4 Vahid Fereydouny 2010-10-18 14:17:19 UTC
Based on the code here are the conditions that we set the state of charge ( soc ) to 800:
1. If the battery is inserted for the first time
2. In charger_init
3. There is an i2c command interface that could be used to set the soc: I2C_BATT_STATE_OF_CHARGE
Comment 5 Vahid Fereydouny 2010-10-18 15:21:14 UTC
In the logs that is attached we go to 800 charge during the charge process(We have a duration that we start discharging the battery and then start charging it until it is fully charged). I do not see this causing any problem for the battery because every time we go back to 2000 charge after around 1/2 hour.
Still looking at the code to understand why this happens.
Comment 6 Vahid Fereydouny 2010-10-19 00:59:54 UTC
Based on my understanding of the source code by now the only condition that could cause the battery's state of charge to be reset to 800 is a power off/reset condition. I do not see any other condition that could cause the battery_soc to be reset to 800.
Comment 7 Vahid Fereydouny 2010-10-19 12:44:17 UTC
Ryan has logs that at run-time we reset to 800:
voltage: 13755 charge: 1951 temperature: 938 chargestate: 2
Fri Oct 15 19:24:26 PDT 2010
voltage: 13776 charge: 1951 temperature: 940 chargestate: 2
Fri Oct 15 19:24:36 PDT 2010
voltage: 13776 charge: 1951 temperature: 938 chargestate: 2
Fri Oct 15 19:24:47 PDT 2010
voltage: 13776 charge: 1951 temperature: 938 chargestate: 2
Fri Oct 15 19:24:57 PDT 2010
voltage: 13755 charge: 1951 temperature: 940 chargestate: 2
Fri Oct 15 19:25:07 PDT 2010
voltage: 14119 charge: 800 temperature: 913 chargestate: 8

This means we went from the Idle state to charging state.
This could happen for the following reason:
Invalid battery detection indication. If there is a glitch that adc read of the battery voltage indicates the battery is not present then we go to idle state and we go back to charging state immediately. In that case the charge is reset to 800 and MSP430 starts charging the battery.
Still reading the code to see if there are other conditions that could cause this.
Comment 8 Ryan 2010-10-26 09:33:02 UTC
preliminary testing with MSP 1418 is looking good. Will update this bug after further testing
Comment 9 Mickey Gee 2010-11-02 14:13:15 UTC
Ryan says fixed, but Vahid wants to test it some more.
Comment 10 Ryan 2010-11-15 13:15:50 UTC
Created attachment 7036 [details]
excell file of battery charge and discharge data
Comment 11 Vahid Fereydouny 2010-11-15 16:42:25 UTC
The battery state of charge gets reset to 800 which is the default state of charge in three cases:
1. No Battery is present
2. The charger is reset.
3. An I2C command is issues to set the value to default.

Alan: Do you know if anyone issuing i2c commands to MSP430 to set the battery_soc?
Comment 12 Alan Young 2010-11-15 22:05:39 UTC
(In reply to comment #11)

> Alan: Do you know if anyone issuing i2c commands to MSP430 to set the
> battery_soc?

No.
Comment 13 Ryan 2010-11-22 14:29:35 UTC
I thought I marked this as reopened last week but I guess not
Comment 14 Vahid Fereydouny 2010-11-25 13:44:56 UTC
The problem seems to be related to extension cables that are used to have both serial output and external battery. The extension cable seem to add a lot of noise to ADC reads which causes bad Temperature/Voltage/AC reads.
Have been running 3 Baby's with external battery directly connected and they are functioning perfectly fine.
More testing is in progress.
Comment 15 Vahid Fereydouny 2010-11-29 16:13:53 UTC
Ryan Ran extensive testing on many Baby's(I think 10) from Wed afternoon to Monday morning. This problem has happened in total of 8 times. Investigating.
Comment 16 SVN Bot 2010-12-01 01:33:53 UTC
 == Auto-comment from SVN commit #7586 to the player repo by ayoung ==
 == http://svn.slimdevices.com/player?view=revision&revision=7586 ==

bug 16560: Random reset to new battery condition 
Do not use low battery temperature as a signal that the battery has been removed.
Other tests should have already determined that.
Comment 17 SVN Bot 2010-12-01 01:43:51 UTC
 == Auto-comment from SVN commit #7587 to the player repo by ayoung ==
 == http://svn.slimdevices.com/player?view=revision&revision=7587 ==

bug 16560: Random reset to new battery condition 
Version msp430_1435.txt
Keep last 5 power-mode readings and only trust changes when they are consistent.
Comment 18 Vahid Fereydouny 2010-12-09 17:39:49 UTC
Added more information to capturing the logs:
Date: Wed Dec  8 20:09:59 PST 2010                    MSP430 clock: Thu Dec  9 04:10:00 2010  0.000000 seconds              power_mode: 7 voltage: 13372 charge: 1847 temperature: 821 chargestate: 2
Date: Wed Dec  8 20:10:01 PST 2010                    MSP430 clock: Thu Dec  9 04:10:01 2010  0.000000 seconds              power_mode: 7 voltage: 13372 charge: 1847 temperature: 821 chargestate: 2
Date: Wed Dec  8 20:10:02 PST 2010                    MSP430 clock: Thu Dec  9 04:10:02 2010  0.000000 seconds              power_mode: 7 voltage: 13352 charge: 1847 temperature: 821 chargestate: 2
Date: Wed Dec  8 20:10:03 PST 2010                    MSP430 clock: Thu Dec  9 04:10:04 2010  0.000000 seconds              power_mode: 7 voltage: 13372 charge: 1847 temperature: 821 chargestate: 2
Date: Wed Dec  8 20:10:05 PST 2010                    MSP430 clock: Thu Dec  9 04:10:05 2010  0.000000 seconds              power_mode: 7 voltage: 13372 charge: 1847 temperature: 821 chargestate: 2
Date: Wed Dec  8 20:10:06 PST 2010                    MSP430 clock: Thu Dec  9 04:10:06 2010  0.000000 seconds              power_mode: 7 voltage: 13372 charge: 800 temperature: 821 chargestate: 1
Date: Wed Dec  8 20:10:07 PST 2010                    MSP430 clock: Thu Jan  1 00:00:01 1970  0.000000 seconds               power_mode: 7 voltage: 13372 charge: 800 temperature: 821 chargestate: 2
Date: Wed Dec  8 20:10:09 PST 2010                    MSP430 clock: Thu Jan  1 00:00:02 1970  0.000000 seconds               power_mode: 7 voltage: 13352 charge: 800 temperature: 817 chargestate: 8
Date: Wed Dec  8 20:10:10 PST 2010                    MSP430 clock: Thu Jan  1 00:00:04 1970  0.000000 seconds              power_mode: 7 voltage: 13594 charge: 800 temperature: 814 chargestate: 8
Date: Wed Dec  8 20:10:12 PST 2010                    MSP430 clock: Thu Jan  1 00:00:05 1970  0.000000 seconds              power_mode: 7 voltage: 13614 charge: 800 temperature: 812 chargestate: 8
Date: Wed Dec  8 20:10:13 PST 2010                    MSP430 clock: Thu Jan  1 00:00:06 1970  0.000000 seconds              power_mode: 7 voltage: 13634 charge: 800 temperature: 812 chargestate: 8
Date: Wed Dec  8 20:10:14 PST 2010                    MSP430 clock: Thu Jan  1 00:00:08 1970  0.000000 seconds              power_mode: 7 voltage: 13634 charge: 800 temperature: 810 chargestate: 8
Date: Wed Dec  8 20:10:16 PST 2010                    MSP430 clock: Thu Jan  1 00:00:09 1970  0.000000 seconds              power_mode: 7 voltage: 13634 charge: 800 temperature: 812 chargestate: 8
Date: Wed Dec  8 20:10:17 PST 2010                    MSP430 clock: Thu Jan  1 00:00:11 1970  0.000000 seconds              power_mode: 7 voltage: 13655 charge: 800 temperature: 810 chargestate: 8
Date: Wed Dec  8 20:10:19 PST 2010                    MSP430 clock: Thu Jan  1 00:00:12 1970  0.000000 seconds               power_mode: 7 voltage: 13634 charge: 800 temperature: 812 chargestate: 8


It seems that the MSP430 got reset. Based on MSP430 manual:
An instruction fetch from the address range 0x0000 − 0x01FF
will reset the device.
Also based on msp430.out.map the stack address range is:
.stack     0    000002f0    00000110     UNINITIALIZED
                  000002f0    00000002     rts430.lib : boot.obj (.stack)
                  000002f2    0000010e     --HOLE--
Comment 19 Vahid Fereydouny 2010-12-09 21:34:20 UTC
More testing shows that any of the following commands could reset the MSP430:
* ( volatile int * ) 0x120 = 0; // bad write to watchdog timer.
(*(void(**)(void))(0xfffe))();
((void (*)())0x170)();
Comment 20 Vahid Fereydouny 2010-12-09 23:19:18 UTC
Implemented an I2C command that does the following commands:
* ( u16 * ) 0x2c6 = 0;
* ( u16 * ) 0x2ca = 0;

After issuing this for a while the MSP430 gets reset. This should be similar to having a stack overflow:

This is the map file:
.bss       0    00000200    000000ce     UNINITIALIZED
                  00000200    00000048     adc.obj (.bss)
                  00000248    00000022     charger.obj (.bss)
                  0000026a    00000018     i2c.obj (.bss)
                  00000282    00000016     power.obj (.bss)
                  00000298    00000013     rtc.obj (.bss)
                  000002ab    00000001     main.obj (.bss)
                  000002ac    0000000c     ir_sensor.obj (.bss)
                  000002b8    0000000a     als.obj (.bss)
                  000002c2    00000004     rotary_encoder.obj (.bss)
                  000002c6    00000004     rts430.lib : _lock.obj (.bss)
                  000002ca    00000004                : exit.obj (.bss)

.stack     0    000002f0    00000110     UNINITIALIZED
                  000002f0    00000002     rts430.lib : boot.obj (.stack)
                  000002f2    0000010e     --HOLE--

.text      0    0000e000    00001c24
                  0000e000    000003f4     charger.obj (.text:charger_set)
Comment 21 Vahid Fereydouny 2010-12-20 10:41:06 UTC
It seems that the bug was related to running the battery/charger task inside the Timer interrupt. I made changes to run the battery/charger task outside the timer interrupt.
The whole weekend testing with more than 15 Baby's(12 Ryans and 3 from my Babys) shows no occurrence of this bug.
Comment 22 Ryan 2010-12-20 11:32:02 UTC
Testing with r9262 with msp version 1463 over the weekend on 12 units with batteries. The reset to new battery condition has not been seen. In addition other battery behaviors were checked and no issues were seen. 

Regression testing of the power state conditions and RTC backup is ongoing at this time