Try out removing the afterscroll slowdown code. Then a scroll will only stop with a touch or IR activity, or when a list end is hit.

This might make it so there is less confusion about whether a press will stop a scroll. With forever scrolling, there is always a stop action requirement, unless the list has visually bumped into the end of the list and stopped.