Yes, your problem is that List is reusing views for each row. So say that you can see 5 items in your list, but your ListAdapter has 15 things. Android will create 5 + 1 instances of your row view instead of 15. One for each row in the + 1 list, when you can see half above and below. When a row moves out of the visible region, List will recycle this instance of the view for another row instead of creating a new one. If you do not use reset all user interface components every time you get artifacts from other lines. You have to make sure that every time you bind your data from objects in the list of arrays to the view, you set each field every time.
For a better description of this, see
http://www.youtube.com/watch?v=N6YdwzAvwOA&feature=related