Place bananas, peanut butter, milk, and sugar in a blender. Process until smooth.
Pour banana peanut butter mixture into a large mixing bowl. Add cream and vanilla. Stir until combined. Place bowl in the fridge or freezer for about 20 mins.*
Churn ice cream according to manufacturer's directions. During the last five minutes of mixing, add peanut butter cups.
Serve immediately for soft serve ice cream. For a firmer consistency, freeze ice cream for several hours.