I have the problem too.
I only add about 1k point, 1k line .
When they all show on the screen, the map is slow to drag/zoom.
I build my application using trunk on silverlight3. And I use accordingLayerPanel to manage the layers/features. It is a so powerful control that most of things can be done with it.
But the performance bothered me.
Now I want to try DE2011 to see the performance , but there is no sample to show the scenario that how custom tiles server is used.
I doubt if the time to struggle against DE is shorter than manage the features with primitive Microsoft MapLayer.