Should tracking in schools be eliminated?

Coming from a student of a school with a set curriculum with no honors classes or AP classes or Special Education classes, I think tracking presents itself as a two edged sword. Personally, I wish I had some sort of track or "education pathway" as some parts of my curriculum are too easy for me, I feel, while other parts are adequate for my level while others are a bit challenging. Tracking can be beneficial in that it gives students something to strive for. It gives the student in the normal track to strive for the honors class, and for students in the honors track to keep on in the honors class. It gives the students a system to work from the ground up, and therefore, proves beneficial in boosting confidence and morale.
Tracking, just as it can heighten confidence, can lower it too, and I see where the flip side in benefits from its elimination comes into play. Students in some cases may be wrongfully placed in a track not suited for them, and it can be detrimental to their academics. There are also stigmas that may inadvertently become attached to these students, which will sour their outlook on their studies and may suppress them from the desire to get out of that track.
I think it all comes down to how each school chooses to do its tracking. As long as it is a properly used system, the students can reap many benefits.