Jesus certainly is the Prince of Peace as you point out. That peace is peace between God and man. Previously man was at enmity with God, but now through Jesus Christ, God and man may be reconciled.
For if while we were enemies we were reconciled to God by the death of his Son, much more, now that we are reconciled, shall we be saved by his life. Romans 5:10 ESV
That is not to suggest that Jesus desires Christians to be violent. On the contrary, we are live lives of love, just as Christ loved us and gave Himself for us.
Therefore be imitators of God, as beloved children. 2 And walk in love, as Christ loved us and gave himself up for us, a fragrant offering and sacrifice to God. Ephesians 5:1-2 ESV
So, the question then becomes, "In what sense did Jesus bring a sword?" Here it is instructive to look at how Jesus taught the world would relate to believers in Jesus:
“If the world hates you, know that it has hated me before it hated you. 19 If you were of the world, the world would love you as its own; but because you are not of the world, but I chose you out of the world, therefore the world hates you. 20 Remember the word that I said to you: ‘A servant is not greater than his master.’ If they persecuted me, they will also persecute you. If they kept my word, they will also keep yours. 21 But all these things they will do to you on account of my name, because they do not know him who sent me. John 15:18-21 ESV
All of the apostles, except for John, died a martyrs death. At its birth, Christianity was enveloped by rampant persecution--not Christians taking the sword against others, but others taking the sword against Christians. It is remarkable, indeed, that with such an intensely persecuted and martyred group of believers in this new revelation of Jesus as the Messiah could have survived.
All throughout history, in fact, true believers in Jesus have been severely persecuted. There are quite a few countries today in which Christians are martyred just as frequently as they were 2,000 years ago. The Soviet Union intensely persecuted Christians as well, prior to its collapse.
In the future, we can expect the same thing. True believers in Jesus will be victims of the sword through persecution and martyrdom. True believers are always on the sharp end of the sword in matters of persecution, because true Christianity spreads not through conquest, but through the drawing of a person's heart and soul.