You have probably missed the warning signs that the cat has had enough. Some cats, even if they enjoy being petted, also tire of it very quickly. Flattened ears and a slight twitching of the tail can signal that they have had their fill. If you have such a cat, try breaking off from stroking it every few seconds and allow it to move away if it wants to.
Cats can get quite frustrated if they are bored or if they are stressed, perhaps because they have just seen a rival cat through the window. Under such circumstances, the slightest movement, perhaps just a speck of dust caught in a shaft of light, can set them off. Regular play sessions can help dissipate all that energy and relieve any stress.