Four wives did Avraham have, Sarah, Hagar, Rivka, and Ketura. Hagar and Ketura, the concubines, gave birth to Ishmael, and to Zimran, Yokshan, Medan, and Midian. Sarah gave birth to Yitzchak. Avraham gave everything he had to Yitzchak. But Yitzchak did not survive the Akeida, leaving behind his wife, Rivka, who was barren. Through Yibum, she became a wife of Avraham. Then, the womb of Rivka was opened and she gave birth to Ya'akov. Finally, she gave birth to Esav. Avraham gave gifts to his sons Ishmael, Zimran, Yokshan, Medan, and Midian, and sent them to the East, and he gave Seir to Esav. To Ya'akov he gave everything that he had previously given to Yitzchak. He sent Yaakov to Lavan, and Ya'akov returned from there to Eretz Yisrael, with his four wives.
Somewhere along the line, probably just before his sending of Ya'akov to Lavan, and his very reason for this sending, G-d spoke to Avraham: "Know with certainty that your offspring shall be aliens in a land not their own, they will serve them, and they will oppress them four hundred years. But also the nation that they shall serve, I shall judge, and afterwards they shall leave with great wealth. As for you - you shall come to your ancestors in peace; you shall be buried in a good old age. And the fourth generation shall return here, for the iniquity of the Amorite shall not yet be full until then."
Ya'akov, then, is a son of Avraham, but is seen as a son of Yitzchak, the first-born son of his widow Rivka. This explains expressions like
The names of the final two sons of Ketura (Bereshit 25:2),